Class BaseTokenizer

  • Direct Known Subclasses:
    Expander, Tokenizer

    public class BaseTokenizer
    extends java.lang.Object
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected char ch  
      protected short column  
      protected static char EOT  
      protected int index  
      protected short line  
      protected Token text  
    • Constructor Summary

      Constructors 
      Constructor Description
      BaseTokenizer​(java.lang.CharSequence text)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected boolean eot()  
      protected char escape()  
      protected void find​(char target, char deeper)  
      protected char getch()  
      protected char getch​(boolean peek)  
      protected boolean isBlank​(char ch)  
      protected char peek()  
      protected void skipQuote()  
      protected void skipSpace()  
      protected void skipSpace​(boolean skipNewLines)  
      Token text()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • text

        protected final Token text
      • line

        protected short line
      • column

        protected short column
      • ch

        protected char ch
      • index

        protected int index
    • Constructor Detail

      • BaseTokenizer

        public BaseTokenizer​(java.lang.CharSequence text)
    • Method Detail

      • text

        public Token text()
      • find

        protected void find​(char target,
                            char deeper)
      • escape

        protected char escape()
      • skipQuote

        protected void skipQuote()
      • skipSpace

        protected void skipSpace()
      • skipSpace

        protected void skipSpace​(boolean skipNewLines)
      • isBlank

        protected boolean isBlank​(char ch)
      • eot

        protected boolean eot()
      • getch

        protected char getch()
      • peek

        protected char peek()
      • getch

        protected char getch​(boolean peek)