More on StreamTokenizer
StreamTokenizer flags are set by the methods
- eolIsSignificant(boolean) - whether EOL is returned as a token or treated as white space
- slashStarComments(boolean) - whether to recognize C style comments
- slashSlashComments(boolean) - whether to recognize C++ style comments
- lowerCaseMode(boolean) - whether to convert all WORD tokens to lower case
Other parsing properties
- parseNumbers() - if false, parses only WORDS - default is to parse both WORD and NUMBER tokens.
- whitespaceChars(int,int) - specifies to use all chars in the range between the two ints as white space.
- Other methods can specify word chars.