de.grogra.grogra
Class LSYTokenizer

java.lang.Object
  de.grogra.grammar.Tokenizer
      de.grogra.grogra.LSYTokenizer

All Implemented Interfaces:

antlr.TokenStream, LSYTokenTypes

public final class LSYTokenizer
extends Tokenizer
implements LSYTokenTypes

Field Summary

Fields inherited from class de.grogra.grammar.Tokenizer

CREATE_TOKEN_LOCATOR, createTokenLocator, ENABLE_TYPE_SUFFIX, enableTypeSuffix, EOL_IS_SIGNIFICANT, eolIsSignificant, EVALUATE_NUMBERS, evaluateNumbers, FLOAT_IS_DEFAULT, floatIsDefault, I18N, MINUS_IS_SIGN, minusIsSign, unicode, UNICODE_ESCAPES

Fields inherited from interface de.grogra.grogra.LSYTokenTypes

ABSTRACT_, ADD, ADD_ASSIGN, AND, AND_ASSIGN, ANNOTATION, ANY, APPLICATION_CONDITION, ARGLIST, ARRAY_DECLARATOR, ARRAY_INIT, ARRAY_ITERATOR, ARROW, ASSERT, ASSIGN, BOOLEAN_, BOOLEAN_LITERAL, BREAK, BYTE_, CAND, CASE, CATCH, CHAR_, CHAR_LITERAL, CLASS, CLASS_LITERAL, CMP, COLON, COM, COMMA, COMPILATION_UNIT, COMPOUND_PATTERN, CONST_, CONSTRUCTOR, CONTEXT, CONTINUE, COR, DEC, DECLARING_TYPE, DEFAULT, DEFERRED_ADD, DEFERRED_AND, DEFERRED_ASSIGN, DEFERRED_DIV, DEFERRED_MUL, DEFERRED_OR, DEFERRED_POW, DEFERRED_RATE_ASSIGN, DEFERRED_REM, DEFERRED_SHL, DEFERRED_SHR, DEFERRED_SUB, DEFERRED_USHR, DEFERRED_XOR, DIMLIST, DIV, DIV_ASSIGN, DO, DOT, DOUBLE_, DOUBLE_ARROW_RULE, DOUBLE_LITERAL, ELIST, EMPTY, ENHANCED_FOR, EOF, EQUALS, EXEC_RULE, EXPR, EXTENDS, FINAL_, FINALLY, FLOAT_, FLOAT_LITERAL, FOLDING, FOR, GE, GT, GUARD, IDENT, IF, IMPLEMENTS, IMPORT_ON_DEMAND, IN, INC, INDEX_OP, INSTANCE_INIT, INSTANCEOF, INSTANTIATOR, INT_, INT_LITERAL, INTERFACE, INVALID_EXPR, INVOKE_OP, ITERATING_, JREG_IDENT, LABEL, LABELED_STATEMENT, LATE_MATCH, LBRACK, LE, LEFT_ARROW, LEFT_RIGHT_ARROW, LINE, LITERAL_binomial, LITERAL_carbon, LITERAL_color, LITERAL_diameter, LITERAL_distribution, LITERAL_function, LITERAL_generation, LITERAL_if, LITERAL_index, LITERAL_length, LITERAL_local, LITERAL_n_value, LITERAL_negbinomial, LITERAL_normal, LITERAL_order, LITERAL_poisson, LITERAL_q_value, LITERAL_register, LITERAL_sum, LITERAL_sumd, LITERAL_sump, LITERAL_table, LITERAL_uniform, LITERAL_v_value, LITERAL_xcoordinate, LITERAL_ycoordinate, LITERAL_zcoordinate, LONG_, LONG_ARROW, LONG_LEFT_ARROW, LONG_LEFT_RIGHT_ARROW, LONG_LITERAL, LPAREN, LT, MARKER, METHOD, METHOD_CALL, METHOD_PATTERN, MINIMAL, MODIFIERS, MODULE, MUL, MUL_ASSIGN, NAME_PATTERN, NATIVE_, NEG, NEW, NODE, NODES, NORMAL, NOT, NOT_EQUALS, NULL_LITERAL, NULL_TREE_LOOKAHEAD, OPTIONAL_MATCH, OR, OR_ASSIGN, PACKAGE, PARAMETER_DEF, PARAMETERIZED_PATTERN, PARAMETERS, PATTERN_WITH_BLOCK, PLUS_ARROW, PLUS_LEFT_ARROW, PLUS_LEFT_RIGHT_ARROW, PLUS_LINE, POS, POST_DEC, POST_INC, POW, POW_ASSIGN, PRIVATE_, PRODUCE, PROTECTED_, PUBLIC_, QUALIFIED_NEW, QUALIFIED_SUPER, QUERY, QUERY_EXPR, QUESTION, QUOTE, RANGE, RANGE_EXACTLY, RANGE_MIN, RBRACK, REM, REM_ASSIGN, RETURN, ROOT, RPAREN, RULE, RULE_BLOCK, SEMI, SEPARATE, SHELL_BLOCK, SHL, SHL_ASSIGN, SHORT_, SHR, SHR_ASSIGN, SINGLE_ELEMENT, SINGLE_MATCH, SINGLE_OPTIONAL_MATCH, SINGLE_STATIC_IMPORT, SINGLE_TYPE_IMPORT, SLASH_ARROW, SLASH_LEFT_ARROW, SLASH_LEFT_RIGHT_ARROW, SLASH_LINE, SLIST, STATIC_, STATIC_IMPORT_ON_DEMAND, STATIC_INIT, STATIC_MEMBER_CLASSES, STRICT_, STRING_LITERAL, SUB, SUB_ASSIGN, SUPER, SWITCH, SWITCH_GROUP, SYNCHRONIZED_, THIS, THROW, THROWS, TRANSIENT_, TRAVERSAL, TREE, TRY, TYPE_PATTERN, TYPECAST, TYPECHECK, UNARY_PREFIX, USHR, USHR_ASSIGN, VARARGS_, VARIABLE_DEF, VOID_, VOLATILE_, WHILE, WITH, WRAPPED_TYPE_PATTERN, X_LEFT_RIGHT_ARROW, XOR, XOR_ASSIGN, YIELD

Constructor Summary

LSYTokenizer()

Method Summary

protected Token	convert(Token t)
protected boolean	isIdentifierPart(char c)

Methods inherited from class de.grogra.grammar.Tokenizer

addToken, addToken, addToken, addToken, addToken, addToken, consume, copyTokenTablesFrom, getFloat, getInput, getInt, getNumberToken, getToken, isIdentifierStart, isWhitespace, nextToken, reset, setInput, setSource, setSource, ungetToken

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LSYTokenizer

public LSYTokenizer()

Method Detail

convert

protected Token convert(Token t)

Overrides:

convert in class Tokenizer

isIdentifierPart

protected boolean isIdentifierPart(char c)

Overrides:

isIdentifierPart in class Tokenizer