The following tokens are the separators:
( ) { } [ ] ; , .
(* *) \( ` @
The second line contains the separators that are introduced by
the XL programming language. The separator @ has also
been introduced by the Java Language Specification, Third Edition.
The following tokens are the operators:
= > < ! ~ ? : == <= >= != && || ++ -- + - * / & | ^ % << >> >>> += -= *= /= &= |= ^= %= <<= >>= >>>= :+= :-= :*= :/= :&= :|= :^= := ** <=> :: ==> ==>> ::> -> <- +> <+ --> <--
The last two lines contain the operators that are introduced by the XL programming language.
For lexical translations, the longest possible match is always used.
Thus, the input characters i<-m and
i-->=0 are tokenized as
i, <-, m,
and i, -->, =,
0, respectively, whereas they would be tokenized as
i, <, -,
m,
and i, --, >=,
0, respectively, in the Java programming language
(because of the absence of the tokens <-
and -->).
Of course, these different tokenizations lead to a different
interpretation of the input. Thus, in some cases additional
white space between tokens is necessary in order to yield the
intended tokenization.