Token

Token produced by scanner.

32 bytes on 64-bit.

Members

Properties

idString
string idString [@property getter]

Get string representation of the token ID.

Variables

directive
DirectiveType directive;

Type of directive for directiveToken.

encoding
Encoding encoding;

Encoding, if this is a stream start token.

endMark
Mark endMark;

End position of the token in file/stream.

id
TokenID id;

Token type.

startMark
Mark startMark;

Start position of the token in file/stream.

style
ScalarStyle style;

Style of scalar token, if this is a scalar token.

value
char[] value;

Value of the token, if any.

valueDivider
uint valueDivider;

Used to split value into 2 substrings for tokens that need 2 values (tagToken)

Meta