Python3.x tokenize
Python 3 系 のtokenizeに関する言語リファレンス情報をまとめたカテゴリです。
constant の一覧
__all__
__author__
__builtins__
__cached__
__credits__
__doc__
__file__
__loader__
__name__
__package__
__spec__
AMPER
AMPEREQUAL
ASYNC
AT
ATEQUAL
AWAIT
Binnumber
blank_re
BOM_UTF8
CIRCUMFLEX
CIRCUMFLEXEQUAL
COLON
COLONEQUAL
COMMA
Comment
COMMENT
ContStr
cookie_re
Decnumber
DEDENT
DOT
Double
Double3
DOUBLESLASH
DOUBLESLASHEQUAL
DOUBLESTAR
DOUBLESTAREQUAL
ELLIPSIS
ENCODING
ENDMARKER
endpats
EQEQUAL
EQUAL
ERRORTOKEN
EXACT_TOKEN_TYPES
EXCLAMATION
Expfloat
Exponent
Floatnumber
FSTRING_END
FSTRING_MIDDLE
FSTRING_START
Funny
GREATER
GREATEREQUAL
Hexnumber
Ignore
Imagnumber
INDENT
Intnumber
LBRACE
LEFTSHIFT
LEFTSHIFTEQUAL
LESS
LESSEQUAL
LPAR
LSQB
MINEQUAL
MINUS
N_TOKENS
Name
NAME
NEWLINE
NL
NOTEQUAL
NT_OFFSET
Number
NUMBER
Octnumber
OP
PERCENT
PERCENTEQUAL
PlainToken
PLUS
PLUSEQUAL
Pointfloat
PseudoExtras
PseudoToken
RARROW
RBRACE
RIGHTSHIFT
RIGHTSHIFTEQUAL
RPAR
RSQB
SEMI
Single
Single3
single_quoted
SLASH
SLASHEQUAL
SOFT_KEYWORD
Special
STAR
STAREQUAL
String
STRING
StringPrefix
tabsize
TILDE
tok_name
Token
Triple
triple_quoted
TYPE_COMMENT
TYPE_IGNORE
VBAR
VBAREQUAL
Whitespace