Message301697
Please don't reopen an issue if it was closed by a core developer.
It's not clear to me what exactly do you want to change in the implementation or documentation.
A sequence of three periods has a special meaning as an ellipsis literal.
is literally describes how the ELLIPSIS token is identified in Parser/tokenizer.c (see PyToken_ThreeChars in that file) So a sequence of three periods is identified as an ellipsis literal which is an expression in Python.
Do you want to change tokenize.tokenize() (it's in Lib/tokenize.py) so it will return
TokenInfo(type=52 (ELLIPSIS), ...)
instead of
TokenInfo(type=53 (OP), ...)
? Note that ELLIPSIS has been added to tokenize.EXACT_TOKEN_TYPES in issue 24622. To me, since it has been added to tokenize.EXACT_TOKEN_TYPES there is no need to special case ELLIPSIS in Lib/tokenize.py.
Or do you want to clarify
To simplify token stream handling, all Operators and Delimiters tokens are returned using the generic token.OP token type.
at https://docs.python.org/3/library/tokenize.html? |
|
| Date |
User |
Action |
Args |
| 2017-09-08 13:31:17 | berker.peksag | set | recipients:
+ berker.peksag, Aivar.Annamaa |
| 2017-09-08 13:31:17 | berker.peksag | set | messageid: <1504877477.8.0.0459917942403.issue31394@psf.upfronthosting.co.za> |
| 2017-09-08 13:31:17 | berker.peksag | link | issue31394 messages |
| 2017-09-08 13:31:17 | berker.peksag | create | |
|