Skip to content

bpo-33064: lib2to3: support trailing comma after **kwargs #6096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ambv merged 4 commits into from
Mar 13, 2018
Merged

bpo-33064: lib2to3: support trailing comma after **kwargs #6096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b393698
lib2to3: support trailing comma after **kwargs
ambv Mar 13, 2018
59fefbe
Update NEWS
ambv Mar 13, 2018
cb4fc74
Update tests
ambv Mar 13, 2018
072da3a
Handle *args, too
ambv Mar 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 4 additions & 23 deletions Lib/lib2to3/Grammar.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,7 @@
# Grammar for 2to3. This grammar supports Python 2.x and 3.x.

# Note: Changing the grammar specified in this file will most likely
# require corresponding changes in the parser module
# (../Modules/parsermodule.c). If you can't make the changes to
# that module yourself, please co-ordinate the required changes
# with someone who can; ask around on python-dev for help. Fred
# Drake <fdrake@acm.org> will probably be listening there.

# NOTE WELL: You should also follow all the steps listed in PEP 306,
# "How to Change Python's Grammar"

# Commands for Kees Blom's railroad program
#diagram:token NAME
#diagram:token NUMBER
#diagram:token STRING
#diagram:token NEWLINE
#diagram:token ENDMARKER
#diagram:token INDENT
#diagram:output\input python.bla
#diagram:token DEDENT
#diagram:output\textwidth 20.04cm\oddsidemargin 0.0cm\evensidemargin 0.0cm
#diagram:rules
# NOTE WELL: You should also follow all the steps listed at
# https://devguide.python.org/grammar/

# Start symbols for the grammar:
# file_input is a module or sequence of commands read from an input file;
Expand All @@ -38,13 +19,13 @@ async_funcdef: 'async' funcdef
funcdef: 'def' NAME parameters ['->' test] ':' suite
parameters: '(' [typedargslist] ')'
typedargslist: ((tfpdef ['=' test] ',')*
('*' [tname] (',' tname ['=' test])* [',' '**' tname] | '**' tname)
('*' [tname] (',' tname ['=' test])* [',' ['**' tname [',']]] | '**' tname [','])
| tfpdef ['=' test] (',' tfpdef ['=' test])* [','])

@serhiy-storchaka serhiy-storchaka Mar 13, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it be just tfpdef ['=' test] [',']? Of course may be there are reasons of writing the rule as it is written due to the parser limitations.

@ambv ambv Mar 13, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe but I want to keep this as close to Grammar/Grammar and this is how it's voiced there.

tname: NAME [':' test]
tfpdef: tname | '(' tfplist ')'
tfplist: tfpdef (',' tfpdef)* [',']
varargslist: ((vfpdef ['=' test] ',')*
('*' [vname] (',' vname ['=' test])* [',' '**' vname] | '**' vname)
('*' [vname] (',' vname ['=' test])* [',' ['**' vname [',']]] | '**' vname [','])
| vfpdef ['=' test] (',' vfpdef ['=' test])* [','])
vname: NAME
vfpdef: vname | '(' vfplist ')'
Expand Down
36 changes: 33 additions & 3 deletions Lib/lib2to3/tests/test_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
# Testing imports
from . import support
from .support import driver, driver_no_print_statement
from test.support import verbose

# Python imports
import difflib
Expand All @@ -22,7 +21,6 @@
import sys
import tempfile
import unittest
import warnings

# Local imports
from lib2to3.pgen2 import driver as pgen2_driver
Expand Down Expand Up @@ -305,6 +303,38 @@ def test_8(self):
*g:6, h:7, i=8, j:9=10, **k:11) -> 12: pass"""
self.validate(s)

def test_9(self):

@serhiy-storchaka serhiy-storchaka Mar 13, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to add tests for signatures like:

(a,)
(*args,)
(a=1,)

and few combinations if they have a separate path in the grammar.

@ambv ambv Mar 13, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOL, that's one subtle way to signal "your patch doesn't handle the vararg case" ;-) Alright, I added *args support, too, and added the relevant tests.

s = """def f(
a: str,
b: int,
*,
c: bool = False,
**kwargs,
) -> None:
call(c=c, **kwargs,)"""
self.validate(s)

def test_10(self):
s = """def f(
a: str,
) -> None:
call(a,)"""
self.validate(s)

def test_11(self):
s = """def f(
a: str = '',
) -> None:
call(a=a,)"""
self.validate(s)

def test_12(self):
s = """def f(
*args: str,
) -> None:
call(*args,)"""
self.validate(s)


# Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.test_var_annot
class TestVarAnnotations(GrammarTest):
Expand Down Expand Up @@ -407,7 +437,7 @@ def test_new_syntax(self):
self.validate("class B(t, *args): pass")
self.validate("class B(t, **kwargs): pass")
self.validate("class B(t, *args, **kwargs): pass")
self.validate("class B(t, y=9, *args, **kwargs): pass")
self.validate("class B(t, y=9, *args, **kwargs,): pass")


class TestParserIdempotency(support.TestCase):
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2018-03-12-19-58-25.bpo-33064.LO2KIY.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
lib2to3 now properly supports trailing commas after ``*args`` and
``**kwargs`` in function signatures.