Message 351970 - Python tracker

➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author	navytux
Recipients	arigo, beazley, benjamin.peterson, brian.curtin, dabeaz, flox, jyasskin, kristjan.jonsson, loewis, navytux, ned.deily, paul.moore, pitrou, python-dev, r.david.murray, rhettinger, ronaldoussoren, sbt, scoder, shihao, techtonik, terry.reedy, tim.golden, tim.peters, torsten, vstinner, ysj.ray
Date	2019-09-11.16:39:46
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<1568219986.67.0.423356853296.issue38106@roundup.psfhosted.org>
In-reply-to

Content
Thanks for feedback. Yes, since for Python-level lock, PyThread_release_lock() is called with GIL held: https://github.com/python/cpython/blob/v2.7.16-129-g58d61efd4cd/Modules/threadmodule.c#L69-L82 the GIL effectively serves as the synchronization device in between T2 releasing the lock, and T1 proceeding after second sema.acquire() when it gets to execute python-level code with `del sema`. However a) there is no sign that this aspect - that release must be called under GIL - is being explicitly relied upon by PyThread_release_lock() code, and b) e.g. _testcapimodule.c already has a test which calls PyThread_release_lock() with GIL released: https://github.com/python/cpython/blob/v2.7.16-129-g58d61efd4cd/Modules/_testcapimodule.c#L1972-L2053 https://github.com/python/cpython/blob/v2.7.16-129-g58d61efd4cd/Modules/_testcapimodule.c#L1998-L2002 Thus, I believe, there is a bug in PyThread_release_lock() and we were just lucky not to hit it due to GIL and Python-level usage. For the reference, I indeed started to observe the problem when I moved locks and other code that implement channels in Pygolang from Python to C level: https://lab.nexedi.com/kirr/pygolang/commit/69db91bf https://lab.nexedi.com/kirr/pygolang/commit/3b241983?expand_all_diffs=1

Thanks for feedback. Yes, since for Python-level lock, PyThread_release_lock() is called with GIL held:

https://github.com/python/cpython/blob/v2.7.16-129-g58d61efd4cd/Modules/threadmodule.c#L69-L82

the GIL effectively serves as the synchronization device in between T2
releasing the lock, and T1 proceeding after second sema.acquire() when it gets to
execute python-level code with `del sema`.

However

a) there is no sign that this aspect - that release must be called under GIL -
   is being explicitly relied upon by PyThread_release_lock() code, and

b) e.g. _testcapimodule.c already has a test which calls
   PyThread_release_lock() with GIL released:

   https://github.com/python/cpython/blob/v2.7.16-129-g58d61efd4cd/Modules/_testcapimodule.c#L1972-L2053
   https://github.com/python/cpython/blob/v2.7.16-129-g58d61efd4cd/Modules/_testcapimodule.c#L1998-L2002

Thus, I believe, there is a bug in PyThread_release_lock() and we were just
lucky not to hit it due to GIL and Python-level usage.

For the reference, I indeed started to observe the problem when I moved locks
and other code that implement channels in Pygolang from Python to C level:

https://lab.nexedi.com/kirr/pygolang/commit/69db91bf
https://lab.nexedi.com/kirr/pygolang/commit/3b241983?expand_all_diffs=1

History
Date	User	Action	Args
2019-09-11 16:39:46	navytux	set	recipients: + navytux, tim.peters, loewis, arigo, rhettinger, terry.reedy, beazley, paul.moore, shihao, ronaldoussoren, pitrou, scoder, kristjan.jonsson, vstinner, techtonik, jyasskin, tim.golden, benjamin.peterson, ned.deily, r.david.murray, brian.curtin, flox, dabeaz, torsten, ysj.ray, python-dev, sbt
2019-09-11 16:39:46	navytux	set	messageid: <1568219986.67.0.423356853296.issue38106@roundup.psfhosted.org>
2019-09-11 16:39:46	navytux	link	issue38106 messages
2019-09-11 16:39:46	navytux	create