This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Daniel Colascione
Recipients Daniel Colascione
Date 2017-10-01.01:50:46
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1506822647.49.0.213398074469.issue31654@psf.upfronthosting.co.za>
In-reply-to
Content
Say we're using multiprocessing to share a counter between two processes and we want to atomically increment that counter. Right now, we need to protect that counter with a multiprocessing semaphore of some sort, then 1) acquire the semaphore, 2) read-modify-write the counter value, and 3) release the semaphore. What if we're preempted by a GIL-acquire request after step #1 but before step #3? We'll hold the semaphore until the OS scheduler gets around to running us again, which might be a while in the case of compute-bound tasks (especially if these tasks call C code that doesn't release the GIL).

Now, if some other process wants to increment the counter, it needs to wait on the first process's GIL! That partially defeats the purpose of multiprocessing: one of the nice things about multiprocessing is avoiding GIL-introduced latency!

If ctypes supported atomic operations, we could skip steps #1 and #3 entirely and operate directly on the shared memory region. Every operating system that supports threads at all also supports some kind of compare-and-exchange primitive. Compare-and-exchange is sufficient for avoiding the GIL contention I describe above.
History
Date User Action Args
2017-10-01 01:50:47Daniel Colascionesetrecipients: + Daniel Colascione
2017-10-01 01:50:47Daniel Colascionesetmessageid: <1506822647.49.0.213398074469.issue31654@psf.upfronthosting.co.za>
2017-10-01 01:50:47Daniel Colascionelinkissue31654 messages
2017-10-01 01:50:46Daniel Colascionecreate