Message303442
Say we're using multiprocessing to share a counter between two processes and we want to atomically increment that counter. Right now, we need to protect that counter with a multiprocessing semaphore of some sort, then 1) acquire the semaphore, 2) read-modify-write the counter value, and 3) release the semaphore. What if we're preempted by a GIL-acquire request after step #1 but before step #3? We'll hold the semaphore until the OS scheduler gets around to running us again, which might be a while in the case of compute-bound tasks (especially if these tasks call C code that doesn't release the GIL).
Now, if some other process wants to increment the counter, it needs to wait on the first process's GIL! That partially defeats the purpose of multiprocessing: one of the nice things about multiprocessing is avoiding GIL-introduced latency!
If ctypes supported atomic operations, we could skip steps #1 and #3 entirely and operate directly on the shared memory region. Every operating system that supports threads at all also supports some kind of compare-and-exchange primitive. Compare-and-exchange is sufficient for avoiding the GIL contention I describe above. |
|
| Date |
User |
Action |
Args |
| 2017-10-01 01:50:47 | Daniel Colascione | set | recipients:
+ Daniel Colascione |
| 2017-10-01 01:50:47 | Daniel Colascione | set | messageid: <1506822647.49.0.213398074469.issue31654@psf.upfronthosting.co.za> |
| 2017-10-01 01:50:47 | Daniel Colascione | link | issue31654 messages |
| 2017-10-01 01:50:46 | Daniel Colascione | create | |
|