This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Value error for string shared memory in multiprocessing
Type: crash Stage: resolved
Components: Library (Lib), Unicode Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, hongweipeng, iritkatriel, magu, xtreak
Priority: normal Keywords:

Created on 2017-12-12 17:00 by magu, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg308144 - (view) Author: Marc Guetg (magu) Date: 2017-12-12 17:00
It seems like sharing a string over processes is not possible.

	#!/usr/bin/env python3
	
	import multiprocessing
	import threading
	import ctypes
	
	
	def fun(share_c, share_s, set_c, set_s, name):
		print(f'{name}: {share_c.value}; {share_s.value}')
		share_c.value = set_c
		share_s.value = set_s
		print(f'{name}: {share_c.value}; {share_s.value}')
	
	
	if __name__ == '__main__':
		share_c = multiprocessing.Value(ctypes.c_wchar, 'a')
		share_s = multiprocessing.Value(ctypes.c_wchar_p, 'aa')
	
		print(f'pre_thread: {share_c.value}; {share_s.value}')
		thread = threading.Thread(target=fun, args=(share_c, share_s, 'b', 'bb', 'thread'))
		thread.start()
		thread.join()
	
		print(f'post_thread: {share_c.value}; {share_s.value}')
		process = multiprocessing.Process(target=fun, args=(share_c, share_s, 'c', 'cc', 'process'))
		process.start()
		process.join()
	
		print(f'post_process: {share_c.value}', end='; ')
		print(share_s.value)  # <--- Blows here

produces: 

	pre_thread: a; aa
	thread: a; aa
	thread: b; bb
	post_thread: b; bb
	process: b; bb
	process: c; cc
	post_process: c; Traceback (most recent call last):
	  File "test2.py", line 30, in <module>
	    print(share_s.value)  # <--- Blows here
	  File "<string>", line 5, in getvalue
	ValueError: character U+ff92f210 is not in range [U+0000; U+10ffff]

Where the character value in the error message is different every time. To me this seems like a bug as it is working properly with threads as well as single characters. (Maybe relevant question also here: https://stackoverflow.com/questions/47763878/how-to-share-string-between-processes?noredirect=1#comment82492062_47763878)

For the case it matters:
Python 3.6.1 (Anaconda 4.4.0) on RHEL 6
msg326673 - (view) Author: hongweipeng (hongweipeng) * Date: 2018-09-29 09:36
This problem seems to be support for str.

import multiprocessing
import ctypes

def child_process_fun(share):
    share.value = 'bb'

if __name__ == '__main__':
    share = multiprocessing.Value(ctypes.c_wchar_p, 'aa')
    process = multiprocessing.Process(target=child_process_fun, args=(share, ))
    process.start()
    process.join()
    print(share.value)

It also raises ValueError.When use c_double or c_int, it works well.

Test in python3.7 and 3.8
msg326698 - (view) Author: hongweipeng (hongweipeng) * Date: 2018-09-30 03:22
I think I know the reason. `c_wchar_p` corresponds to the string pointer `wchar_t *`.It's not a good idea to pass pointers between processes. As quoted from `multiprocessing` docs:

Note Although it is possible to store a pointer in shared memory remember that this will refer to a location in the address space of a specific process. However, the pointer is quite likely to be invalid in the context of a second process and trying to dereference the pointer from the second process may cause a crash.

https://docs.python.org/3.6/library/multiprocessing.html?#module-multiprocessing.sharedctypes
msg404145 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-10-17 20:48
Is there anything left to do here?

It seems hongweipeng's explanation and the link to the documentation pretty much cover it.
History
Date User Action Args
2022-04-11 14:58:55adminsetgithub: 76472
2021-11-05 19:46:29iritkatrielsetstatus: pending -> closed
stage: resolved
2021-10-18 10:12:34iritkatrielsetstatus: open -> pending
2021-10-18 10:01:10vstinnersetstatus: pending -> open
nosy: - vstinner
2021-10-17 20:48:34iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg404145

resolution: not a bug
2018-09-30 03:22:21hongweipengsetmessages: + msg326698
2018-09-29 09:36:59hongweipengsetmessages: + msg326673
2018-09-29 08:59:25hongweipengsetnosy: + hongweipeng
2018-09-20 13:18:21xtreaksetnosy: + xtreak
2017-12-12 17:00:36magucreate