Python Forum
sharing memory(read-write) between multiple processes
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
sharing memory(read-write) between multiple processes
#1
I need multiple processes working to be updating(adding data) to an array in memory, so it will be growing large.
the code I'm posting here doesn't do the desired...

import multiprocessing
import ctypes
import numpy as np
shared_array = None


shared_array_base = multiprocessing.Array(ctypes.c_int64, 1 * 10)
shared_array = np.ctypeslib.as_array(shared_array_base.get_obj())
shared_array = shared_array.reshape(1, 10)
# Parallel processing
def my_func(i):
    '''
    anytime a process runs this code, the desire is for the shared array to be stacked with the contents of arr,
    so by running it for example ten times, after all processes return, printing  shape(shared_array)==(11,10)
    //any indication in the right direction how to do that?

    '''
    global shared_array
    arr=np.array([44,44,44,44,44,44,44,44,44,44])
    shared_array=np.vstack((shared_array,arr))
    #print (shared_array)


if __name__ == '__main__':

    pool = multiprocessing.Pool(processes=2, )
    pool.map(my_func, range(10))

    print(shared_array)  # gives [[0 0 0 0 0 0 0 0 0 0]]
Reply
#2
Quote:the code I'm posting here doesn't do the desired...
What in the heck does that mean?

To pass data to/from/between a Process you use a Manager object.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Getting an error while trying to process data, low memory when memory is not low? bkeith12 0 1,222 Dec-20-2024, 03:06 PM
Last Post: bkeith12
  python read PDF Statement and write it into excel mg24 1 1,787 Sep-22-2024, 11:42 AM
Last Post: Pedroski55
  Delete file with read-only permission, but write permission to parent folder cubei 6 29,209 Jun-01-2024, 07:22 AM
Last Post: Eleanorreo
  Best way to secure API key when sharing quarinteen 2 1,847 Jan-19-2024, 04:46 PM
Last Post: deanhystad
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 5,858 Nov-09-2023, 10:56 AM
Last Post: mg24
Question Special Characters read-write Prisonfeed 1 2,140 Sep-17-2023, 08:26 PM
Last Post: Gribouillis
  How do I read and write a binary file in Python? blackears 6 35,977 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Read text file, modify it then write back Pavel_47 5 6,984 Feb-18-2023, 02:49 PM
Last Post: deanhystad
  how to read txt file, and write into excel with multiply sheet jacklee26 14 21,103 Jan-21-2023, 06:57 AM
Last Post: jacklee26
  processes shall be parallel flash77 4 2,685 Sep-20-2022, 11:46 AM
Last Post: DeaD_EyE

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020