How to pass a list by value for manipulation within a process?

[bRitch022]

I am attempting to update a list that lives in an object that I have named Registry. This list is named Registry.registry, and it is a numpy.zeros list. A process calls the list, and attempts to write new data to the list at a specific address. In my test case, I just use the very first index of the list 0x00. The problem that is arising is that the process manipulates a copy of the list, rather than the original list itself. I can see this by printing the object's address during a read or write. So, when I attempt to read the list from a separate process, it only reads yet another copy of the list, not the original that is intended. Here is my code:

import multiprocessing
import numpy as np

class Registry(object):
    def __init__(self, registry_size=128, register_size=16):
        # Registry list to write to
        self.registry = np.zeros((registry_size, register_size), dtype=int)

        # Intermediate buffer for use in bit manipulation
        self.buff = np.zeros((1,16))

    def write16_register(self, reg_addr, data):
        # Convert data to 16 digit binary and sign extend
        data = int(data)
        #print("int(data): " + str(data))
    
        # Split data into 8 bit MSB and LSB
        data_MSB = (data & 0xFF00) >> 8
        data_LSB = (data & 0x00FF)
  
    
        # Load MSB
        self.buff = format(data_MSB, '08b')
        for i in range(0,8,1):
             self.registry[reg_addr][i] = self.buff[i]

        # Load LSB
        self.buff = format(data_LSB, '08b')
        for i in range(8,16,1):
             self.registry[reg_addr][i] = self.buff[i-8]

        # Print statement to see and verify what was written
        print("Write: {0}  to  {1}".format(self.registry[reg_addr], self))

    def read_register(self, reg_addr):
        # Print statement to see and verify what is read before returning
        print("Read:  {0} from {1}".format(self.registry[reg_addr], self))

        return self.registry[reg_addr]

if __name__ == '__main__':
    Main_Registry = Registry()
    p1 = multiprocessing.Process(target=Main_Registry.write16_register, args=(0x00, 0xB594,))
    p2 = multiprocessing.Process(target=Main_Registry.read_register, args=(0x00,))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

Output:
Write: [1 0 1 1 0 1 0 1 1 0 0 1 0 1 0 0]  to  <__main__.Registry object at 0xb5c7de30>
Read:  [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] from <__main__.Registry object at 0xb5c7de70>

As you can see in the output, the __main__.Registry objects are at different addresses. How can I share the same object between processes to ensure that I am reading and writing to the same list?

---

I've seen some sources talking about using a Pipe or a Queue to accomplish this?

[bowlofred]

Your title says "within a process", but your body looks like you're trying to share it between multiple processes.

I think if you want to share an object between multiprocessing calls, you need to set up a proxy-object. The proxy uses the object in only one of the processes. That said, I've never attempted to use that configuration.

[bRitch022]

Your title says "within a process", but your body looks like you're trying to share it between multiple processes.

Yes, you are correct, I am trying to share this across multiple processes.

I think if you want to share an object between multiprocessing calls, you need to set up a proxy-object. The proxy uses the object in only one of the processes. That said, I've never attempted to use that configuration.

Do you have a suggestion on how I might better accomplish this? Absent other suggestions I will certainly look into the proxy-object. I am building a machine which has mutiple data points (encoder positions, digital sensor inputs, color sensor, thermocouple, etc) and I am trying to have all of the data stored into this one registry to be accessed by front end developers. I want the Registry to be constantly updating in the background.

[buran]

not used it, but in 3.8 there is multiprocessing.shared_memory. I think that's what you need. Look at the docs

[bRitch022]

I was able to resolve this issue by getting rid of the numpy.zeroes array, and instead using the multiprocessing.Array datatype. In this way, all the separate processes were able to access and modify the same array, rather than a copy.