multiprocessing difference between Linux and Windows

[direl]

I was trying to accept timed input when I noticed that Windows did not behave as I expected. A simplified version of the code to demonstrate this is below:

import multiprocessing, sys, os

def _input_with_timeout_process(stdin_file_descriptor, queue, prompt):
    sys.stdin = os.fdopen(stdin_file_descriptor)
    queue.put(input(prompt))
    return

print("=== 1 ===", 'parent process:', os.getppid(), 'my pid:', os.getpid())

if __name__ == '__main__':
    queue = multiprocessing.Queue()
    process = multiprocessing.Process(target=_input_with_timeout_process, 
        args=(sys.stdin.fileno(), queue, "Say something: "))
    process.start()
    try:
        process.join(10)
        if process.is_alive():
            raise ValueError("Timed out waiting for input.")
        print("I saw:", queue.get())
    except ValueError:
        print("Sorry, too slow.")
    finally:
        process.terminate()

Running this with Python 3.8.2 on Ubuntu 20.04 gives:

Output:
direl@MD-3670:~/Shared$ python min_mp_test.py
=== 1 === parent process: 30833 my pid: 32234
Say something: g
I saw: g
direl@MD-3670:~/Shared$

However on Windows 10 with Python 3.8.6 I get

Output:
PS C:\Users\mdavi\Documents> python z:min_mp_test.py
=== 1 === parent process: 4288 my pid: 5408
=== 1 === parent process: 5408 my pid: 5764
Say something: f
I saw: f
PS C:\Users\mdavi\Documents>

It looks as though the second process is escaping from its function definition and executing any subsequent code that is not in either a function definition or an
if __name__ == '__main__': statement.

While making sure that everything is indented under something will avoid the problem I was curious if this is actually the correct or even accepted behaviour. Note that the original program did not have the if __name__ == '__main__': and its behaviour was a lot stranger.

[deanhystad]

IDLE does not like sys.stdin.fileno(). It is not supported by IDLE's version of stdin. Interesting.

I changed your example slightly.

# Change
print("=== 1 ===", 'parent process:', os.getppid(), 'my pid:', os.getpid())
# to
print(__name__, 'parent process:', os.getppid(), 'my pid:', os.getpid())

Output:
__main__ parent process: 37800 my pid: 36160
__mp_main__ parent process: 36160 my pid: 33788
Say something: a
I saw: a

To me this means that a windows process must read the Python source. It doesn't matter if the process is started by running Python or calling multiprocessing.Process(). The windows process is a fresh new thing that must be taught. On the other hand the Linux process must be more like a clone, knowing what the parent process knows when it is spawned.

Not being a Linux expert, but having some significant Unix experience from back n the days when BSD was new, I remembered the "fork" call that would create a "child" process. The child process was essentially a copy of the parent process and started life knowing everything the parent knew. Turns out that Linux has the same call and this is likely what Python uses on Linux.

So the reason things look different running a process on Windows and on Linux is that a Windows process starts up knowing nothing and a Linux process starts up knowing what its parent knows. The windows process has to "load" your Python program and that is why you get the extra print.

[direl]

That makes sense and I should probably start including the name check in all my programs just in case.

Thanks, I am happier knowing there is some logic in it.