I have 2 processes (one C, one python) that are started at the same time and both need a certain directory tree to exist to continue, so both processes create the directory tree while ignoring errors about it already existing.

In the python process, I'm using pathlib's mkdir function (https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir) with parents=True, exist_ok=True, but this is sometimes throwing a FileExistsError. I don't expect this because the documentation says "If exist_ok is true, FileExistsError exceptions will be ignored (same behavior as the POSIX mkdir -p command), but only if the last path component is not an existing non-directory file." The last component is never a non-directory file.

I believe what is happening is that mkdir realizes that the parent doesn't exist, so it recursively tries to make the parent directory. However, when it recurses, it uses the default exists_ok value of False. Before the recursive call can make the parent directory, the other process makes the directory. This causes the inner call to throw a FileExistsError.

I believe the correct thing to do is to always pass True for exists_ok in the recursive call.

Thanks for the report. There was a related issue : issue29694 . This was fixed in 3.7 and was backported with https://github.com/python/cpython/commit/d7abeb7024b9755c291c29bdc8c4494246e975ad (3.5.4 and above) . Can you please specify the full version of Python 3.5 you are using and if the linked issue is related ? If possible please attach a reproducer so  that this can be tested on other versions.

Thank your for your response. I'm running python 3.5.2. The linked issue is indeed a duplicate of this one.

To reproduce, you can run two instances of the attached script at the same time, i.e. python3 pymkdir.py & python3 pymkdir.py. It is a race condition so won't happen every time, but with the number of directories created it should happen pretty frequently (remove the directories between tests with rm -rf 1)

I'm running Ubuntu 16.04.5 LTS. I don't know if you know anything about Ubuntu's release process, but it would be nice if the linked fix could be backported to work on that platform.

> Thank your for your response. I'm running python 3.5.2. The linked issue is indeed a duplicate of this one.

Thanks for the confirmation and script.

> I'm running Ubuntu 16.04.5 LTS. I don't know if you know anything about Ubuntu's release process, but it would be nice if the linked fix could be backported to work on that platform.

I have limited knowledge of Ubuntu release process and from what I can see from https://packages.ubuntu.com/xenial/python3.5 3.5.2 is the latest release for python 3.5 ans the backport is at 3.4 and above. Core python team doesn't maintain the releases for Ubuntu and it's upto the relevant maintainer to package this. You can try a release from https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa.

Closing this as duplicate of issue29694. I hope this is backported to Ubuntu repos and there is not much pending work to this issue. Thanks.