http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Refleaks%203.5/builds/27/steps/test/logs/stdio

[336/398/5] test_zipfile
beginning 6 repetitions
123456
......
test_zipfile leaked [3, 9, 8] references, sum=20
test_zipfile leaked [3, 9, 8] memory blocks, sum=20
(...)
Re-running test 'test_zipfile' in verbose mode
(...)
OK (skipped=1)



http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.5/builds/30/steps/test/logs/stdio

[135/398/3] test_zipfile
beginning 6 repetitions
123456
......
test_zipfile leaked [6, 6, 9] references, sum=21
test_zipfile leaked [6, 8, 11] memory blocks, sum=25
(...)
Re-running test 'test_zipfile' in verbose mode
(...)
OK (skipped=1)

I tried "./python -m test -R 3:3 test_zipfile" on Linux and my Windows VM, but I'm unable to reproduce the issue. Maybe I passed the wrong options to regrtest (-m test)?

$ git checkout upstream/3.5
$ make -j8
$ ./python.exe -m test -R 3:3 test_zipfile
[1/1] test_zipfile
beginning 6 repetitions
123456
......
1 test OK.


That is a pass on my macOS with upstream/3.5

I succeeded to reproduce the bug, but only when running the -jN option, like:

./python -m test -j2 -R 3:3 test_zipfile

I reduced the test to:

./python -m test -j2 -R 3:3 test_zipfile -m test_random_open

This test heavily use random.randint().

In fact, the bug doesn't come from test_zipfile, but it's an issue Python singleton objects. There is a bug in regrtest which doesn't call warmup_caches() in worker processes when using the -jN option (multiprocessing mode).

My PR https://github.com/python/cpython/pull/2220 should fix the bug.

Sadly, I didn't backport test_regrtest functional tests from master to 3.5 yet: see bpo-30383.

It seems like Python 3.6 and master (3.7) are not impacted by this bug, thanks to my rework on regrtest (now libregrtest) in Python 3.6.

It's unclear to me if Python 2.7 has the bug: I don't see the warm_caches() function.

Ok, using attached test_x.py, I'm able to reproduce this bug on Python 2.7.

New changeset 33cf0c4cd6e8abe138c3469ca9ec1502410945f0 by Victor Stinner in branch '3.5':
bpo-30675: Fix multiprocessing code in regrtest (#2220)
https://github.com/python/cpython/commit/33cf0c4cd6e8abe138c3469ca9ec1502410945f0

New changeset 8bb0863e0282c71d21bc2c1b5184d05d8f3a0988 by Victor Stinner in branch '2.7':
bpo-30675: Fix refleak hunting in regrtest (#2227)
https://github.com/python/cpython/commit/8bb0863e0282c71d21bc2c1b5184d05d8f3a0988

I tested: "./python -m test -j2 -R 3:3 -m test_random_open test_zipfile" now pass on 2.7, 3.5, 3.6 and master branches. So this issue is now fixed.

By the way, bpo-30776 should reduce even further false positive when hunting reference leaks.