[Python-Dev] Opcode cache in ceval loop

[Serhiy Storchaka]

On 02.02.16 19:45, Yury Selivanov wrote:
> On 2016-02-02 12:41 PM, Serhiy Storchaka wrote:
>> On 01.02.16 21:10, Yury Selivanov wrote:
>>> To measure the max/average memory impact, I tuned my code to optimize
>>> *every* code object on *first* run.  Then I ran the entire Python test
>>> suite.  Python test suite + standard library both contain around 72395
>>> code objects, which required 20Mb of memory for caches.  The test
>>> process consumed around 400Mb of memory.  Thus, the absolute worst case
>>> scenario, the overhead is about 5%.
>>
>> Test process consumes such much memory because few tests creates huge
>> objects. If exclude these tests (note that tests that requires more
>> than 1Gb are already excluded by default) and tests that creates a
>> number of threads (threads consume much memory too), the rest of tests
>> needs less than 100Mb of memory. Absolute required minimum is about
>> 25Mb. Thus, the absolute worst case scenario, the overhead is about 100%.
> Can you give me the exact configuration of tests (command line to run)
> that would only consume 25mb?

I don't remember what exact tests consume the most of memory, but 
following tests are failed when run with less than 30Mb of memory:

test___all__ test_asynchat test_asyncio test_bz2 test_capi 
test_concurrent_futures test_ctypes test_decimal test_descr 
test_distutils test_docxmlrpc test_eintr test_email test_fork1 
test_fstring test_ftplib test_functools test_gc test_gdb test_hashlib 
test_httplib test_httpservers test_idle test_imaplib test_import 
test_importlib test_io test_itertools test_json test_lib2to3 test_list 
test_logging test_longexp test_lzma test_mmap test_multiprocessing_fork 
test_multiprocessing_forkserver test_multiprocessing_main_handling 
test_multiprocessing_spawn test_os test_pickle test_poplib test_pydoc 
test_queue test_regrtest test_resource test_robotparser test_shutil 
test_smtplib test_socket test_sqlite test_ssl test_subprocess 
test_tarfile test_tcl test_thread test_threaded_import 
test_threadedtempfile test_threading test_threading_local 
test_threadsignals test_tix test_tk test_tools test_ttk_guionly 
test_ttk_textonly test_tuple test_unicode test_urllib2_localnet 
test_wait3 test_wait4 test_xmlrpc test_zipfile test_zlib