Intermittent “OSError: [Errno 7] Argument list too long” with short command (~125 chars)

This code running under apache2+mod_wsgi on Linux sometimes produces the following output.

   notes.py
            cmd_list = [
                    'abc_generate_pdf',
                    '--cdb-url-prefix', model.config('cdb_url_prefix'),
                    '--request-cid', request_cid,
            ]
            log.info("About to run: {!r}".format(cmd_list))
            subprocess.Popen(cmd_list)
...
2014-07-16 11:03:44,779 INFO     pid:17925,140353357215488 abc.webapp.check.notes:198:approval_revoke About to run: ['abc_generate_pdf', '--cdb-url-prefix', 'xxxxdev', '--request-cid', u'xxxxx_xxx_2014-07-15_16.12.50.108807_685932a62c7c6226987acdeda367dbc3']
2014-07-16 11:03:45,250 ERROR    pid:17925,140353357215488 abc.webapp.check.notes:208:approval_revoke Unknown error; approval not updated.
Traceback (most recent call last):
  File "/home/abc/abc/webapp/check/notes.py", line 199, in approval_revoke
    subprocess.Popen(cmd_list)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 7] Argument list too long

The output has been anonymized slightly, but the character counts remain exact.

The problem seems to happen after the web server has been running for a long time, and goes away with a restart, however it’s difficult to pin down exactly what the trigger might be.

Why am I getting an error, and what can I do to prevent it?

This question is very similar to Python OSError no 7 (Argument list too long ) in linux , which doesn’t have a satisfactory answer.

Edit: I should note that the problem also happens under the pure-python werkzeug server.

Answer

We had a similar problem and bothered us for a quite while too. Today, we found the root cause.

The error message is little bit misleading, it is not command line’s “argument” too long, it is actually the OS environment passed along with command. In this case, Python os.environ. In our code base, there is one line code:

os.environ['PATH'] =  ':'.join([os.environ['PATH'], self.clitoolsdir])

It is part of a class’s __init__(). Anytime a new object was created, os.environ got longer, eventually it became “too long”.