‘exec(): Unable to fork’ on Ubuntu 14.04.3 LTS

I’m running unittest scripts and for test case setup I need to run shell command from time to time. As a result shell command is run around 50 times during unittest execution.

It works ok on CentOS release 6.6 (Final) server but doesn’t work on Ubuntu 14.04.3 LTS.

The problem is that after around 20th time shell script execution I get the following error:

exec(): Unable to fork [$(which mysql) -u user -ppwd test_db < "/opt/jenkins/workspace/some - project/application/../tests/Application/assets/sql/test_db.sql" 2> /dev/null]

I’ve checked all the topics around the internet and no one helped me. I’ve increased process number and open files limit to enormous values

# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3885
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 100000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 6553500
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

PHP reports similar numbers. (file test.php contains <?php echo system("ulimit -a");)

# php test.php
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        8192
coredump(blocks)     0
memory(kbytes)       unlimited
locked memory(kbytes) 64
process              6553500
nofiles              100000
vmemory(kbytes)      unlimited
locks                unlimited
locks                unlimited

I run scripts as a non-root user from CLI.

I’ve increased php memory size to 2Gb

memory_limit => 2048M => 2048M

I’ve tried to use system, passthrough, shel_exec and it makes not difference.

PHP version is 5.6

PHP 5.6.14-1+deb.sury.org~trusty+1 (cli)

Does anybody know how to solve the problem?

Answer

The issue in my case was very simple and not related to configuration changes I’ve done. I think it is reasonable to keep my experience here so that it can help other developers to avoid the same issues.

The problem was in lack of memory on server. It was virtual machine with 1Gb RAM. When script memory usage was around 700Mb it start failing with ‘unable to fork’ error. Which is strange and doesn’t sound like memory issues.

Increasing server’s memory size up to 4Gb solved the issue.

Advertisement

Answer