Python multiprocessing crashes docker container

  • A+

There is simple python multiprocessing code that works like a charm, when I run it in console:

# import multiprocessing as mp   def do_smth():     print('something')   if __name__ == '__main__':     ctx = mp.get_context("spawn")     p = ctx.Process(target=do_smth, args=tuple())     p.start()     p.join() 


> $ python3 something 

Then I've created a simple Docker container with Dockerfile:

FROM python:3.6  ADD . /app WORKDIR /app 

And docker-compose.yml:

version: '3.6'  services:   bug:     build:       context: .     environment:       - PYTHONUNBUFFERED=1     command: su -c "python3.6" 

Where is:

from time import sleep  if __name__ == '__main__':     i = 0     while True:         sleep(1.0)         i += 1         print(f'hello {i:3}') 

Now I run with docker compose:

> $ docker-compose build && docker-compose up  ... some output ... Attaching to mpbug_bug_1 bug_1  | hello   1 bug_1  | hello   2 bug_1  | hello   3 bug_1  | hello   4 

Up to this moment everything is good and understandable. But when I'm trying to run in the docker container it crashes without any message:

> $ docker exec -it mpbug_bug_1 /bin/bash root@09779ec47f9d:/app# python  something root@09779ec47f9d:/app# %  

Gist with the code can be found here:

Can you explain why docker container is crashed and how to do it without crashing?

Thank you in advance!


for a quick fix, do not use spawn start method, and/or do not use su -c ..., both are unnecessary IMO. change to:

p = mp.Process(target=do_smth, args=tuple()) 

or you could start container with --init option.

with spawn start method, Python will also start a semaphore tracker process to prevent semaphore leaking, you could see this process by pausing in the middle, it looks like:

472   463 /usr/local/bin/python3 -c from multiprocessing.semaphore_tracker import main;main(3) 

this process is started by but exited after, thus it will not be reaped by, but is supposed to be reaped by init by design.

the problem is there is no init in this container(namespace), instead of init, PID 1 is su -c, therefore the dead semaphore tracker process is adopted by su.

it appears that su think the dead child process is the command process( mistakenly, without checking the relationship, so su exited blindly, as PID 1 exit, kernel kills all other processes in the container, including

this behavior could be observed with strace:

docker run --security-opt seccomp:unconfined --rm -it ex_bug strace -e trace=process -f su -c 'python3' 

will output error message like:

strace: Exit of unknown pid 14 ignored 

ref: Docker and the PID 1 zombie reaping problem (


:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: