I write a Dockerfile and docker-compose.yml to build a custom image which combines beakerx and cling, which contents are below.
FROM beakerx/beakerx
MAINTAINER liudonghua123 <liudonghua123@gmail.com>
# not works
#RUN conda install xeus-cling notebook -c QuantStack -c conda-forge -y --quiet
USER root
# install gcc7
RUN add-apt-repository -y ppa:jonathonf/gcc-7.2
RUN apt-get update -y
RUN apt-get install -y gcc-7
# revert to beakerx user
USER beakerx
ARG CLING_FILENAME=cling_2018-09-04_ubuntu16
RUN echo "use ${CLING_FILENAME} for build"
# add the prebuild cling packages
ADD ${CLING_FILENAME}.tar.bz2 /home/beakerx
ENV PATH=/home/beakerx/${CLING_FILENAME}/bin:$PATH
USER root
RUN pip install --upgrade pip
# install steps, https://github.com/root-project/cling/tree/master/tools/Jupyter
RUN cd /home/beakerx/${CLING_FILENAME}/share/cling/Jupyter/kernel && pip install -e . && jupyter-kernelspec install --user cling-cpp17 && jupyter-kernelspec install --user cling-cpp1z && jupyter-kernelspec install --user cling-cpp14 && jupyter-kernelspec install --user cling-cpp11
RUN ln -s /usr/bin/gcc-7 /usr/bin/gcc
RUN echo "root:root" | chpasswd
RUN echo "beakerx:beakerx" | chpasswd
RUN chown -R beakerx:beakerx /home/beakerx/.local
RUN find /home/beakerx/.local -type d -exec chmod 755 {} ;
RUN find /home/beakerx/.local -type f -exec chmod 644 {} ;
RUN id
RUn ls -la /home/beakerx/.local
RUn ls -la /home/beakerx/.local/share
USER beakerx
and
version: '2'
services:
beakerx-cling-prebuild:
build: .
image: liudonghua123/beakerx-cling-prebuild:latest
ports:
- "28888:8888"
volumes:
- ./work:/work
restart: always
Some logs of docker-compose build were
Step 22/24 : RUN ls -la /home/beakerx/.local ---> Running in 95457585aed0 total 12 drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 25 beakerx beakerx 4096 Sep 6 00:51 .. drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 share ---> affcb9f1ca94 Removing intermediate container 95457585aed0 Step 23/24 : RUN ls -la /home/beakerx/.local/share ---> Running in 15ea51bcc3bf total 12 drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 .. drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 jupyter ---> 6f2ca28d4d21 Removing intermediate container 15ea51bcc3bf Step 24/24 : USER beakerx ---> Running in 0ab6574079d7 ---> 9561902b99ef Removing intermediate container 0ab6574079d7 Successfully built 9561902b99ef
But When I run docker-compose up -d, I got PermissionError: [Errno 13] Permission denied: '/home/beakerx/.local/share' error. So I drive into the docker image.
ldh@ldh55:~/docker/cling/jupinger/beakerx-cling-prebuild$ docker run -it --entrypoint='' --rm liudonghua123/beakerx-cling-prebuild bash beakerx@716d9a8334ca:~$ ls -la ~ total 108 drwxr-xr-x 25 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 11 root root 4096 Sep 6 00:51 .. -rw-r--r-- 1 beakerx beakerx 220 Aug 31 2015 .bash_logout -rw-r--r-- 1 beakerx beakerx 3771 Aug 31 2015 .bashrc drwxr-xr-x 3 root root 4096 Mar 13 13:32 .config -rw-r--r-- 1 beakerx beakerx 938 Mar 8 14:03 .gitignore -rw-r--r-- 1 beakerx beakerx 53 Feb 27 2018 .jscsrc drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 .local drwxr-xr-x 3 root root 4096 Mar 13 13:32 .npm -rw-r--r-- 1 beakerx beakerx 655 May 16 2017 .profile -rw-r--r-- 1 beakerx beakerx 2285 Feb 27 2018 CONTRIBUTING.md -rw-r--r-- 1 beakerx beakerx 11325 Feb 27 2018 LICENSE -rw-r--r-- 1 beakerx beakerx 193 Feb 27 2018 NOTICE -rw-r--r-- 1 beakerx beakerx 8682 Mar 12 14:27 README.md -rw-r--r-- 1 beakerx beakerx 5821 Mar 12 20:04 StartHere.ipynb -rw-r--r-- 1 beakerx beakerx 6 Mar 13 13:32 VERSION drwxr-xr-x 16 beakerx beakerx 4096 Mar 12 20:11 beakerx drwxr-xr-x 8 14806 2735 4096 Sep 5 07:44 cling_2018-09-04_ubuntu16 drwxr-xr-x 22 beakerx beakerx 4096 Feb 27 2018 doc -rw-r--r-- 1 beakerx beakerx 81 Feb 27 2018 environment.yml drwxr-xr-x 6 beakerx beakerx 4096 Feb 27 2018 js -rwxr-xr-x 1 beakerx beakerx 927 Mar 13 12:42 setup.sh beakerx@716d9a8334ca:~$ ls -la ~/.local/ ls: cannot access '/home/beakerx/.local/share': Permission denied total 8 drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 25 beakerx beakerx 4096 Sep 6 00:51 .. d????????? ? ? ? ? ? share beakerx@716d9a8334ca:~$ ls -la ~/.local/share ls: cannot access '/home/beakerx/.local/share': Permission denied beakerx@716d9a8334ca:~$ beakerx@716d9a8334ca:~$ su Password: root@716d9a8334ca:/home/beakerx# ls -la ~ total 28 drwx------ 4 root root 4096 Mar 13 13:27 . drwxr-xr-x 77 root root 4096 Sep 6 01:06 .. -rw-r--r-- 1 root root 3106 Oct 22 2015 .bashrc drwxr-xr-x 3 root root 4096 Mar 13 13:20 .conda -rw-r--r-- 1 root root 38 Mar 13 13:20 .condarc drwxr-xr-x 5 root root 4096 Mar 13 13:28 .gradle -rw-r--r-- 1 root root 148 Aug 17 2015 .profile root@716d9a8334ca:/home/beakerx# ls -la /home/beakerx/ total 108 drwxr-xr-x 25 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 11 root root 4096 Sep 6 00:51 .. -rw-r--r-- 1 beakerx beakerx 220 Aug 31 2015 .bash_logout -rw-r--r-- 1 beakerx beakerx 3771 Aug 31 2015 .bashrc drwxr-xr-x 3 root root 4096 Mar 13 13:32 .config -rw-r--r-- 1 beakerx beakerx 938 Mar 8 14:03 .gitignore -rw-r--r-- 1 beakerx beakerx 53 Feb 27 2018 .jscsrc drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 .local drwxr-xr-x 3 root root 4096 Mar 13 13:32 .npm -rw-r--r-- 1 beakerx beakerx 655 May 16 2017 .profile -rw-r--r-- 1 beakerx beakerx 2285 Feb 27 2018 CONTRIBUTING.md -rw-r--r-- 1 beakerx beakerx 11325 Feb 27 2018 LICENSE -rw-r--r-- 1 beakerx beakerx 193 Feb 27 2018 NOTICE -rw-r--r-- 1 beakerx beakerx 8682 Mar 12 14:27 README.md -rw-r--r-- 1 beakerx beakerx 5821 Mar 12 20:04 StartHere.ipynb -rw-r--r-- 1 beakerx beakerx 6 Mar 13 13:32 VERSION drwxr-xr-x 16 beakerx beakerx 4096 Mar 12 20:11 beakerx drwxr-xr-x 8 14806 2735 4096 Sep 5 07:44 cling_2018-09-04_ubuntu16 drwxr-xr-x 22 beakerx beakerx 4096 Feb 27 2018 doc -rw-r--r-- 1 beakerx beakerx 81 Feb 27 2018 environment.yml drwxr-xr-x 6 beakerx beakerx 4096 Feb 27 2018 js -rwxr-xr-x 1 beakerx beakerx 927 Mar 13 12:42 setup.sh root@716d9a8334ca:/home/beakerx# ls -la /home/beakerx/.local/ total 12 drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 25 beakerx beakerx 4096 Sep 6 00:51 .. drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 share root@716d9a8334ca:/home/beakerx# ls -la /home/beakerx/.local/share/ total 12 drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 .. drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 jupyter root@716d9a8334ca:/home/beakerx# exit exit beakerx@716d9a8334ca:~$ ls -la ~/.local/share total 12 drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 . drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 .. drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 jupyter beakerx@716d9a8334ca:~$
Why the permission of ~/.local/share was a lot of question marks at the first sight, and when I run as root or return to the original user, it was correct.
I tried run start-notebook.sh manually, the first time it was failed, then the second time it was successfully.
beakerx@716d9a8334ca:~$ cd /usr/local/bin/
beakerx@716d9a8334ca:/usr/local/bin$ ls
start-notebook.sh start-singleuser.sh start.sh
beakerx@716d9a8334ca:/usr/local/bin$
beakerx@716d9a8334ca:/usr/local/bin$
beakerx@716d9a8334ca:/usr/local/bin$ start-notebook.sh
Execute the command
Traceback (most recent call last):
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/traitlets/traitlets.py", line 528, in get
value = obj._trait_values[self.name]
KeyError: 'runtime_dir'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/envs/beakerx/bin/jupyter-notebook", line 6, in <module>
sys.exit(notebook.notebookapp.main())
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/jupyter_core/application.py", line 266, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "<decorator-gen-7>", line 2, in initialize
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/notebook/notebookapp.py", line 1505, in initialize
self.init_configurables()
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/notebook/notebookapp.py", line 1209, in init_configurables
connection_dir=self.runtime_dir,
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/traitlets/traitlets.py", line 556, in __get__
return self.get(obj, cls)
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/traitlets/traitlets.py", line 535, in get
value = self._validate(obj, dynamic_default())
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/jupyter_core/application.py", line 99, in _runtime_dir_default
ensure_dir_exists(rd, mode=0o700)
File "/opt/conda/envs/beakerx/lib/python3.6/site-packages/jupyter_core/utils/__init__.py", line 13, in ensure_dir_exists
os.makedirs(path, mode=mode)
File "/opt/conda/envs/beakerx/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/beakerx/.local/share/jupyter/runtime'
beakerx@716d9a8334ca:/usr/local/bin$ ll /home/beakerx/.local/share/jupyter/runtime
ls: cannot access '/home/beakerx/.local/share/jupyter/runtime': Permission denied
beakerx@716d9a8334ca:/usr/local/bin$ ll /home/beakerx/.local/share/jupyter/
ls: cannot access '/home/beakerx/.local/share/jupyter/kernels': Permission denied
total 8
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 ./
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 ../
d????????? ? ? ? ? ? kernels/
beakerx@716d9a8334ca:/usr/local/bin$ ll /home/beakerx/.local/share/
total 12
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 ./
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 ../
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 jupyter/
beakerx@716d9a8334ca:/usr/local/bin$ su
Password:
root@716d9a8334ca:/usr/local/bin# ls -l /home/beakerx/.local/share/jupyter/runtime
ls: cannot access '/home/beakerx/.local/share/jupyter/runtime': No such file or directory
root@716d9a8334ca:/usr/local/bin# ls -l /home/beakerx/.local/share/jupyter/
total 4
drwxr-xr-x 18 beakerx beakerx 4096 Sep 6 00:51 kernels
root@716d9a8334ca:/usr/local/bin# ls -l /home/beakerx/.local/share/
total 4
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 00:51 jupyter
root@716d9a8334ca:/usr/local/bin# ls -l /home/beakerx/.local/share/jupyter/kernels/
total 16
drwxr-xr-x 2 beakerx beakerx 4096 Sep 6 00:51 cling-cpp11
drwxr-xr-x 2 beakerx beakerx 4096 Sep 6 00:51 cling-cpp14
drwxr-xr-x 2 beakerx beakerx 4096 Sep 6 00:51 cling-cpp17
drwxr-xr-x 2 beakerx beakerx 4096 Sep 6 00:51 cling-cpp1z
root@716d9a8334ca:/usr/local/bin# exit
exit
beakerx@716d9a8334ca:/usr/local/bin$ id
uid=1000(beakerx) gid=1000(beakerx) groups=1000(beakerx)
beakerx@716d9a8334ca:/usr/local/bin$ start-notebook.sh
Execute the command
[I 01:20:56.433 NotebookApp] Writing notebook server cookie secret to /home/beakerx/.local/share/jupyter/runtime/notebook_cookie_secret
[W 01:20:56.567 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 01:20:56.647 NotebookApp] [beakerx] enabled
[I 01:20:56.736 NotebookApp] JupyterLab beta preview extension loaded from /opt/conda/envs/beakerx/lib/python3.6/site-packages/jupyterlab
[I 01:20:56.736 NotebookApp] JupyterLab application directory is /opt/conda/envs/beakerx/share/jupyter/lab
[I 01:20:56.971 NotebookApp] Serving notebooks from local directory: /usr/local/bin
[I 01:20:56.971 NotebookApp] 0 active kernels
[I 01:20:56.971 NotebookApp] The Jupyter Notebook is running at:
[I 01:20:56.971 NotebookApp] http://[all ip addresses on your system]:8888/?token=1b94f5bf7e14e4ed5defece6870addc630d81eb8aae85990
[I 01:20:56.971 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 01:20:56.972 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8888/?token=1b94f5bf7e14e4ed5defece6870addc630d81eb8aae85990
If I modified Dockerfile (move up USER beakerx)
FROM beakerx/beakerx
MAINTAINER liudonghua123 <liudonghua123@gmail.com>
# not works
#RUN conda install xeus-cling notebook -c QuantStack -c conda-forge -y --quiet
USER root
# install gcc7
RUN add-apt-repository -y ppa:jonathonf/gcc-7.2
RUN apt-get update -y
RUN apt-get install -y gcc-7
# revert to beakerx user
USER beakerx
ARG CLING_FILENAME=cling_2018-09-04_ubuntu16
RUN echo "use ${CLING_FILENAME} for build"
# add the prebuild cling packages
ADD ${CLING_FILENAME}.tar.bz2 /home/beakerx
ENV PATH=/home/beakerx/${CLING_FILENAME}/bin:$PATH
USER root
RUN pip install --upgrade pip
# install steps, https://github.com/root-project/cling/tree/master/tools/Jupyter
RUN cd /home/beakerx/${CLING_FILENAME}/share/cling/Jupyter/kernel && pip install -e . && jupyter-kernelspec install --user cling-cpp17 && jupyter-kernelspec install --user cling-cpp1z && jupyter-kernelspec install --user cling-cpp14 && jupyter-kernelspec install --user cling-cpp11
RUN ln -s /usr/bin/gcc-7 /usr/bin/gcc
RUN echo "root:root" | chpasswd
RUN echo "beakerx:beakerx" | chpasswd
RUN usermod -aG sudo beakerx
RUN echo beakerx | sudo -S chown -R beakerx:beakerx /home/beakerx/
RUN echo beakerx | sudo -S find /home/beakerx/ -type d -exec chmod 755 {} ;
RUN echo beakerx | sudo -S find /home/beakerx/ -type f -exec chmod 644 {} ;
USER beakerx
RUN id
RUn ls -la /home/beakerx/.local
RUn ls -la /home/beakerx/.local/share
Then some errors occurred (ls -la /home/beakerx/.local).
Step 18/25 : RUN usermod -aG sudo beakerx
---> Using cache
---> 2b341f8539b9
Step 19/25 : RUN echo beakerx | sudo -S chown -R beakerx:beakerx /home/beakerx/
---> Running in 8adb3c593f96
---> bd6237aa0196
Removing intermediate container 8adb3c593f96
Step 20/25 : RUN echo beakerx | sudo -S find /home/beakerx/ -type d -exec chmod 755 {} ;
---> Running in 5dc9ff3d8d1b
---> 98d3338124ce
Removing intermediate container 5dc9ff3d8d1b
Step 21/25 : RUN echo beakerx | sudo -S find /home/beakerx/ -type f -exec chmod 644 {} ;
---> Running in 63a4ce864b75
---> 0ca175cb1663
Removing intermediate container 63a4ce864b75
Step 22/25 : USER beakerx
---> Running in 42f8c91f6930
---> 94e9f6099aa3
Removing intermediate container 42f8c91f6930
Step 23/25 : RUN id
---> Running in 6f24ee8ce894
uid=1000(beakerx) gid=1000(beakerx) groups=1000(beakerx),27(sudo)
---> 58627cedc9de
Removing intermediate container 6f24ee8ce894
Step 24/25 : RUN ls -la /home/beakerx/.local
---> Running in cfd7bdd70666
ls: cannot access '/home/beakerx/.local/share': Permission denied
total 8
drwxr-xr-x 6 beakerx beakerx 4096 Sep 6 03:08 .
drwxr-xr-x 43 beakerx beakerx 4096 Sep 6 03:09 ..
d????????? ? ? ? ? ? share
ERROR: Service 'beakerx-cling-prebuild' failed to build: The command '/bin/sh -c ls -la /home/beakerx/.local' returned a non-zero code: 1
ldh@ldh55:~/docker/cling/jupinger/beakerx-cling-prebuild$
Advertisement
Answer
This problem is related to the storage-driver bug, see https://github.com/moby/moby/issues/28391, https://github.com/moby/moby/issues/20240. Currently I can only change storage-driver to overlay, use the default aufs or recommended overlay2 will break.