jupyter notebook can’t detect conda kernels only on boot

I’m trying to automatically start jupyter server on boot. (ec2, amazon linux)

I got the jupyter server working and correctly loading all the conda kernels.

and I tried to run jupyter on boot. (tried rc.local, systemd, crontab and all fails to load conda kernels and only loads basic python2 and python3 kernels)

I’ve traced it down to the nb_conda_kernels not finding being able to call ‘conda’ on boot, as below:

Oct 30 01:07:38 ip-172-31-17-102 rc.local: [E 01:07:38.816 NotebookApp] [nb_conda_kernels] couldn't call conda:
Oct 30 01:07:38 ip-172-31-17-102 rc.local:    [Errno 2] No such file or directory: 'conda': 'conda'
Oct 30 01:07:38 ip-172-31-17-102 rc.local: [I 01:07:38.816 NotebookApp] [nb_conda_kernels] enabled, 0 kernels found
Oct 30 01:07:39 ip-172-31-17-102 rc.local: [I 01:07:39.645 NotebookApp] Loading IPython parallel extension
Oct 30 01:07:39 ip-172-31-17-102 rc.local: [I 01:07:39.701 NotebookApp] JupyterLab extension loaded from /home/ec2-user/anaconda3/lib/python3.7/site-packages/jupyterlab
Oct 30 01:07:39 ip-172-31-17-102 rc.local: [I 01:07:39.702 NotebookApp] JupyterLab application directory is /home/ec2-user/anaconda3/share/jupyter/lab
Oct 30 01:07:40 ip-172-31-17-102 rc.local: [I 01:07:40.061 NotebookApp] [nb_conda] enabled
Oct 30 01:07:40 ip-172-31-17-102 rc.local: [I 01:07:40.061 NotebookApp] Serving notebooks from local directory: /home/ec2-user/browse
Oct 30 01:07:40 ip-172-31-17-102 rc.local: [I 01:07:40.061 NotebookApp] The Jupyter Notebook is running at:
Oct 30 01:07:40 ip-172-31-17-102 rc.local: [I 01:07:40.061 NotebookApp] http://172.31.17.102:8888/
Oct 30 01:07:40 ip-172-31-17-102 rc.local: [I 01:07:40.061 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

The weirdest thing is that when I kill the running server with “sudo ss –tulpn | grep 8888” (to find pid for jupyter notebook which is listening on port 8888) and “sudo kill -9 {pid}” (to kill the process. checked that I killed it successfully), and I restart the server via the same command used in boot, then jupyter successfully loads the conda kernels, as below:

[I 01:12:51.369 NotebookApp] [nb_conda_kernels] enabled, 22 kernels found
[I 01:12:51.612 NotebookApp] Loading IPython parallel extension
[I 01:12:51.641 NotebookApp] JupyterLab extension loaded from /home/ec2-user/anaconda3/lib/python3.7/site-packages/jupyterlab
[I 01:12:51.641 NotebookApp] JupyterLab application directory is /home/ec2-user/anaconda3/share/jupyter/lab
[I 01:12:51.791 NotebookApp] [nb_conda] enabled
[I 01:12:51.792 NotebookApp] Serving notebooks from local directory: /home/ec2-user/browse
[I 01:12:51.792 NotebookApp] The Jupyter Notebook is running at:
[I 01:12:51.792 NotebookApp] http://172.31.17.102:8888/
[I 01:12:51.792 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

I’ve seen one particular question on stackoverflow about this issue. (Jupyter notebook can’t find kernel when run through /etc/rc.local) but the issue here is that when I type “jupyter kernelspec list”, I only get python3 and python2, even when jupyter has already loaded conda kernels successfully.

python3    /home/ec2-user/anaconda3/share/jupyter/kernels/python3
  python2    /usr/share/jupyter/kernels/python2

Also, I’ve tried changing environmentspecmanager in jupyter config file to see if it solves the issue but it didn’t. (same error log, “couldn’t call conda”)

what is the problem here?

Answer

conda has not been configured yet when rc.local is executed. This is usually done through the .bashrc file. But you can try to just add it to the rc.local script (before launching jupyter):

source /home/ec2-user/anaconda3/etc/profile.d/conda.sh

This should set up the use of conda