I wrote a python script that uses numpy, multiprocessing, tqdm and a feq other Python libraries. Additionally, I run packages (e.g. samtools, bwa, GATK) set are necessary to be installed in linux (apt-get install).
I’d like to somehow wrap all these dependencies up to make the final installation as user-friendly and stable as possible.
It seems as pip is not an option here as non-python-packages are included for my example.
Maybe Docker or creating a conda environment with all these dependecies might be possible but I did not really get how to manage this.
Advertisement
Answer
I wouldn’t go as far as installing software onto some computer/server because it might break other software running on that system. Instead, I would create a deb package and set dependencies on which software has to be installed to guide the user through installation.
1. Create the deb directory structure
The directory deb holds final scripts/binaries for the package. source has your python code.
$ cd $HOME/path/to/some/directory
$ mkdir -p ./deb/DEBIAN ./source
$ touch ./Makefile ./source/Makefile ./deb/DEBIAN/{control,preinst} ./source/hello-world.py
$ chmod +x ./deb/DEBIAN/preinst
2. Add files
Below I’ve added the content of every file. You should be able to implement it to your needs.
./Makefile: This file is responsible for generating the source and the deb.
PACKAGE := hello-world
DEB_DIR := deb
DEB_FILE := $(PACKAGE).deb
SOURCE_DIR := source
SOURCE_IN := $(SOURCE_DIR)/$(PACKAGE).py
SOURCE_OUT := $(DEB_DIR)/usr/bin/
.PHONY: all clean
all:
@make -C $(SOURCE_DIR) CROSS_COMPILE=$(CROSS_COMPILE)
@mkdir -p $(SOURCE_OUT)
@cp -r $(SOURCE_IN) $(SOURCE_OUT)
@chmod -R a-s $(DEB_DIR)
@dpkg-deb --build $(DEB_DIR) $(DEB_FILE)
clean:
@rm -rf $(DEB_FILE) $(SOURCE_OUT)
@make -C $(SOURCE_DIR) clean
./deb/DEBIAN/control: The control file for Debian’s package manager.
Package: hello-world Version: 1 Section: misc Priority: optional Architecture: all Depends: libc6 (>= 2.24), python (>=2.7.13), virtualenv (>=1.11.6) Maintainer: Your name <your-name@domain.org> Description: This is a deb package to install an application
./deb/DEBIAN/preinst: The file in which you can install python packages. It is run before the installation of hello-world.py. Again, make sure you don’t just (re)install/update a library. It could cause incompatibility problems in other software.
#! /bin/bash virtualenv -p /usr/bin/python3 venv/ source venv/bin/activate pip install -r ./requirements.txt
./source/Makefile: A file I used to test this setup.
MYFILE=hello-world.py
all:
@echo "#!/usr/bin/pythonn
print('Hello world!')" > $(MYFILE)
@chmod +x $(MYFILE)
clean:
@rm $(MYFILE)
3. Installation and removal
Installing is very easy, just use dpkg.
dpkg -i hello-world.deb # for installation dpkg -r hello-world # for removal
A nice feature to have is you don’t need to worry about versioning your software, since it’s done in the control file. Debian has a pretty good tutorial if you’re interested in reading more.
EDIT:
Added virtualenv to example. Python packages should be installed in a virtual environment instead of globally.