I am new to scientific computing and am working on a project which involves running many simulations. The infrastructure with which I am provided for this is about 20 different linux machines which all share storage. What I’ve been doing so far is the horrendously inefficient task of ssh’ing into each computer individually and running the simulation programs manually.
Given that I have a list of all the names of the computers and the same password for each one, could someone provide a sparse outline of how to go about automatically running many executables on many linux machines? An example of what I must do right now for each simulation is:
$ ssh username@computername.university.edu username@computername.university.edu's password: username@computername:~$ ./executeprogram 2 5 8 & username@computername:~$ ./executeprogram 2 5 10 & username@computername:~$ ./executeprogram 2 5 12 &
Where the numbers are just arguments the simulation executeprogram in the home directory takes in. Hopefully you can appreciate that for 20 different computername‘s this is pretty tedious to do by hand. Apologies if it is unclear what I am asking; I ask here because I do not have the vocabulary to know what is I should be googling to find the solution to this problem.
Advertisement
Answer
There are few things to this question.
First step would be to install your ssh key on those machines to avoid having to type the password every time. Assuming you’ve already setup a ssh key:
ssh-copy-id username@computername.university.edu
Then, you don’t need to ssh into the machine to run commands. You can pass it as part of the initial ssh call. e.g.:
ssh username@computername.university.edu ./executeprogram 2 5 8
But if executeprogram is long-running, this might not be ideal as it would wait for it to finish before exiting the ssh connection. There are a few ways around this
Now all thats left would be to write the above into a single script.