Executing several commands at once [closed]

I am trying to write something in Perl (but I’m also happy with Python or PHP to do the same) which:

• Takes a list (of say, for example, files or URLs)

• Adds an additional system command (say cp as an example)

• Executes a maximum of ten these commands in the list of, say, a thousand in total

• Waits for one of the ten slots to become and starts a new procedure, thus moving through the 1000 elements.

I’m short-cutting right now and using back ticks to execute the commands in Perl, but if there is a better way to achieve this, I’d be really appreciative.

Answer

You should show what you have written, or we have no way of knowing what might be a “better way”

You should probably take a look at Parallel::ForkManager. Using that module your program could look something like this

There’s a lot missing from this code, most importantly use strict and use warnings 'all', and the declaration and population of @list. I trust that you can fill in the blanks

Note that the limit on ten parallel process is imposed by the module, and that "..." must be filled in by yourself as the destination path. It’s always best to do what you need within Perl instead of shelling out just to call cp, so I’ve involved File::Copy instead

use Parallel::ForkManager ();
use File::Copy 'copy';

my $pm = Parallel::ForkManager->new(10);

for my $file ( @list ) {

    next if my $pid = $pm->start;

    # Child code

    copy $file, "...";
    $pm->finish;
}