Traversing a directory using Makefile

I am making use of a makefile to traverse a directory , store the file names in a variable , extract the file number using regex and print the file number .

Below is the code block I am using :

1. set -e ;
2. for file in $$(MY_HOME)/mydir/python_files/ ;
3. do 
4. string =$${file} ;
5. [[ $$string =~ .*clause([0-9]*).py ]] ; 
6. file_num=$${BASH_REMATCH[1]} ; 
7. python $$(My_Home)/base_py_<file_num>.py ;
8. done ;

LINE 7 is the command in makefile I want to include in the for loop. I want the actual file_num in the above < file_num > placeholder (line 7). How can I do the same. Is there any alternative approach for the same ?

Thanks in advance

Answer

I would make a native make approach instead of looping in bash, like so:

$ cat Makefile
MY_HOME := myhome
CLAUSE_FILES_DIR := $(MY_HOME)/mydir/python_files

clause_files := $(wildcard $(CLAUSE_FILES_DIR)/*clause*.py)
clause_numbers := $(foreach clause_file,$(notdir $(clause_files:.py=)), 
                    $(lastword $(subst clause, ,$(clause_file))))

.PHONY: execute-clause-%
execute-clause-%: $(MY_HOME)/base_py_%.py $(CLAUSE_FILES_DIR)/*clause%.py
        echo python $<

all: $(addprefix execute-clause-,$(clause_numbers))

clause_files will keep all existing files matching the pattern. clause_numbers will process the file names by stripping extension and directory, then split on clause to get only the part between clause and extension.

execute-clause-% is a generic rule to run based on existence of a specific base_py_*.py script and a matching clause file. If one or the other does not exist, the rule will not be run.

Finally all rule executes all existing clauses. And since every processing is done by a separate rule, all of them might be executed in parallel by just running make -j.

Sample output:

## Preparation stage
$ mkdir -p myhome/mydir/python_files
$ for i in `seq 1 5`; do touch myhome/base_py_$i.py; done
$ for i in `seq 1 5`; do touch myhome/mydir/python_files/${RANDOM}_clause$i.py; done
$ touch myhome/mydir/python_files/foo.py  # Just to show it does not match
$ touch myhome/base_py_100.py             # To demonstrate missing clause file

$ ls -R myhome/
myhome/:
base_py_1.py  base_py_100.py  base_py_2.py  base_py_3.py  base_py_4.py  base_py_5.py  mydir

myhome/mydir:
python_files

myhome/mydir/python_files:
14363_clause1.py  31198_clause2.py  4514_clause5.py  4767_clause4.py  7812_clause3.py  foo.py

## Execution
$ make -s
python myhome/base_py_3.py
python myhome/base_py_2.py
python myhome/base_py_5.py
python myhome/base_py_4.py
python myhome/base_py_1.py

Note that neither foo.py nor base_py_100.py did not cause running the rule.