Run bash command from python and capture file object

in jupyter notebook I can run the following command and get a list of file objects that I can then open:

PATH = someuser/data
files = get_ipython().getoutput('ls {PATH}') #%ls {PATH} in notebook
#then I can run
text = get_ipython().getoutput('less {PATH}{files[0]}')
print(text)

I have tried the following:

path = f"{PATH}"
files = subprocess.call(['ls', path])

files = subprocess.run(['ls', path], stdout=subprocess.PIPE).stdout.decode('utf-8')

files = os.system(path)

But I can only get a string object not file objects that I can then open and get data from.

Answer

Filenames are strings. That’s what you pass to open to get a file object, which you can then get data from. For example, let’s say you have a script called spam.py:

filename = 'spam.py'
with open(filename) as fileobj:
    data = fileobj.read()
print(data)

When you run it, it prints itself out.

Now, the output you got back from calling subprocess.run on ls and reading its stdout is not a string for each file, but one big string.

The simple answer here is to just not call out to ls. Python can already get a list of filenames for you:

filenames = os.listdir(path)
for filename in filenames:
    with open(filename) as fileobj:
        # do stuff with fileobj

But if you really must, you can use, e.g., the splitlines method to turn the output you have into a list of lines, which will be much the same as what listdir returns, just retrieved in a more convoluted way.