python subprocess.stdout.readline doesn’t

I have a piece of python code that calls a binary via the subprocess module. This binary reads data from stdin and outputs other data to stdout, which should then be collected by the python code.

Here’s how I use this binary in a shell:

$ ./check # start reading from stdin here
param_card_orig.dat
0.123809
42.821 0 0 42.821 
91.2246 0 0 -91.2246 
14.5521 -12.9194 -0.441262 6.68258 
33.8782 -4.04952 5.37574 -33.2029 
35.3909 22.0683 2.04142 27.5923 
50.2244 -5.09935 -6.9759 -49.4755 

with a return after each line (including the last), from which I get the output

1.53852538054399053E-004

which is again terminated by a newline. However, trying to put this in python code like this:

import subprocess

p = subprocess.Popen("./check", stdout=subprocess.PIPE, stdin=subprocess.PIPE)
p.stdin.write("param_card_orig.datn")
p.stdin.write("0.123809n")
p.stdin.write("42.821 0 0 42.821 n")
p.stdin.write("91.2246 0 0 -91.2246 n")
p.stdin.write("14.5521 -12.9194 -0.441262 6.68258 n")
p.stdin.write("33.8782 -4.04952 5.37574 -33.2029 n")
p.stdin.write("35.3909 22.0683 2.04142 27.5923 n")
p.stdin.write("50.2244 -5.09935 -6.9759 -49.4755 n")

print("now waiting for input")

print(p.stdout.readline())

I receive exactly this:

now waiting for input

The program is hanging, obviously waiting for some sort of communication to happen. This is partially intended – the check binary is supposed to stay active and wait for further input, returning the output of the calculation as soon as a further complete input is supplied – but since the first input is complete, I should have received the result of the calculation, which I obviously didn’t, or else we would have seen the printout. When I break the execution manually with CTRL+C, it tells me

^CTraceback (most recent call last):
File "testpipe.py", line 15, in <module>
print(p.stdout.readline())

What’s going on here? Is this a problem with the way I handle the pipes? Is this a problem with a missing EOF or missing newline?

PS: I found this issue Python subprocess.stdout.readline() hang, which is possibly the same, but didn’t receive an answer.

PPS: In case it matters, the python version is Python 2.6.6.

Answer

Have you tried using p.stdout.read() instead of readline()? It works for the following test code where invoker.py creates a pipe with invoked.py.

invoked.py

#!/usr/bin/python
import sys

# Print a Hello notice
sys.stdout.write("I have been invoked!n")

# Read the value coming from the pipe
value = sys.stdin.readline()

# Return a "processed" value
sys.stdout.write("New: " + value)

invoker.py

import subprocess

# Start pipe with other file
p = subprocess.Popen("./invoked.py", stdout=subprocess.PIPE, stdin=subprocess.PIPE)

# Send a value to the other file
p.stdin.write("Value!n")

# Print the result coming from the other file
print(p.stdout.read())