Read from stdin in Python Process? -
i'm trying read sys.stdin inside of python process object, keep getting "valueerror: i/o operation on closed file" result. here's quick example:
import sys multiprocessing import process def do_something(input_data): x in input_data: print x input=sys.stdin p = process(target=do_something, args=(input,)) p.start() p.join() #wait process complete
the above script fails with:
traceback (most recent call last): file "/usr/local/cellar/python/2.7.5/frameworks/python.framework/versions/2.7/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() file "/usr/local/cellar/python/2.7.5/frameworks/python.framework/versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) file "example.py", line 6, in do_something x in input_data: valueerror: i/o operation on closed file
just calling do_something(input)
works fine without using process, of course. creating pipe()
object seems - can write contents of stdin pipe , results in string form within process - need input file-like form downstream operations. dump contents file , re-read in within process, seems pretty clumsy, if stdin big. there easy way read sys.stdin within process?
this because before process started, stdin
closed. otherwise happen both parent , child process (or multiple child processes) try read same stdin, bad idea.
in child process sys.stdin
redirected /dev/null
:
from multiprocessing import process import sys def test(*args): print(args) print(sys.stdin, sys.stdin.fileno()) if __name__ == '__main__': p = process(target=test, args=(sys.stdin,)) p.start() p.join()
should print similar this:
(<closed file '<stdin>', mode 'r' @ 0x7f3b4564b0c0>,) (<open file '/dev/null', mode 'r' @ 0x7f3b43a9e0c0>, 3)
the passed argument here reference closed file object, trying use raise error you've seen.
you around using os.dup()
on sys.stdin.fileno()
in parent , pass returned copy of file descriptor child argument, can use os.fdopen()
work it.
the cleaner solution read input in parent process , pass child using multiprocessing.queue
.
Comments
Post a Comment