Programming Python (18 page)

Example 3-5. PP4E\System\Streams\teststreams.py

"read numbers till eof and show squares"
def interact():
print('Hello stream world')                     # print sends to sys.stdout
while True:
try:
reply = input('Enter a number>')        # input reads sys.stdin
except EOFError:
break                                   # raises an except on eof
else:                                       # input given as a string
num = int(reply)
print("%d squared is %d" % (num, num ** 2))
print('Bye')
if __name__ == '__main__':
interact()                                      # when run, not imported

On Windows
and Unix-like platforms, it’s also possible to send the
standard output of one program to the standard input of another using
the
|
shell character between two
commands. This is usually called a “pipe” operation because the shell
creates a pipeline that connects the output and input of two commands.
Let’s send the output of the Python script to the standard
more
command-line program’s input to see how
this works:

C:\...\PP4E\System\Streams>
python teststreams.py < input.txt | more
Hello stream world
Enter a number>8 squared is 64
Enter a number>6 squared is 36
Enter a number>Bye

Here,
teststreams
’s standard
input comes from a file again, but its output (written by
print
calls) is sent to another program, not
to a file or window. The receiving program is
more
, a standard command-line paging program
available on Windows and Unix-like platforms. Because Python ties
scripts into the standard stream model, though, Python scripts can be
used on both ends. One Python script’s output can always be piped into
another Python script’s input:

C:\...\PP4E\System\Streams>
type writer.py
print("Help! Help! I'm being repressed!")
print(42)
C:\...\PP4E\System\Streams>
type reader.py
print('Got this: "%s"' % input())
import sys
data = sys.stdin.readline()[:-1]
print('The meaning of life is', data, int(data) * 2)
C:\...\PP4E\System\Streams>
python writer.py
Help! Help! I'm being repressed!
42
C:\...\PP4E\System\Streams>
python writer.py | python reader.py
Got this: "Help! Help! I'm being repressed!"
The meaning of life is 42 84

This time, two Python programs are connected. Script
reader
gets input from script
writer
; both scripts simply read and write,
oblivious to stream mechanics. In practice, such chaining of programs
is a simple form of cross-program communications. It makes it easy to
reuse
utilities written to communicate via
stdin
and
stdout
in ways we never anticipated. For
instance, a Python program that sorts
stdin
text could be applied to any data
source we like, including the output of other scripts. Consider the
Python command-line utility scripts in Examples
3-6
and
3-7
which sort and sum lines in the
standard input stream.

import sys                                  # or sorted(sys.stdin)
lines = sys.stdin.readlines()               # sort stdin input lines,
lines.sort()                                # send result to stdout
for line in lines: print(line, end='')      # for further processing

import sys
sum = 0
while True:
try:
line = input()                     # or call sys.stdin.readlines()
except EOFError:                       # or for line in sys.stdin:
break                              # input strips \n at end
else:
sum += int(line)                   # was sting.atoi() in 2nd ed
print(sum)

We can apply such general-purpose tools in a variety of ways at
the shell command line to sort and sum arbitrary files and program
outputs (Windows note: on my prior XP machine and Python 2.X, I had to
type “python file.py” here, not just “file.py,” or else the input
redirection failed; with Python 3.X on Windows 7 today, either form
works):

C:\...\PP4E\System\Streams>
type data.txt
123
000
999
042
C:\...\PP4E\System\Streams>
python sorter.py < data.txt
sort a file
000
042
123
999
C:\...\PP4E\System\Streams>
python adder.py < data.txt
sum file
1164
C:\...\PP4E\System\Streams>
type data.txt | python adder.py
sum type output
1164
C:\...\PP4E\System\Streams>
type writer2.py
for data in (123, 0, 999, 42):
print('%03d' % data)
C:\...\PP4E\System\Streams>
python writer2.py | python sorter.py
sort py output
000
042
123
999
C:\...\PP4E\System\Streams>
writer2.py | sorter.py
shorter form
...same output as prior command on Windows...
C:\...\PP4E\System\Streams>
python writer2.py | python sorter.py | python adder.py
1164

The last command here connects three Python scripts by standard
streams—the output of each prior script is fed to the input of the
next via pipeline shell syntax.

A few coding pointers here: if you look closely, you’ll notice
that
sorter.py
reads all of
stdin
at once with the
readlines
method, but
adder.py
reads one line at a time. If the
input source is another program, some platforms run programs connected
by pipes in
parallel
. On such systems, reading
line by line works better if the data streams being passed are large,
because readers don’t have to wait until writers are completely
finished to get busy processing data. Because
input
just reads
stdin
, the line-by-line scheme used by
adder.py
can always be coded with
manual
sys.stdin
reads too:

C:\...\PP4E\System\Streams>
type adder2.py
import sys
sum = 0
while True:
line = sys.stdin.readline()
if not line: break
sum += int(line)
print(sum)

This version utilizes the fact that
int
allows the digits to be surrounded by
whitespace (
readline
returns a line
including its
\n
, but we don’t have
to use
[:-1]
or
rstrip()
to remove it for
int
). In fact, we can use Python’s more
recent file iterators to achieve the same effect—the
for
loop, for example, automatically grabs
one line each time through when we iterate over a file object directly
(more on file iterators in the next chapter):

C:\...\PP4E\System\Streams>
type adder3.py
import sys
sum = 0
for line in sys.stdin: sum += int(line)
print(sum)

Changing
sorter
to read line
by line this way may not be a big performance boost, though, because
the list
sort
method requires that
the list already be complete. As we’ll see in
Chapter 18
, manually coded sort algorithms are
generally prone to be much slower than the Python list sorting
method.

Interestingly, these two scripts can also be coded in a much
more compact fashion in Python 2.4 and later by using the
new
sorted
built-in
function, generator expressions, and file iterators. The following
work the same way as the originals, with noticeably less source-file
real estate:

C:\...\PP4E\System\Streams>
type sorterSmall.py
import sys
for line in sorted(sys.stdin): print(line, end='')
C:\...\PP4E\System\Streams>
type adderSmall.py
import sys
print(sum(int(line) for line in sys.stdin))

In its argument to
sum
, the
latter of these employs a generator expression, which is much like a
list comprehension, but results are returned one at a time, not in a
physical list. The net effect is space optimization. For more details,
see a core language resource, such as the
book
Learning
Python
.