Programming Python (16 page)

That’s as much of a tour around
os
as we have space for here. Since most other
os
module tools are even more
difficult to appreciate outside the context of larger application
topics, we’ll postpone a deeper look at them until later chapters. But
to let you sample the flavor of this module, here is a quick preview for
reference. Among the
os
module’s
other weapons are these:

And so on. One caution up front: the
os
module provides a set of file
open
,
read
,
and
write
calls, but all of these
deal with low-level file access and are entirely distinct from Python’s
built-in
stdio
file objects that we
create with the built-in
open
function. You should normally use the built-in
open
function, not the
os
module, for all but very special
file-processing needs (e.g., opening with exclusive access file
locking).

In the next chapter we will apply
sys
and
os
tools such as those we’ve introduced here to implement common
system-level tasks, but this book doesn’t have space to provide an
exhaustive list of the contents of modules we will meet along the way.
Again, if you have not already done so, you should become acquainted
with the contents of modules such as
os
and
sys
using the resources described earlier. For now, let’s move on to explore
additional system tools in the context of broader system programming
concepts—
the context surrounding a
running
script.

>>>
import os
>>>
for line in os.popen('dir /B *.py'): print(line, end='')
...
helloshell.py
more.py
__init__.py

>>>
I = os.popen('dir /B *.py')
>>>
I

>>>
I = os.popen('dir /B *.py')
>>>
I.__next__()
'helloshell.py\n'
>>>
I = os.popen('dir /B *.py')
>>>
next(I)
TypeError: _wrap_close object is not an iterator

>>>
I = os.popen('dir /B *.py'
)
>>>
I = iter(I)
# what for loops do
>>>
I.__next__()
# now both forms work
'helloshell.py\n'
>>>
next(I)
'more.py\n'

Python scripts don’t run in a vacuum (despite what you may have
heard). Depending on platforms and startup procedures, Python programs may
have all sorts of enclosing context—information automatically passed in to
the program by the operating system when the program starts up. For
instance, scripts have access to the following sorts of system-level
inputs and interfaces:

Such tools can serve as inputs to scripts, configuration parameters,
and so on. In this chapter, we will explore all these four context’s
tools—both their Python interfaces and their typical roles.

When you run a
Python script by typing a shell command line such as
python dir1\dir2\file.py
, the CWD is
the directory you were in when you typed this command, not
dir1
\
dir2
. On the other hand,
Python automatically adds the identity of the script’s home directory to
the front of the module search path such that
file.py
can always import other files in
dir1
\
dir2
no matter where it
is run from. To illustrate, let’s write a simple script to echo both its
CWD and its module search path:

C:\...\PP4E\System>
type whereami.py
import os, sys
print('my os.getcwd =>', os.getcwd())           # show my cwd execution dir
print('my sys.path  =>', sys.path[:6])          # show first 6 import paths
input()                                         # wait for keypress if clicked

Now, running this script in the directory in which it resides sets
the CWD as expected and adds it to the front of the module import search
path. We met the
sys.path
module search
path earlier; its first entry might also be the empty string to
designate CWD when you’re working interactively, and most of the CWD has
been truncated to “...” here for display:

C:\...\PP4E\System>
set PYTHONPATH=C:\PP4thEd\Examples
C:\...\PP4E\System>
python whereami.py
my os.getcwd => C:\...\PP4E\System
my sys.path  => ['C:\\...\\PP4E\\System', 'C:\\PP4thEd\\Examples',
...more...
]

But if we run this script from other places, the CWD moves with us
(it’s the directory where we type commands), and Python adds a directory
to the front of the module search path that allows the script to still
see files in its own home directory. For instance, when running from one
level up (
..
), the
System
name added to the front of
sys.path
will be the first directory that
Python searches for imports within
whereami.py
; it
points imports back to the directory containing the script that was run.
Filenames without complete paths, though, will be mapped to the CWD
(
C:\PP4thEd\Examples\PP4E
), not the
System
subdirectory nested there:

C:\...\PP4E\System>
cd ..
C:\...\PP4E>
python System\whereami.py
my os.getcwd => C:\...\PP4E
my sys.path  => ['C:\\...\\PP4E\\System', 'C:\\PP4thEd\\Examples',
...more...
]
C:\...\PP4E>
cd System\temp
C:\...\PP4E\System\temp>
python ..\whereami.py
my os.getcwd => C:\...\PP4E\System\temp
my sys.path  => ['C:\\...\\PP4E\\System', 'C:\\PP4thEd\\Examples', ...]

The net effect is that filenames without
directory paths in a script will be mapped to the place
where the
command
was typed (
os.getcwd
), but imports still have access to
the directory of the
script
being run (via the
front of
sys.path
). Finally, when a
file is launched by clicking its icon, the CWD is just the directory
that contains the clicked file. The following output, for example,
appears in a new DOS console box when
whereami.py
is double-clicked in Windows Explorer:

my os.getcwd => C:\...\PP4E\System
my sys.path  => ['C:\\...\\PP4E\\System',
...more...
]

In this case, both the CWD used for filenames and the first import
search directory are the directory containing the script file. This all
usually works out just as you expect, but there are two pitfalls to
avoid:

For example, scripts in this book, regardless of how they are run,
can always import other files in their own home directories without
package path imports (
import
filehere
), but must go through the PP4E package root to find
files anywhere else in the examples tree (
from
PP4E.dir1.dir2 import filethere
), even if they are run from
the directory containing the desired external module. As usual for
modules, the
PP4E\dir1\dir2
directory name could
also be added to
PYTHONPATH
to make
files there visible everywhere without package path imports (though
adding more directories to
PYTHONPATH
increases the likelihood of name clashes). In either case, though,
imports are always resolved to the script’s home directory or other
Python search path settings, not to the CWD.

This distinction
between the CWD and import search paths explains why many
scripts in this book designed to operate in the current working
directory (instead of one whose name is passed in) are run with command
lines such as this one:

C:\temp>
python C:\...\PP4E\Tools\cleanpyc.py
process cwd

In this example, the Python script file itself lives in the
directory
C:\...\PP4E\Tools
, but because it is run
from
C:\temp
, it processes the files located in
C:\temp
(i.e., in the CWD, not in the script’s home
directory). To process files elsewhere with such a script, simply
cd
to the directory to be processed
to change the CWD:

C:\temp>
cd C:\PP4thEd\Examples
C:\PP4thEd\Examples>
python C:\...\PP4E\Tools\cleanpyc.py
process cwd

Because the CWD is always implied, a
cd
command tells the
script which directory to process in no less certain terms than passing
a directory name to the script explicitly, like this (portability note:
you may need to add quotes around the
*.py
in this and other command-line examples
to prevent it from being expanded in some Unix shells):

C:\...\PP4E\Tools>
python find.py *.py C:\temp
process named dir

In this command line, the CWD is the directory containing the
script to be run (notice that the script filename has no directory path
prefix); but since this script processes a directory named explicitly on
the command line (
C:\temp
), the CWD is irrelevant.
Finally, if we want to run such a script located in some other directory
in order to process files located in yet another directory, we can
simply give directory paths to both:

C:\temp>
python C:\...\PP4E\Tools\find.py *.cxx C:\PP4thEd\Examples\PP4E

Here, the script has import visibility to files in its
PP4E\Tools
home directory and processes files in
the directory named on the command line, but the CWD is something else
entirely (
C:\temp
). This last form is more to type,
of course, but watch for a variety of CWD and explicit script-path
command lines like these in this
book.