Thesys
module is
also where Python makes available the words typed on the
command that is used to start a Python script. These words are usually
referred to as command-line arguments and show up insys.argv
, a built-in
list of strings. C programmers may notice its similarity to the Cargv
array (an array of C strings). It’s not
much to look at interactively, because no command-line arguments are
passed to start up Python in this mode:
>>>import sys
>>>sys.argv
['']
To really see what arguments are about, we need to run a script from
the shell command line.
Example 3-1
shows an unreasonably simple one that just prints theargv
list for inspection.
Example 3-1. PP4E\System\testargv.py
import sys
print(sys.argv)
Running this script prints the command-line arguments list; note
that the first item is always the name of the executed Python script file
itself, no matter how the script was started (see
Executable Scripts on Unix
).
C:\...\PP4E\System>python testargv.py
['testargv.py']
C:\...\PP4E\System>python testargv.py spam eggs cheese
['testargv.py', 'spam', 'eggs', 'cheese']
C:\...\PP4E\System>python testargv.py -i data.txt -o results.txt
['testargv.py', '-i', 'data.txt', '-o', 'results.txt']
The last command here illustrates a common convention. Much like
function arguments, command-line options are sometimes passed by position
and sometimes by name using a “-name value” word pair. For instance, the
pair-i data.txt
means the-i
option’s value isdata.txt
(e.g., an input filename). Any
words can be listed, but programs usually impose some sort of structure on
them.
Command-line arguments play the same role in programs that function
arguments do in functions: they are simply a way to pass information to a
program that can vary per program run. Because they don’t have to be
hardcoded, they allow scripts to be more generally useful. For example, a
file-processing script can use a command-line argument as the name of the
file it should process; see
Chapter 2
’s
more.py
script (
Example 2-1
) for a prime example. Other
scripts might accept processing mode flags, Internet addresses, and so
on.
Once you
start using command-line arguments regularly, though,
you’ll probably find it inconvenient to keep writing code that fishes
through the list looking for words. More typically, programs translate
the arguments list on startup into structures that are more conveniently
processed. Here’s one way to do it: the script in
Example 3-2
scans theargv
list looking for-optionname optionvalue
word pairs and stuffs
them into a dictionary by option name for easy retrieval.
Example 3-2. PP4E\System\testargv2.py
"collect command-line options in a dictionary"
def getopts(argv):
opts = {}
while argv:
if argv[0][0] == '-': # find "-name value" pairs
opts[argv[0]] = argv[1] # dict key is "-name" arg
argv = argv[2:]
else:
argv = argv[1:]
return opts
if __name__ == '__main__':
from sys import argv # example client code
myargs = getopts(argv)
if '-i' in myargs:
print(myargs['-i'])
print(myargs)
You might import and use such a function in all your command-line
tools. When run by itself, this file just prints the formatted argument
dictionary:
C:\...\PP4E\System>python testargv2.py
{}
C:\...\PP4E\System>python testargv2.py -i data.txt -o results.txt
data.txt
{'-o': 'results.txt', '-i': 'data.txt'}
Naturally, we could get much more sophisticated here in terms of
argument patterns, error checking, and the like. For more complex
command lines, we could also use command-line processing tools in the
Python standard library to parse arguments:
Thegetopt
module,
modeled after a Unix/C utility of the same name
Theoptparse
module, a
newer alternative, generally considered to be more
powerful
Both of these are documented in Python’s library manual, which
also provides usage examples which we’ll defer to here in the interest
of space. In general, the more configurable your scripts, the more you
must invest in command-line processing logic
complexity
.
Executable Scripts on Unix
Unix and Linux users:
you can also make text files of Python source code
directly executable by adding a special line at the top with the path
to the Python interpreter and giving the file executable permission.
For instance, type this code into a text file called
myscript
:
#!/usr/bin/python
print('And nice red uniforms')
The first line is normally taken as a comment by Python (it
starts with a#
); but when this
file is run, the operating system sends lines in this file to the
interpreter listed after#!
in line
1. If this file is made directly executable with a shell command of
the formchmod +x myscript
, it can
be run directly without typingpython
in the command, as though it were a
binary executable program:
%myscript a b c
And nice red uniforms
When run this way,sys.argv
will still have the script’s name as the first word in the list:["myscript", "a", "b", "c"]
,
exactly as if the script had been run with the more explicit and
portable command formpython myscript a b
. Making scripts directly executable is actually a Unix
c
trick, not a Python feature, but it’s worth pointing out that it can
be made a bit less machine dependent by listing the Unixenv
command at the top instead of a
hardcoded path to the Python executable:
#!/usr/bin/env python
print('Wait for it...')
When coded this way, the operating system will employ your
environment variable settings to locate your Python interpreter (yourPATH
variable, on most platforms).
If you run the same script on many machines, you need only change your
environment settings on each machine (you don’t need to edit Python
script code). Of course, you can always run Python files with a more
explicit command line:
%python myscript a b c
This assumes that thepython
interpreter
program is on your system’s search path setting (otherwise, you need
to type its full path), but it works on any Python platform with a
command line. Since this is more portable, I generally use this
convention in the book’s examples, but consult your Unix manpages for
more details on any of the topics mentioned here. Even so, these
special#!
lines will show up in
many examples in this book just in case readers want to run them as
executables on Unix or Linux; on other platforms, they are simply
ignored as Python comments.
Note that on recent flavors of Windows, you can usually also
type a script’s filename directly (without the word
python
) to make it go, and you don’t have to add
a#!
line at the top. Python uses
the Windows registry on this platform to declare itself as the program
that opens files with Python extensions (
.py
and
others). This is also why you can launch files on Windows by clicking
on them.
Shell variables,
sometimes known as environment variables, are made available
to Python scripts asos.environ
, a Python
dictionary-like object with one entry per variable setting in the shell.
Shell variables live outside the Python system; they are often set at your
system prompt or within startup files or control-panel GUIs and typically
serve as system-wide configuration inputs to programs.
In fact, by now you should be familiar with a prime example:
thePYTHONPATH
module search
path setting is a shell variable used by Python to import modules. By
setting it once in your operating system, its value is available every
time a Python program is run. Shell variables can also be set by programs
to serve as inputs to other programs in an application; because their
values are normally inherited by spawned programs, they can be used as a
simple form of interprocess communication.
In Python,
the surrounding shell environment becomes a simple preset
object, not special syntax. Indexingos.environ
by the desired shell variable’s
name string (e.g.,os.environ['USER']
) is the moral equivalent of
adding a dollar sign before a variable name in most Unix shells (e.g.,$USER
), using surrounding percent
signs on DOS (%USER%
), and callinggetenv("USER")
in a C program. Let’s start up
an interactive session to experiment (run in Python 3.1 on a Windows 7
laptop):
>>>import os
>>>os.environ.keys()
KeysView()
>>>list(os.environ.keys())
['TMP', 'COMPUTERNAME', 'USERDOMAIN', 'PSMODULEPATH', 'COMMONPROGRAMFILES',
...many more deleted...
'NUMBER_OF_PROCESSORS', 'PROCESSOR_LEVEL', 'USERPROFILE', 'OS', 'PUBLIC', 'QTJAVA']
>>>os.environ['TEMP']
'C:\\Users\\mark\\AppData\\Local\\Temp'
Here, thekeys
method returns
an iterable of assigned variables, and indexing fetches the value of the
shell variableTEMP
on Windows. This
works the same way on Linux, but other variables are generally preset
when Python starts up. Since we know aboutPYTHONPATH
, let’s peek at its setting within
Python to verify its content (as I wrote this, mine was set to the root
of the book examples tree for this fourth edition, as well as a
temporary development location):
>>>os.environ['PYTHONPATH']
'C:\\PP4thEd\\Examples;C:\\Users\\Mark\\temp'
>>>for srcdir in os.environ['PYTHONPATH'].split(os.pathsep):
...print(srcdir)
...
C:\PP4thEd\Examples
C:\Users\Mark\temp
>>>import sys
>>>sys.path[:3]
['', 'C:\\PP4thEd\\Examples', 'C:\\Users\\Mark\\temp']
PYTHONPATH
is a string of
directory paths separated by whatever character is used to separate
items in such paths on your platform (e.g.,;
on DOS/Windows,:
on Unix and Linux). To split it into its
components, we pass to thesplit
string method anos.pathsep
delimiter—a portable setting that gives the proper separator for the
underlying machine. As usual,sys.path
is the actual search path at runtime,
and reflects the result of merging in thePYTHONPATH
setting after the current
directory.
Like normal dictionaries,
theos.environ
object
supports both key indexing and
assignment
. As for
dictionaries, assignments change the value of the key:
>>>os.environ['TEMP']
'C:\\Users\\mark\\AppData\\Local\\Temp
>>>os.environ['TEMP'] = r'c:\temp'
>>>os.environ['TEMP']
'c:\\temp'
But something extra happens here. In all recent Python releases,
values assigned toos.environ
keys in
this fashion are automatically
exported
to other
parts of the application. That is, key assignments change both theos.environ
object in the Python
program as well as the associated variable in the enclosing
shell
environment of the running program’s process.
Its new value becomes visible to the Python program, all linked-in C
modules, and any programs spawned by the Python process.
Internally, key assignments toos.environ
callos.putenv
—a function that changes the shell
variable outside the boundaries of the Python interpreter. To
demonstrate how this works, we need a couple of scripts that set and
fetch shell variables; the first is shown in
Example 3-3
.
Example 3-3. PP4E\System\Environment\setenv.py
import os
print('setenv...', end=' ')
print(os.environ['USER']) # show current shell variable value
os.environ['USER'] = 'Brian' # runs os.putenv behind the scenes
os.system('python echoenv.py')
os.environ['USER'] = 'Arthur' # changes passed to spawned programs
os.system('python echoenv.py') # and linked-in C library modules
os.environ['USER'] = input('?')
print(os.popen('python echoenv.py').read())
This
setenv.py
script simply changes a shell
variable,USER
, and spawns another
script that echoes this variable’s value, as shown in
Example 3-4
.
Example 3-4. PP4E\System\Environment\echoenv.py
import os
print('echoenv...', end=' ')
print('Hello,', os.environ['USER'])
No matter how we run
echoenv.py
, it displays
the value ofUSER
in the enclosing
shell; when run from the command line, this value comes from whatever
we’ve set the variable to in the shell itself:
C:\...\PP4E\System\Environment>set USER=Bob
C:\...\PP4E\System\Environment>python echoenv.py
echoenv... Hello, Bob
When spawned by another script such as
setenv.py
using theos.system
andos.popen
tools we met earlier, though,
echoenv.py
gets whateverUSER
settings its parent program has
made:
C:\...\PP4E\System\Environment>python setenv.py
setenv... Bob
echoenv... Hello, Brian
echoenv... Hello, Arthur
?Gumby
echoenv... Hello, Gumby
C:\...\PP4E\System\Environment>echo %USER%
Bob
This works the same way on Linux. In general terms, a spawned
program always
inherits
environment settings from
its parents.
Spawned
programs are
programs started with Python tools such asos.spawnv
, theos.fork/exec
combination on Unix-like
platforms, andos.popen
,os.system
, and thesubprocess
module on a variety of platforms.
All programs thus
launched get the environment variable settings that exist
in the parent at launch time.
[
7
]
From a larger perspective, setting shell variables like this
before starting a new program is one way to pass information into the
new program. For instance, a Python configuration script might tailor
thePYTHONPATH
variable to include
custom directories just before launching another Python script; the
launched script will have the custom search path in itssys.path
because shell variables are passed
down to children (in fact, watch for such a launcher script to appear at
the end of
Chapter 6
).
Notice the last command in the preceding example—theUSER
variable is back to its original value
after the top-level Python program exits. Assignments toos.environ
keys are passed outside the
interpreter and
down
the spawned programs chain,
but never back
up
to parent program processes
(including the system shell). This is also true in C programs that use
theputenv
library call, and it isn’t
a Python limitation per se.
It’s also likely to be a nonissue if a Python script is at the top
of your application. But keep in mind that shell settings made within a
program usually endure only for that program’s run and for the run of
its spawned children. If you need to export a shell variable setting so
that it lives on after Python exits, you may be able to find
platform-specific extensions that do this; search
http://www.python.org
or the Web at large.
Another subtlety: as implemented today, changes toos.environ
automatically callos.putenv
, which runs
theputenv
call in the C library if
it is available on your platform to export the setting outside Python to
any linked-in C code. However, althoughos.environ
changes callos.putenv
, direct calls toos.putenv
do not updateos.environ
to reflect the change. Because of
this, theos.environ
mapping
interface is generally preferred toos.putenv
.
Also note that environment
settings are loaded intoos.environ
on startup and not on each fetch;
hence, changes made by linked-in C code after startup may not be
reflected inos.environ
. Python does
have a more focusedos.getenv
call today,
but it is simply translated into anos.environ
fetch on most platforms (or all, in
3.X), not into a call togetenv
in
the C library. Most applications won’t need to care, especially if they
are pure Python code. On platforms without aputenv
call,os.environ
can be passed as a parameter to
program startup tools to set the spawned
program’s environment.
[
7
]
This is by default. Some program-launching tools also let
scripts pass environment settings that are different from their own
to child programs. For instance, theos.spawnve
call is likeos.spawnv
, but it accepts a dictionary
argument representing the shell environment to be passed to the
started program. Someos.exec*
variants (ones with an “e” at the end of their names) similarly
accept explicit environments; see theos.exec*
call formats in
Chapter 5
for more details.