Our next example is a C extension module that integrates the
standard C
library’sgetenv
andputenv
shell environment variable calls
for use in Python scripts.
Example 20-8
is a C file that achieves
this goal in a hand-coded, manual fashion.
Example 20-8. PP4E\Integrate\Extend\Cenviron\cenviron.c
/******************************************************************
* A C extension module for Python, called "cenviron". Wraps the
* C library's getenv/putenv routines for use in Python programs.
******************************************************************/
#include
#include
#include
/***********************/
/* 1) module functions */
/***********************/
static PyObject * /* returns object */
wrap_getenv(PyObject *self, PyObject *args) /* self not used */
{ /* args from python */
char *varName, *varValue;
PyObject *returnObj = NULL; /* null=exception */
if (PyArg_Parse(args, "(s)", &varName)) { /* Python -> C */
varValue = getenv(varName); /* call C getenv */
if (varValue != NULL)
returnObj = Py_BuildValue("s", varValue); /* C -> Python */
else
PyErr_SetString(PyExc_SystemError, "Error calling getenv");
}
return returnObj;
}
static PyObject *
wrap_putenv(PyObject *self, PyObject *args)
{
char *varName, *varValue, *varAssign;
PyObject *returnObj = NULL;
if (PyArg_Parse(args, "(ss)", &varName, &varValue))
{
varAssign = malloc(strlen(varName) + strlen(varValue) + 2);
sprintf(varAssign, "%s=%s", varName, varValue);
if (putenv(varAssign) == 0) {
Py_INCREF(Py_None); /* C call success */
returnObj = Py_None; /* reference None */
}
else
PyErr_SetString(PyExc_SystemError, "Error calling putenv");
}
return returnObj;
}
/**************************/
/* 2) registration table */
/**************************/
static PyMethodDef cenviron_methods[] = {
{"getenv", wrap_getenv, METH_VARARGS, "getenv doc"}, /* name, &func,... */
{"putenv", wrap_putenv, METH_VARARGS, "putenv doc"}, /* name, &func,... */
{NULL, NULL, 0, NULL} /* end of table marker */
};
/*************************/
/* 3) module definition */
/*************************/
static struct PyModuleDef cenvironmodule = {
PyModuleDef_HEAD_INIT,
"cenviron", /* name of module */
"cenviron doc", /* module documentation, may be NULL */
−1, /* size of per-interpreter module state, −1=in global vars */
cenviron_methods /* link to methods table */
};
/*************************/
/* 4) module initializer */
/*************************/
PyMODINIT_FUNC
PyInit_cenviron() /* called on first import */
{ /* name matters if loaded dynamically */
return PyModule_Create(&cenvironmodule);
}
Though demonstrative, this example is arguably less useful now than
it was in the first edition of this book—as we learned in
Part II
, not only can you fetch shell environment
variables by indexing theos.environ
table, but
assigning to a key in this table automatically calls C’sputenv
to export the new setting to the C code
layer in the process. That is,os.environ['key']
fetches the value of the shell
variable'key'
, andos.environ
['key']
=value
assigns a variable both in Python
and in C.
The second action—pushing assignments out to C—was added to Python
releases after the first edition of this book was published. Besides
illustrating additional extension coding techniques, though, this example
still serves a practical purpose: even today, changes made to shell
variables by the C code linked into a Python process are not picked up
when you indexos.environ
in Python
code. That is, once your program starts,os.environ
reflects only subsequent changes made
by Python code in the process.
Moreover, although Python now has both aputenv
and agetenv
call in itsos
module, their integration seems incomplete.
Changes toos.environ
callos.putenv
,
but direct calls toos.putenv
do not updateos.environ
, so the two can become out of sync.
Andos.getenv
today simply translates
to anos.environ
fetch, and hence will
not pick up environment changes made in the process outside of Python code
after startup time. This may rarely, if ever, be an issue for you, but
this C extension module is not completely without purpose; to truly
interface environment variables with linked-in C code, we need to call the
C library routines directly (at least until Python changes this
model again!
).
The
cenviron.c
C file in
Example 20-8
creates a Python module
calledcenviron
that does a bit
more than the prior examples—it exports two functions, sets some exception
descriptions explicitly, and makes a reference count call for the PythonNone
object (it’s not created anew, so
we need to add a reference before passing it to Python). As before, to add
this code to Python, compile and link into an object file; the Cygwin
makefile in
Example 20-9
builds
the C source code for dynamic binding on imports.
Example 20-9. PP4E\Integrate\Extend\Cenviron\makefile.cenviron
##################################################################
# Compile cenviron.c into cenviron.dll--a shareable object file
# on Cygwin, which is loaded dynamically when first imported.
##################################################################
PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
cenviron.dll: cenviron.c
gcc cenviron.c -g -I$(PYINC) -shared -L$(PYLIB) -lpython3.1 -o [email protected]
clean:
rm -f *.pyc cenviron.dll
To build, typemake -f
at your shell. To run, make sure the resulting
makefile.cenviron
.dll
file is in a directory on Python’s module path
(the current working directory works too):
.../PP4E/Integrate/Extend/Cenviron$python
>>>import cenviron
>>>cenviron.getenv('USER')
# like os.environ[key] but refetched
'mark'
>>>cenviron.putenv('USER', 'gilligan')
# like os.environ[key]=value
>>>cenviron.getenv('USER')
# C sees the changes too
'gilligan'
As before,cenviron
is a bona
fide Python module object after it is imported, with all the usual
attached information, and errors are raised and reported correctly on
errors:
>>>dir(cenviron)
['__doc__', '__file__', '__name__', '__packge__', 'getenv', 'putenv']
>>>cenviron.__file__
'cenviron.dll'
>>>cenviron.__name__
'cenviron'
>>>cenviron.getenv
>>>cenviron
>>>cenviron.getenv('HOME')
'/home/mark'
>>>cenviron.getenv('NONESUCH')
SystemError: Error calling getenv
Here is an example of the problem this
module addresses (but you have to pretend that some of these calls are
made by linked-in C code, not by Python; I changed USER in the shell prior
to this session with anexport
command):
.../PP4E/Integrate/Extend/Cenviron$python
>>>import os
>>>os.environ['USER']
# initialized from the shell
'skipper'
>>>from cenviron import getenv, putenv
# direct C library call access
>>>getenv('USER')
'skipper'
>>>putenv('USER', 'gilligan')
# changes for C but not Python
>>>getenv('USER')
'gilligan'
>>>os.environ['USER']
# oops--does not fetch values again
'skipper'
>>>os.getenv('USER')
# ditto
'skipper'
As is, the C extension module exports a function-based interface,
but it’s easy to wrap its functions in Python code that makes the
interface look any way you like. For instance,
Example 20-10
makes the functions
accessible by dictionary indexing and integrates with theos.environ
object—it guarantees that the
object will stay in sync with fetches and changes made by calling our C
extension functions.
Example 20-10. PP4E\Integrate\Extend\Cenviron\envmap.py
import os
from cenviron import getenv, putenv # get C module's methods
class EnvMapping: # wrap in a Python class
def __setitem__(self, key, value):
os.environ[key] = value # on writes: Env[key]=value
putenv(key, value) # put in os.environ too
def __getitem__(self, key):
value = getenv(key) # on reads: Env[key]
os.environ[key] = value # integrity check
return value
Env = EnvMapping() # make one instance
To use this module, clients may import itsEnv
object usingEnv['var']
dictionary syntax to refer
to environment variables.
Example 20-11
goes a step further and
exports the functions as qualified attribute names rather than as calls
or keys—variables are referenced withEnv.var
attribute syntax.
Example 20-11. PP4E\Integrate\Extend\Cenviron\envattr.py
import os
from cenviron import getenv, putenv # get C module's methods
class EnvWrapper: # wrap in a Python class
def __setattr__(self, name, value):
os.environ[name] = value # on writes: Env.name=value
putenv(name, value) # put in os.environ too
def __getattr__(self, name):
value = getenv(name) # on reads: Env.name
os.environ[name] = value # integrity check
return value
Env = EnvWrapper() # make one instance
The following shows our Python wrappers running atop our C
extension module’s functions to access environment variables. The main
point to notice here is that you can graft many different sorts of
interface models on top of extension functions by providing Python
wrappers in addition to C extensions:
>>>from envmap import Env
>>>Env['USER']
'skipper'
>>>Env['USER'] = 'professor'
>>>Env['USER']
'professor'
>>>
>>>from envattr import Env
>>>Env.USER
'professor'
>>>Env.USER = 'gilligan'
>>>Env.USER
'gilligan'
You can
manually code extension modules like we just did, but you
don’t necessarily have to. Because this example really just wraps
functions that already exist in standard C libraries, the entire
cenviron.c
C code file in
Example 20-8
can be replaced with a
simple SWIG input file that looks like
Example 20-12
.
Example 20-12. PP4E\Integrate\Extend\Swig\Environ\environ.i
/***************************************************************
* Swig module description file, to generate all Python wrapper
* code for C lib getenv/putenv calls: "swig -python environ.i".
***************************************************************/
%module environ
extern char * getenv(const char *varname);
extern int putenv(char *assignment);
And you’re done. Well, almost; you still need to run this file
through SWIG and compile its output. As before, simply add a SWIG step
to your makefile and compile its output file into a shareable object for
dynamic linking, and you’re in business.
Example 20-13
is a Cygwin makefile
that does the job.
Example 20-13. PP4E\Integrate\Extend\Swig\Environ\makefile.environ-swig
# build environ extension from SWIG generated code
PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
SWIG = /cygdrive/c/temp/swigwin-2.0.0/swig
_environ.dll: environ_wrap.c
gcc environ_wrap.c -g -I$(PYINC) -L$(PYLIB) -lpython3.1 -shared -o [email protected]
environ_wrap.c: environ.i
$(SWIG) -python environ.i
clean:
rm -f *.o *.dll *.pyc core environ_wrap.c environ.py
When run on
environ.i
, SWIG generates two
files and two modules—
environ.py
(the Python
interface module we import) and
environ_wrap.c
(the
lower-level glue code module file we compile into
_environ.dll
to be imported by the
.py
). Because the functions being wrapped here live
in standard linked-in C libraries, there is nothing to combine with the
generated code; this makefile simply runs SWIG and compiles the wrapper
file into a C extension module, ready to be imported:
.../PP4E/Integrate/Extend/Swig/Environ$make -f makefile.environ-swig
/cygdrive/c/temp/swigwin-2.0.0/swig -python environ.i
gcc environ_wrap.c -g -I/usr/local/include/python3.1 -L/usr/local/bin -lpython3.1
-shared -o _environ.dll
And now you’re really done. The resulting C extension module is
linked when imported, and it’s used as before (except that SWIG handled
all the gory
bits):
.../PP4E/Integrate/Extend/Swig/Environ$ls
_environ.dll environ.i environ.py environ_wrap.c makefile.environ-swig
.../PP4E/Integrate/Extend/Swig/Environ$python
>>>import environ
>>>environ.getenv('USER')
'gilligan'
>>>environ.__name__, environ.__file__, environ
('environ', 'environ.py',)
>>>dir(environ)
[ ... '_environ', 'getenv', 'putenv' ... ]
If you look closely, you may notice that I didn’t callputenv
this time. It turns out there’s good
cause: the C library’sputenv
wants
a string of the form “USER=Gilligan” to be passed, which becomes part
of the environment. In C code, this means we must create a new piece
of memory to pass in; we usedmalloc
in
Example 20-8
to satisfy this
constraint. However, there’s no simple and direct way to guarantee
this on the Python side of the fence. In a prior Python release, it
was apparently sufficient to hold on to the string passed toputenv
in a temporary Python variable, but
this no longer works with Python 3.X and/or SWIG 2.0. A fix may
require either a custom C function or SWIG’s typemaps which allow its
handling of data translations to be customized. In the interest of
space, we’ll leave addressing this as suggested exercise; see SWIG for
details.