Read Programming Python Online

Authors: Mark Lutz

Tags: #COMPUTERS / Programming Languages / Python

Programming Python (199 page)

Basic Embedding Techniques

As you can
probably tell from the preceding overview, there is much
flexibility in the embedding domain. To illustrate common embedding
techniques in action, this section presents a handful of short C programs
that run Python code in one form or another. Most of these examples will
make use of the simple Python module file shown in
Example 20-22
.

Example 20-22. PP4E\Integrate\Embed\Basics\usermod.py

"""
#############################################################
C code runs Python code in this module in embedded mode.
Such a file can be changed without changing the C layer.
This is just standard Python code (C handles conversions).
Must be on the Python module search path if imported by C.
C can also run code in standard library modules like string.
#############################################################
"""
message = 'The meaning of life...'
def transform(input):
input = input.replace('life', 'Python')
return input.upper()

If you know any Python at all, you probably know that this file
defines a string and a function; the function returns whatever it is
passed with string substitution and uppercase conversions applied. It’s
easy to use from Python:

.../PP4E/Integrate/Embed/Basics$
python
>>>
import usermod
# import a module
>>>
usermod.message
# fetch a string
'The meaning of life...'
>>>
usermod.transform(usermod.message)
# call a function
'THE MEANING OF PYTHON...'

With a little Python API wizardry, it’s not much more difficult to
use this module the same way in C.

Running Simple Code Strings

Perhaps the simplest
way to run Python code from C is by calling the
PyRun_SimpleString
API function. With it, C
programs can execute Python programs represented as C character string
arrays. This call is also very limited: all code runs in the same
namespace (the module
__main__
), the
code strings must be Python statements (not expressions), and there is
no direct way to communicate inputs or outputs with the Python code
run.

Still, it’s a simple place to start. Moreover, when augmented with
an imported C extension module that the embedded Python code can use to
communicate with the enclosing C layer, this technique can satisfy many
embedding goals. To demonstrate the basics, the C program in
Example 20-23
runs Python code to
accomplish the same results as the Python interactive session listed in
the prior section.

Example 20-23. PP4E\Integrate\Embed\Basics\embed-simple.c

/*******************************************************
* simple code strings: C acts like the interactive
* prompt, code runs in __main__, no output sent to C;
*******************************************************/
#include /* standard API def */
main() {
printf("embed-simple\n");
Py_Initialize();
PyRun_SimpleString("import usermod"); /* load .py file */
PyRun_SimpleString("print(usermod.message)"); /* on Python path */
PyRun_SimpleString("x = usermod.message"); /* compile and run */
PyRun_SimpleString("print(usermod.transform(x))");
Py_Finalize();
}

The first thing you should notice here is that when Python is
embedded, C programs always call
Py_Initialize
to initialize linked-in Python
libraries before using any other API functions and normally call
Py_Finalize
to shut the interpreter
down.

The rest of this code is straightforward—C submits hardcoded
strings to Python that are roughly what we typed interactively. In fact,
we could concatenate all the Python code strings here with
\n
characters between, and submit it once as a
single string. Internally,
PyRun_SimpleString
invokes
the Python compiler and interpreter to run the strings
sent from C; as usual, the Python compiler is always available in
systems that contain Python.

Compiling and running

To build a standalone executable from this C source file, you
need to link its compiled form with the Python library file. In this
chapter, “library” usually means the binary library file that is
generated when Python is compiled, not the Python source code standard
library.

Today, everything in Python that you need in C is compiled into
a single Python library file when the interpreter is built (e.g.,
libpython3.1.dll
on Cygwin). The program’s
main
function comes from your C
code, and depending on your platform and the extensions installed in
your Python, you may also need to link any external libraries
referenced by the Python library.

Assuming no extra extension libraries are needed,
Example 20-24
is a minimal makefile
for building the C program in
Example 20-23
under
Cygwin on Windows. Again, makefile details vary per
platform, but see Python manuals for hints. This makefile uses the
Python include-files path to find
Python.h
in the
compile step and adds the Python library file to the final link step
to make API calls available to the C program.

Example 20-24. PP4E\Integrate\Embed\Basics\makefile.1

# a Cygwin makefile that builds a C executable that embeds
# Python, assuming no external module libs must be linked in;
# uses Python header files, links in the Python lib file;
# both may be in other dirs (e.g., /usr) in your install;
PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
embed-simple: embed-simple.o
gcc embed-simple.o -L$(PYLIB) -lpython3.1 -g -o embed-simple
embed-simple.o: embed-simple.c
gcc embed-simple.c -c -g -I$(PYINC)

To build a program with this file, launch
make
on it as usual (as before, make sure
indentation in rules is tabs in your copy of this makefile):

.../PP4E/Integrate/Embed/Basics$
make -f makefile.1
gcc embed-simple.c -c -g -I/usr/local/include/python3.1
gcc embed-simple.o -L/usr/local/bin -lpython3.1 -g -o embed-simple

Things may not be quite this simple in practice, though, at
least not without some coaxing. The makefile in
Example 20-25
is the one I actually
used to build all of this section’s C programs on Cygwin.

Example 20-25. PP4E\Integrate\Embed\Basics\makefile.basics

# cygwin makefile to build all 5 basic embedding examples at once
PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
BASICS = embed-simple.exe \
embed-string.exe \
embed-object.exe \
embed-dict.exe \
embed-bytecode.exe
all: $(BASICS)
embed%.exe: embed%.o
gcc embed$*.o -L$(PYLIB) -lpython3.1 -g -o [email protected]
embed%.o: embed%.c
gcc embed$*.c -c -g -I$(PYINC)
clean:
rm -f *.o *.pyc $(BASICS) core

On some platforms, you may need to also link in other libraries
because the Python library file used may have been built with external
dependencies enabled and required. In fact, you may have to link in
arbitrarily many more externals for your Python library, and frankly,
chasing down all the linker dependencies can be tedious. Required
libraries may vary per platform and Python install, so there isn’t a
lot of advice I can offer to make this process simple (this is C,
after all). The standard C development techniques will apply.

One hint here: if you’re going to do much embedding work and you
run into external dependency issues, on some platforms you might want
to build Python on your machine from its source with all unnecessary
extensions
disabled
in its build configuration
files (see the Python source package for details). This produces a
Python library with minimal external requirements, which may links
more easily.

Once you’ve gotten the makefile to work, run it to build all of
this section’s C programs at once with Python libraries linked
in:

.../PP4E/Integrate/Embed/Basics$
make -f makefile.basics clean
rm -f *.o *.pyc embed-simple.exe embed-string.exe embed-object.exe
embed-dict.exe embed-bytecode.exe core
.../PP4E/Integrate/Embed/Basics$
make -f makefile.basics
gcc embed-simple.c -c -g -I/usr/local/include/python3.1
gcc embed-simple.o -L/usr/local/bin -lpython3.1 -g -o embed-simple.exe
gcc embed-string.c -c -g -I/usr/local/include/python3.1
gcc embed-string.o -L/usr/local/bin -lpython3.1 -g -o embed-string.exe
gcc embed-object.c -c -g -I/usr/local/include/python3.1
gcc embed-object.o -L/usr/local/bin -lpython3.1 -g -o embed-object.exe
gcc embed-dict.c -c -g -I/usr/local/include/python3.1
gcc embed-dict.o -L/usr/local/bin -lpython3.1 -g -o embed-dict.exe
gcc embed-bytecode.c -c -g -I/usr/local/include/python3.1
gcc embed-bytecode.o -L/usr/local/bin -lpython3.1 -g -o embed-bytecode.exe
rm embed-dict.o embed-object.o embed-simple.o embed-bytecode.o embed-string.o

After building with either makefile, you can run the resulting C
program as usual:

.../PP4E/Integrate/Embed/Basics$
./embed-simple
embed-simple
The meaning of life...
THE MEANING OF PYTHON...

Most of this output is produced by Python
print
statements sent from C to the
linked-in Python library. It’s as if C has become an interactive
Python programmer.

Naturally, strings of Python code run by C probably would not be
hardcoded in a C program file like this. They might instead be loaded
from a text file or GUI, extracted from HTML or XML files, fetched
from a persistent database or socket, and so on. With such external
sources, the Python code strings that are run from C could be changed
arbitrarily without having to recompile the C program that runs them.
They may even be changed on site, and by end users of a system. To
make the most of code strings, though, we need to move on to more
flexible API
tools.

Note

Pragmatic details
: Under Python 3.1 and
Cygwin on Windows, I had to first set my
PYTHONPATH
to include the current
directory in order to run the embedding examples, with the shell
command
export PYTHONPATH=.
. I
also had to use the shell command
./embed-simple
to execute the program
because
.
was also not on my
system
path
setting and isn’t
initially when you install Cygwin.

Your mileage may vary; but if you have trouble, try running
the embedded Python commands
import
sys
and
print sys.path
from C to see what Python’s path looks like, and take a look at the
Python/C API
manual for more on path
configuration for embedded applications.

Running Code Strings with Results and Namespaces

Example 20-26
uses the
following API calls to run code strings that return
expression results back to C:

Py_Initialize

Initializes
linked-in Python libraries as before

PyImport_ImportModule

Imports a Python
module and returns a pointer to it

PyModule_GetDict

Fetches a
module’s attribute dictionary object

PyRun_String

Runs a
string of code in explicit namespaces

PyObject_SetAttrString

Assigns
an object attribute by
namestring

PyArg_Parse

Converts a
Python return value object to C form

The import calls are used to fetch the namespace of the
usermod
module listed in
Example 20-22
so that code strings
can be run there directly and will have access to names defined in that
module without qualifications.
Py_Import_ImportModule
is like a Python
import
statement, but the imported
module object is returned to C; it is not assigned to a Python variable
name. As a result, it’s probably more similar to the Python
__import__
built-in function.

The
PyRun_String
call is the
one that actually runs code here, though. It takes a code string, a
parser mode flag, and dictionary object pointers to serve as the global
and local namespaces for running the code string. The mode flag can be
Py_eval_input
to run an expression or
Py_file_input
to run a statement;
when running an expression, the result of evaluating the expression is
returned from this call (it comes back as a
PyObject*
object pointer). The two namespace
dictionary pointer arguments allow you to distinguish global and local
scopes, but they are typically passed the same dictionary such that code
runs in a single namespace.

Example 20-26. PP4E\Integrate\Embed\Basics\embed-string.c

/* code-strings with results and namespaces */
#include
main() {
char *cstr;
PyObject *pstr, *pmod, *pdict;
printf("embed-string\n");
Py_Initialize();
/* get usermod.message */
pmod = PyImport_ImportModule("usermod");
pdict = PyModule_GetDict(pmod);
pstr = PyRun_String("message", Py_eval_input, pdict, pdict);
/* convert to C */
PyArg_Parse(pstr, "s", &cstr);
printf("%s\n", cstr);
/* assign usermod.X */
PyObject_SetAttrString(pmod, "X", pstr);
/* print usermod.transform(X) */
(void) PyRun_String("print(transform(X))", Py_file_input, pdict, pdict);
Py_DECREF(pmod);
Py_DECREF(pstr);
Py_Finalize();
}

When compiled and run, this file produces the same result as its
predecessor:

.../PP4E/Integrate/Embed/Basics$
./embed-string
embed-string
The meaning of life...
THE MEANING OF PYTHON...

However, very different work goes into producing this output. This
time, C fetches, converts, and prints the value of the Python module’s
message
attribute directly by running
a string expression and assigning a global variable (
X
) within the module’s namespace to serve as
input for a Python
print
statement
string.

Because the string execution call in this version lets you specify
namespaces, you can better partition the embedded code your system
runs—each grouping can have a distinct namespace to avoid overwriting
other groups’ variables. And because this call returns a result, you can
better communicate with the embedded code; expression results are
outputs, and assignments to globals in the namespace in which code runs
can serve as inputs.

Before we move on, I need to explain three coding issues here.
First, this program also decrements the reference count on objects
passed to it from Python, using the
Py_DECREF
call described in Python’s C API
manuals. These calls are not strictly needed here (the objects’ space is
reclaimed when the programs exits anyhow), but they demonstrate how
embedding interfaces must manage reference counts when Python passes
object ownership to C. If this was a function called from a larger
system, for instance, you would generally want to decrement the count to
allow Python to reclaim the
objects
.

Second, in a realistic program, you should generally test the
return values of
all
the API calls in this program
immediately to detect errors (e.g., import failure). Error tests are
omitted in this section’s example to keep the code simple, but they
should be included in your programs to make them more robust.

And third, there is a related function that lets you run entire
files
of code, but it is not demonstrated in this
chapter:
PyRun_File
. Because you can
always load a file’s text and run it as a single code string with
PyRun_String
, the
PyRun_File
call’s main advantage is to avoid
allocating memory for file content. In such multiline code strings, the
\n
character terminates lines and
indentation group blocks as
usual.

Other books

The Immaculate by Mark Morris
Starting Fires by Makenzie Smith
Best Friend Next Door by Carolyn Mackler
Another Me by Cathy MacPhail
Unbreakable by Nancy Mehl
Codebreakers Victory by Hervie Haufler
The Burning Horizon by Erin Hunter


readsbookonline.com Copyright 2016 - 2024