For more
industrial-strength state retention, Python scripts can
employ full-blown database solutions in the server. We will study these
options in depth in
Chapter 17
. Python
scripts have access to a variety of server-side data stores, including
flat files, persistent object pickles and shelves, object-oriented
databases such as ZODB, and relational SQL-based databases such as
MySQL, PostgreSQL, Oracle, and SQLite. Besides data storage, such
systems may provide advanced tools such as transaction commits and
rollbacks, concurrent update synchronization, and more.

Full-blown databases are the ultimate storage solution. They can
be used to represent state both between the pages of a single session
(by tagging the data with generated per-session keys) and across
multiple sessions (by storing data under per-user keys).

Given a user’s login name, for example, CGI scripts can fetch all
of the context we have gathered in the past about that user from the
server-side database. Server-side databases are ideal for storing more
complex cross-session information; a shopping cart application, for
instance, can record items added in the past in a server-side
database.

Databases outlive both pages and sessions. Because data is kept
explicitly, there is no need to embed it within the query parameters or
hidden form fields of reply pages. Because the data is kept on the
server, there is no need to store it on the client in cookies. And
because such schemes employ general-purpose databases, they are not
subject to the size constraints or optional nature of cookies.

In exchange for their added utility, full-blown databases require
more in terms of installation, administration, and coding. As we’ll see
in
Chapter 17
, luckily the extra
coding part of that trade-off is remarkably simple in Python. Moreover,
Python’s database interfaces may be used in any application, web-based
or otherwise.

Finally, there are more
advanced protocols and frameworks for retaining state on
the server, which we won’t cover in this book. For instance, the Zope
web application framework, discussed briefly in
Chapter 12
, provides a product interface, which
allows for the construction of web-based objects that are automatically
persistent.

Other schemes, such as FastCGI,
as well as server-specific extensions such as
mod_python
for Apache, may attempt to work
around the autonomous, one-shot nature of CGI scripts, or otherwise
extend the basic CGI model to support long-lived memory stores. For
example:

Such models are not universally supported, though, and may come
with some added cost in complexity—for example, to synchronize access to
persistent data with locks. Moreover, a failure in a FastCGI-style web
application impacts the entire application, not just a single page, and
things like memory leaks become much more costly. For more on persistent
CGI models, and support in Python for things such as FastCGI, search the
Web or consult web-specific resources.

Naturally, these techniques
may be combined to achieve a variety of memory strategies,
both for interaction sessions and for more permanent storage needs. For
example:

The choice of technique is driven by the application’s storage
needs. Although not as straightforward as the in-memory variables and
objects of single process GUI programs running on a client, with
creativity, CGI script state retention is entirely
possible.

Example 15-17. PP4E\Internet\Web\languages.html

Hello World selector
This demo shows how to display a "hello world" message in various
programming languages' syntax.  To keep this simple, only the output command
is shown (it takes more code to make a complete program in some of these
languages), and only text-based solutions are given (no GUI or HTML
construction logic is included). This page is a simple HTML file; the one
you see after pressing the button below is generated by a Python CGI script
which runs on the server. Pointers:

To see this page's HTML, use the 'View Source' command in your browser.
To view the Python CGI script on the server,
click here or
here.
To see an alternative version that generates this page dynamically,
click here.



Select a programming language:

All
Python
Python2
Perl
Tcl
Scheme
SmallTalk
Java
C
C++
Basic
Fortran
Pascal
Other




        	
			  
        	

Figure 15-21. The “Hello World” main page

Example 15-18. PP4E\Internet\Web\cgi-bin\languages.py

#!/usr/bin/python
"""
show hello world syntax for input language name; note that it uses r'...'
raw strings so that '\n' in the table are left intact, and cgi.escape()
on the string so that things like '<<' don't confuse browsers--they are
translated to valid HTML code; any language name can arrive at this script,
since explicit URLs "http://servername/cgi-bin/languages.py?language=Cobol"
can be typed in a web browser or sent by a script (urllib.request.urlopen).
caveats: the languages list appears in both the CGI and HTML files--could
import from single file if selection list generated by a CGI script too;
"""
debugme  = False                                 # True=test from cmd line
inputkey = 'language'                            # input parameter name
hellos = {
'Python':    r" print('Hello World')               ",
'Python2':   r" print 'Hello World'                ",
'Perl':      r' print "Hello World\n";             ',
'Tcl':       r' puts "Hello World"                 ',
'Scheme':    r' (display "Hello World") (newline)  ',
'SmallTalk': r" 'Hello World' print.               ",
'Java':      r' System.out.println("Hello World"); ',
'C':         r' printf("Hello World\n");           ',
'C++':       r' cout << "Hello World" << endl;     ',
'Basic':     r' 10 PRINT "Hello World"             ',
'Fortran':   r" print *, 'Hello World'             ",
'Pascal':    r" WriteLn('Hello World');            "
}
class dummy:                                       # mocked-up input obj
def __init__(self, str): self.value = str
import cgi, sys
if debugme:
form = {inputkey: dummy(sys.argv[1])}          # name on cmd line
else:
form = cgi.FieldStorage()                      # parse real inputs
print('Content-type: text/html\n')                 # adds blank line
print('')
print('
Syntax
')
def showHello(form):                               # HTML for one language
choice = form[inputkey].value
print('
%s
' % choice)
try:
print(cgi.escape(hellos[choice]))
except KeyError:
print("Sorry--I don't know that language")
print('

')
if not inputkey in form or form[inputkey].value == 'All':
for lang in hellos.keys():
mock = {inputkey: dummy(lang)}
showHello(mock)
else:
showHello(form)
print('
')

Raw strings and quotes

Notice the use of
raw strings
(string
constants preceded by an “r” character) in the language syntax
dictionary. Recall that raw strings retain
\
backslash
characters in the string literally, instead of
interpreting them as string escape-code introductions. Without them,
the
\n
newline character
sequences in some of the language’s code snippets would be
interpreted by Python as line feeds, instead of being printed in the
HTML reply as
\n
. The code also
uses double quotes for strings that embed an unescaped single-quote
character, per Python’s normal string rules.

Escaping text embedded in HTML and
URLs

This script takes care to format the text of each language’s
code snippet with
the
cgi.escape
utility function. This standard Python utility automatically
translates characters that are special in HTML into HTML escape code
sequences, so that they are not treated as HTML operators by
browsers. Formally,
cgi.escape
translates characters to escape code sequences, according to the
standard HTML
convention
:
<
,
>
, and
&
become
<
,
>
, and
&
. If you pass a second true
argument, the double-quote character (
"
) is translated to
"
.

For example, the
<<
left-shift operator in the C++ entry is translated to
<<
—
a pair of HTML
escape codes. Because printing each code snippet effectively embeds
it in the HTML response stream, we must escape any special HTML
characters it contains. HTML parsers (including Python’s standard
html.parser
module presented in
Chapter 19
) translate escape codes back to
the original characters when a page is rendered.

More generally, because CGI is based upon the notion of
passing formatted strings across the Net, escaping special
characters is a ubiquitous operation. CGI scripts almost always need
to escape text generated as part of the reply to be safe. For
instance, if we send back arbitrary text input from a user or read
from a data source on the server, we usually can’t be sure whether
it will contain HTML characters, so we must escape it just in
case.

In later examples, we’ll also find that characters inserted
into URL address strings generated by our scripts may need to be
escaped as well. A literal
&
in a URL is special, for example, and must be escaped if it appears
embedded in text we insert into a URL. However, URL syntax reserves
different special characters than HTML code, and so different
escaping conventions and tools must be used. As we’ll see later in
this chapter,
cgi.escape
implements escape translations in HTML code, but
urllib.parse.quote
(and its relatives)
escapes characters in URL strings.

Mocking up form inputs

Here again, form
inputs are “mocked up” (simulated), both for debugging
and for responding to a request for all languages in the table. If
the script’s global
debugme
variable is set to a true value, for instance, the script creates a
dictionary that is plug-and-play compatible with the result of a
cgi.FieldStorage
call—its “languages” key references an instance of the
dummy
mock-up class. This class
in turn creates an object that has the same interface as the
contents of a
cgi.FieldStorage
result—it makes an object with a
value
attribute set to a passed-in
string.

The net effect is that we can test this script by running it
from the system command line: the generated dictionary fools the
script into thinking it was invoked by a browser over the Net.
Similarly, if the requested language name is “All,” the script
iterates over all entries in the languages table, making a mocked-up
form dictionary for each (as though the user had requested each
language in turn).

This lets us reuse the existing
showHello
logic to display each language’s
code in a single page. As always in Python, object interfaces and
protocols are what we
usually
code for, not specific datatypes. The
showHello
function will happily process
any object that responds to the syntax
form['language'].value
.
^[
62
]
Notice that we could achieve similar results with a
default argument in
showHello
,
albeit at the cost of introducing a special case in its code.