Read Programming Python Online

Authors: Mark Lutz

Tags: #COMPUTERS / Programming Languages / Python

Programming Python (33 page)

BOOK: Programming Python
6.36Mb size Format: txt, pdf, ePub
ads
Sockets: A First Look

Sockets, implemented
by the Python
socket
module, are a more general IPC device than the pipes we’ve seen so far.
Sockets let us transfer data between programs running on the same
computer, as well as programs located on remote networked machines. When
used as an IPC mechanism on the same machine, programs connect to
sockets by a machine-global port number and transfer data. When used as
a networking connection, programs provide both a machine name and port
number to transfer data to a remotely-running program.

Socket basics

Although sockets are
one of the most commonly used IPC tools, it’s impossible
to fully grasp their API without also seeing its role in networking.
Because of that, we’ll defer most of our socket coverage until we can
explore their use in network scripting in
Chapter 12
. This section provides a brief
introduction and preview, so you can compare with the prior section’s
named pipes (a.k.a. fifos). In short:

  • Like fifos, sockets are global across a machine; they do not
    require shared memory among threads or processes, and are thus
    applicable to independent programs.

  • Unlike fifos, sockets are identified by port number, not
    filesystem path name; they employ a very different nonfile API,
    though they can be wrapped in a file-like object; and they are
    more portable: they work on nearly every Python platform,
    including standard Windows Python.

In addition, sockets support networking roles that go beyond
both IPC and this chapter’s scope. To illustrate the basics, though,
Example 5-25
launches a server
and 5 clients in threads running in parallel on the same machine, to
communicate over a socket—because all threads connect to the same
port, the server consumes the data added by each of the
clients.

Example 5-25. PP4E\System\Processes\socket_preview.py

"""
sockets for cross-task communication: start threads to communicate over sockets;
independent programs can too, because sockets are system-wide, much like fifos;
see the GUI and Internet parts of the book for more realistic socket use cases;
some socket servers may also need to talk to clients in threads or processes;
sockets pass byte strings, but can be pickled objects or encoded Unicode text;
caveat: prints in threads may need to be synchronized if their output overlaps;
"""
from socket import socket, AF_INET, SOCK_STREAM # portable socket api
port = 50008 # port number identifies socket on machine
host = 'localhost' # server and client run on same local machine here
def server():
sock = socket(AF_INET, SOCK_STREAM) # ip addresses tcp connection
sock.bind(('', port)) # bind to port on this machine
sock.listen(5) # allow up to 5 pending clients
while True:
conn, addr = sock.accept() # wait for client to connect
data = conn.recv(1024) # read bytes data from this client
reply = 'server got: [%s]' % data # conn is a new connected socket
conn.send(reply.encode()) # send bytes reply back to client
def client(name):
sock = socket(AF_INET, SOCK_STREAM)
sock.connect((host, port)) # connect to a socket port
sock.send(name.encode()) # send bytes data to listener
reply = sock.recv(1024) # receive bytes data from listener
sock.close() # up to 1024 bytes in message
print('client got: [%s]' % reply)
if __name__ == '__main__':
from threading import Thread
sthread = Thread(target=server)
sthread.daemon = True # don't wait for server thread
sthread.start() # do wait for children to exit
for i in range(5):
Thread(target=client, args=('client%s' % i,)).start()

Study this script’s code and comments to see how the socket
objects’ methods are used to transfer data. In a nutshell, with this
type of socket the server accepts a client connection, which by
default blocks until a client requests service, and returns a new
socket connected to the client. Once connected, the client and server
transfer byte strings by using send and receive calls instead of
writes and reads, though as we’ll see later in the book, sockets can
be wrapped in file objects much as we did earlier for pipe
descriptors. Also like pipe descriptors, unwrapped sockets deal in
binary
bytes
strings, not text
str
; that’s why string formatting
results are manually encoded again here.

Here is this script’s output on Windows:

C:\...\PP4E\System\Processes>
socket_preview.py
client got: [b"server got: [b'client1']"]
client got: [b"server got: [b'client3']"]
client got: [b"server got: [b'client4']"]
client got: [b"server got: [b'client2']"]
client got: [b"server got: [b'client0']"]

This output isn’t much to look at, but each line reflects data
sent from client to server, and then back again: the server receives a
bytes string from a connected client and echoes it back in a larger
reply string. Because all threads run in parallel, the order in which
the clients are served is random on this
machine.

Sockets and independent programs

Although sockets
work for threads, the shared memory model of threads
often allows them to employ simpler communication devices such as
shared names and objects and queues. Sockets tend to shine brighter
when used for IPC by separate processes and independently launched
programs.
Example 5-26
, for
instance, reuses the server
and
client
functions of the prior example, but runs them in both
processes and threads of independently launched programs.

Example 5-26. PP4E\System\Processes\socket-preview-progs.py

"""
same socket, but talk between independent programs too, not just threads;
server here runs in a process and serves both process and thread clients;
sockets are machine-global, much like fifos: don't require shared memory
"""
from socket_preview import server, client # both use same port number
import sys, os
from threading import Thread
mode = int(sys.argv[1])
if mode == 1: # run server in this process
server()
elif mode == 2: # run client in this process
client('client:process=%s' % os.getpid())
else: # run 5 client threads in process
for i in range(5):
Thread(target=client, args=('client:thread=%s' % i,)).start()

Let’s run this script on Windows, too (again, this portability
is a major advantage of sockets). First, start the server in a process
as an independently launched program in its own window; this process
runs perpetually waiting for clients to request connections (and as
for our prior pipe example you may need to use Task Manager or a
window close to kill the server process eventually):

C:\...\PP4E\System\Processes>
socket-preview-progs.py 1

Now, in another window, run a few clients in both processes and
thread, by launching them as independent programs—using 2 as the
command-line argument runs a single client process, but 3 spawns five
threads to converse with the server on parallel:

C:\...\PP4E\System\Processes>
socket-preview-progs.py 2
client got: [b"server got: [b'client:process=7384']"]
C:\...\PP4E\System\Processes>
socket-preview-progs.py 2
client got: [b"server got: [b'client:process=7604']"]
C:\...\PP4E\System\Processes>
socket-preview-progs.py 3
client got: [b"server got: [b'client:thread=1']"]
client got: [b"server got: [b'client:thread=2']"]
client got: [b"server got: [b'client:thread=0']"]
client got: [b"server got: [b'client:thread=3']"]
client got: [b"server got: [b'client:thread=4']"]
C:\..\PP4E\System\Processes>
socket-preview-progs.py 3
client got: [b"server got: [b'client:thread=3']"]
client got: [b"server got: [b'client:thread=1']"]
client got: [b"server got: [b'client:thread=2']"]
client got: [b"server got: [b'client:thread=4']"]
client got: [b"server got: [b'client:thread=0']"]
C:\...\PP4E\System\Processes>
socket-preview-progs.py 2
client got: [b"server got: [b'client:process=6428']"]
Socket use cases

This section’s
examples illustrate the basic IPC role of sockets, but
this only hints at their full utility. Despite their seemingly limited
byte string nature, higher-order use cases for sockets are not
difficult to imagine. With a little extra work, for instance:

  • Arbitrary Python
    objects
    like lists and
    dictionaries (or at least copies of them) can be transferred over
    sockets, too, by shipping the serialized byte strings produced by
    Python’s
    pickle
    module
    introduced in
    Chapter 1
    and covered in
    full in
    Chapter 17
    .

  • As we’ll see in
    Chapter 10
    ,
    the printed output of a simple script can be
    redirected
    to a GUI window, by connecting the
    script’s output stream to a socket on which a GUI is listening in
    nonblocking mode.

  • Programs that fetch arbitrary
    text
    off
    the Web might read it as byte strings over sockets, but manually
    decode it using encoding names embedded in content-type headers or
    tags in the data itself.

  • In fact,
    the entire Internet
    can be
    seen as a socket use case—as we’ll see in
    Chapter 12
    , at the bottom, email, FTP, and web
    pages are largely just formatted byte string messages shipped over
    sockets.

Plus any other context in which programs exchange data—sockets
are a general, portable, and flexible tool. For instance, they would
provide the same utility as fifos for the GUI/debugger example used
earlier, but would also work in Python on Windows and would even allow
the GUI to connect to a debugger running on a different computer
altogether. As such, they are seen by many as a more powerful IPC
tool.

Again, you should consider this section just a preview; because
the grander socket story also entails networking concepts, we’ll defer
a more in-depth look at the socket API until
Chapter 12
. We’ll also see sockets again briefly
in
Chapter 10
in the GUI stream
redirection use case listed above, and we’ll explore a variety of
additional socket use cases in the Internet part of this book. In
Part IV
, for instance, we’ll use
sockets to transfer entire files and write more robust socket servers
that spawn threads or processes to converse with clients to avoid
denying connections. For the purposes of this chapter, let’s move on
to one last traditional IPC tool—
the signal.

Signals

For lack of a
better analogy, signals are a way to poke a stick at a
process. Programs generate signals to trigger a handler for that signal
in another process. The operating system pokes, too—some signals are
generated on unusual system events and may kill the program if not
handled. If this sounds a little like raising exceptions in Python, it
should; signals are software-generated events and the cross-process
analog of exceptions. Unlike exceptions, though, signals are identified
by number, are not stacked, and are really an asynchronous event
mechanism outside the scope of the Python interpreter controlled by the
operating system.

In order to make signals available to scripts, Python provides
a
signal
module that
allows Python programs to register Python functions as handlers for
signal events. This module is available on both Unix-like platforms and
Windows (though the Windows version may define fewer kinds of signals to
be caught). To illustrate the basic signal interface, the script in
Example 5-27
installs a Python
handler function for the signal number passed in as a command-line
argument.

Example 5-27. PP4E\System\Processes\signal1.py

"""
catch signals in Python; pass signal number N as a command-line arg,
use a "kill -N pid" shell command to send this process a signal; most
signal handlers restored by Python after caught (see network scripting
chapter for SIGCHLD details); on Windows, signal module is available,
but it defines only a few signal types there, and os.kill is missing;
"""
import sys, signal, time
def now(): return time.ctime(time.time()) # current time string
def onSignal(signum, stackframe): # python signal handler
print('Got signal', signum, 'at', now()) # most handlers stay in effect
signum = int(sys.argv[1])
signal.signal(signum, onSignal) # install signal handler
while True: signal.pause() # wait for signals (or: pass)

There are only two
signal
module calls at work here:

signal.signal

Takes a
signal number and function object and installs that
function to handle that signal number when it is raised. Python
automatically restores most signal handlers when signals occur, so
there is no need to recall this function within the signal handler
itself to reregister the handler. That is, except for
SIGCHLD
, a signal handler remains
installed until explicitly reset (e.g., by setting the handler to
SIG_DFL
to restore default
behavior or to
SIG_IGN
to
ignore the signal).
SIGCHLD
behavior is platform specific.

signal.pause

Makes the
process sleep until the next signal is caught. A
time.sleep
call is similar but
doesn’t work with signals on my Linux box; it generates an
interrupted system call error. A busy
while True: pass
loop here would pause
the script, too, but may squander CPU resources.

Here is what this script looks like running on Cygwin on Windows
(it works the same on other Unix-like platforms like Linux): a signal
number to watch for (12) is passed in on the command line, and the
program is made to run in the background with an
&
shell operator (available in most
Unix-like shells):

[C:\...\PP4E\System\Processes]$
python signal1.py 12 &
[1] 8224
$
ps
PID PPID PGID WINPID TTY UID STIME COMMAND
I 8944 1 8944 8944 con 1004 18:09:54 /usr/bin/bash
8224 7336 8224 10020 con 1004 18:26:47 /usr/local/bin/python
8380 7336 8380 428 con 1004 18:26:50 /usr/bin/ps
$
kill −12 8224
Got signal 12 at Sun Mar 7 18:27:28 2010
$
kill −12 8224
Got signal 12 at Sun Mar 7 18:27:30 2010
$
kill −9 8224
[1]+ Killed python signal1.py 12

Inputs and outputs can be a bit jumbled here because the process
prints to the same screen used to type new shell commands. To send the
program a signal, the
kill
shell
command takes a signal number and a process ID to be signaled (8224);
every time a new
kill
command sends a
signal, the process replies with a message generated by a Python signal
handler function. Signal 9 always kills the process altogether.

The
signal
module also exports
a
signal.alarm
function
for scheduling a
SIGALRM
signal to
occur at some number of seconds in the future. To trigger and catch
timeouts, set the alarm and install a
SIGALRM
handler as shown in
Example 5-28
.

Example 5-28. PP4E\System\Processes\signal2.py

"""
set and catch alarm timeout signals in Python; time.sleep doesn't play
well with alarm (or signal in general in my Linux PC), so we call
signal.pause here to do nothing until a signal is received;
"""
import sys, signal, time
def now(): return time.asctime()
def onSignal(signum, stackframe): # python signal handler
print('Got alarm', signum, 'at', now()) # most handlers stay in effect
while True:
print('Setting at', now())
signal.signal(signal.SIGALRM, onSignal) # install signal handler
signal.alarm(5) # do signal in 5 seconds
signal.pause() # wait for signals

Running this script on Cygwin on Windows causes its
onSignal
handler
function to be invoked every five seconds:

[C:\...\PP4E\System\Processes]$
python signal2.py
Setting at Sun Mar 7 18:37:10 2010
Got alarm 14 at Sun Mar 7 18:37:15 2010
Setting at Sun Mar 7 18:37:15 2010
Got alarm 14 at Sun Mar 7 18:37:20 2010
Setting at Sun Mar 7 18:37:20 2010
Got alarm 14 at Sun Mar 7 18:37:25 2010
Setting at Sun Mar 7 18:37:25 2010
Got alarm 14 at Sun Mar 7 18:37:30 2010
Setting at Sun Mar 7 18:37:30 2010
...Ctrl-C to exit...

Generally speaking, signals must be used with cautions not made
obvious by the examples we’ve just seen. For instance, some system calls
don’t react well to being interrupted by signals, and only the main
thread can install signal handlers and respond to signals in a
multithreaded program.

When used well, though, signals provide an event-based
communication mechanism. They are less powerful than data streams such
as pipes, but are sufficient in situations in which you just need to
tell a program that something important has occurred and don’t need to
pass along any details about the event itself. Signals are sometimes
also combined with other IPC tools. For example, an initial signal may
inform a program that a client wishes to communicate over a named
pipe—the equivalent of tapping someone’s shoulder to get their attention
before speaking. Most platforms reserve one or more
SIGUSR
signal numbers for user-defined events
of this sort. Such an integration structure is sometimes an alternative
to running a blocking input call in a spawned thread.

See also the
os.kill(
pid
,
sig
)
call for sending signals to known processes
from within a Python script on Unix-like platforms, much like
the
kill
shell command
used earlier; the required process ID can be obtained from the
os.fork
call’s child process ID return value
or from other interfaces. Like
os.fork
, this call is also available in Cygwin
Python, but not in standard Windows Python. Also watch for the
discussion about using signal handlers to clean up “zombie” processes in
Chapter 12
.

BOOK: Programming Python
6.36Mb size Format: txt, pdf, ePub
ads

Other books

The Beauty of the Mist by May McGoldrick
Books of the Dead by Morris Fenris
My Favorite Senior Moments by Karen O'Connor
The River of Night's Dreaming by Karl Edward Wagner
Bride in Flight by Essie Summers
A Spring Affair by Johnson, Milly
The Game You Played by Anni Taylor
Rules of Surrender by Christina Dodd


readsbookonline.com Copyright 2016 - 2024