在网络通信中socket几乎无处不在,它可以看成是应用层与TCP/IP协议簇通信的中间软件抽象层,是两个应用程序彼此进行通信的接口,并且把复杂的TCP/IP协议细节隐藏在接口之后。Python提供了socket模块,可以非常方便的进行socket编程。
创建一个server socket
使用socket
方法创建一个新的socket,通常提供两个参数,第一个参数是address family, 第二个是socket type。
#create an INET, STREAMing socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
以上创建了一个address family为IP协议,并且传输协议为TCP的socket。
服务器端在创建一个socket之后,需要绑定到一个IP地址与端口,提供服务,这就要用到bind
方法。
# bind the socket to all available interfaces on port 8888
s.bind(('', 8888))
使用listen
方法,将socket设置为监听状态。listen方法后面跟一个参数,表示最多可以在队列里面接收的请求数目。
#become a server socket
serversocket.listen(5)
现在,我们已经创建了一个server socket,然后编写一个无限循环体,接收来自客户端的连接,利用accept
方法,它返回一个新的socket,表示与远端的连接,同时返回一个地址对,表示远端连接的IP地址与端口号。
# enter the main loop of the web server
while 1:
#accept connections from outside, return a pair (conn, addr)
(clientsocket, address) = serversocket.accept()
#now do something with the clientsocket
#in this case, we'll pretend this is a threaded server
ct = client_thread(clientsocket)
ct.run()
通常,循环体中的server socket不会发送和接收任何数据,而仅仅是接收来自远端的连接,将新的连接交给其他的线程去处理,然后就继续侦听更多其他的连接,否则的话在处理一个连接的时候,就无法接受其他新的连接了。
接下来,我们新建一个线程对连接进行处理。首先,使用recv
方法接受来自socket的数据,后面带着一个参数表示一次最多可以接受数据的大小。然后使用sendall
方法回复socket,sendall不断发送数据直到所有数据发送完毕或者发生了异常。最后,需要记得把socket关闭,因为每个socket就代表了系统中的一个文件描述符,如果没有及时关闭,可能会超过系统最大文件描述符的数量,导致无法创建新的socket。
def handle_request(sock, addr):
print "Accept new connection from {addr}".format(addr = addr)
request_data = client_connection.recv(1024)
print request_data.decode()
http_response = "Hello, world!"
# send response data to the socket
sock.sendall(http_response)
sock.close()
总结server socket主要由以下几个步骤:
- 创建一个新的server socket;
- 将socket绑定到一个地址和端口;
- 侦听远程进来的连接;
- 接受连接, 分发给其他线程处理。
以下是完整的server socket示例代码:
import socket
import threading
SERVER_ADDRESS = (HOST, PORT) = '', 8888
REQUEST_QUEUE_SIZE = 1024
def handle_request(sock, addr):
print "Accept new connection from {addr}".format(addr = addr)
request_data = client_connection.recv(1024)
print request_data.decode()
http_response = "Hello, world!"
# send response data to the socket
sock.sendall(http_response)
sock.close()
def serve_forever():
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# reuse socket immediately
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind(SERVER_ADDRESS)
server_socket.listen(REQUEST_QUEUE_SIZE)
print 'Serving HTTP on port {port} ...'.format(port = PORT)
while True:
try:
client_connection, client_address = server_socket.accept()
except IOError as e:
code, msg = e.args
# restart 'accept' if it was interrupted
if code == errno.EINTR:
continue
else:
raise
# create a thread to handle this request
t = threading.Thread(target=handle_request, args=(sock, addr))
t.start()
if __name__ == '__main__':
serve_forever()
创建一个client socket
客户端的socket就比较简单,主要包括以下几个步骤:
- 创建一个socket;
- 连接到服务器;
- 给服务器发送数据;
- 接收服务器返回的数��;
- 关闭socket。
import socket
HOST = 'localhost' # the remote host
PORT = 8888 # port used by server
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# connect to server
client_socket.connect((HOST, PORT))
# send something to the server
client_socket.sendall("Hello, world")
data = client_socket.recv(1024)
client_socket.close()
print 'Received', repr(data)
IO复用
IO多路复用是指,先构造一张有关描述符的列表,然后调用一个函数,直到这些描述符中的一个已经准备好进行I/O时,该函数才返回。在返回时,它告诉进程哪些描述符已经准备好可以进行I/O。
Python的select模块提供了IO复用的功能,如select和poll等都能够执行I/O多路复用。
select
select方法的调用形式如下:
select.select(rlist, wlist, xlist[, timeout]
前面3个参数表示我们正在等待的对象,即我们所关心的文件描述符:
- rlist: 读事件,等待直到可读
- wlist: 写事件,等待直到可写
- xlist: 错误事件,等待直到发生了异常
第4个参数是可选的超时时间的秒数,如果没有选择超时参数,那么一直等待直到至少可以文件描述符准备就绪。如果超时时间设置为0,那么完全不等待,测试所有指定的描述符并立即返回,而不阻塞。
select方法返回的是一个三元组的列表表示准备好的对象。 一个示例程序: server端等待至少一个socket就绪:
import socket
import select
import sys
import Queue
# create a server socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server.bind(('localhost', 8888))
server.listen(5)
# socketss ready for reading
rlist = [server]
# sockets ready for writing
wlist = []
# exception
elist =[]
# each connection needs a queue to act as a buffer for the data to be sent
# through it
message_queues = {}
# main loop
while rlist:
# wait for at least one socket is ready
print "waiting for the next event"
readable, writable, exceptional = select.select(rlist, wlist, elist)
# handle readable lists
for sock in readable:
# server socket: ready to accept another connection
if sock == server:
conn, addr = sock.accept()
print "new connection from {addr}".format(addr = addr)
conn.setblocking(0)
# add new connection to rlist
rlist.append(conn)
# give the connection a queue for data to send
message_queues[conn] = Queue.Queue()
# established connection from client
else:
request_data = sock.recv(1024)
if request_data:
print "received {data} from {addr}".format(
data = request_data, addr = sock.getpeername())
message_queues[sock].put(request_data)
# add to response
if sock not in wlist:
wlist.append(sock)
# without data should be closed
else:
print "closing {addr} after no reading data".format(addr = sock.getpeername())
if sock in wlist:
wlist.remove(sock)
rlist.remove(sock)
sock.close()
del message_queues[sock]
# handle writable lists
for sock in writable:
try:
next_msg = message_queues[sock].get_nowait()
except Queue.Empty:
# no message to send
print "output queue for {addr} is empty".format(addr = sock.getpeername())
wlist.remove(sock)
else:
print "sending {msg} to {addr}".format(msg = next_msg, addr = sock.getpeername())
sock.send(next_msg)
# handle exceptional lists
for sock in exceptional:
print "handling exeptional condition for", sock.getpeername()
rlist.remove(sock)
if sock in wlist:
wlist.remove(sock)
sock.close()
del message_queues[sock]
client端,创建两个socket向server发送消息:
import socket
import sys
# create 2 sockets for clients
clients = [socket.socket(socket.AF_INET, socket.SOCK_STREAM),
socket.socket(socket.AF_INET, socket.SOCK_STREAM)
]
# tow messages to send
messages = ["Hello, server.", "This is my message"]
# connect to the server
server_address = ('localhost', 8888)
print "connecting to {addr}".format(addr = server_address)
for sock in clients:
sock.connect(server_address)
# send one piece of messages once
for msg in messages:
# send message to the server
for sock in clients:
print "{addr} : sending {msg}".format(addr = sock.getpeername(), msg = msg)
sock.send(msg)
# read response from the server
for sock in clients:
response_data = sock.recv(1024)
print "{addr} : received {data}".format(addr = sock.getpeername(), data = response_data)
if not response_data:
print "closing socket {addr}".format(addr = sock.getpeername())
sock.close()
poll
Unix系统的中poll()比select()有着更好的可扩展性,也就是说同时可以支持更多的客户端连接。因为poll()系统调用只需真正感兴趣的那些文件描述符,而select()是通过位图将感兴趣的文件描述符在位图中对应的bit置1,然后需要线性扫描位图中所有的bit。所以select()复杂度是O(N),而poll()是O(M),其中N是最大文件描述符个数,M是目前的文件描述符个数,通常M < N。
- 使用
select.poll()
新建一个polling对象poller,可以在这个对象上面注册(register)或者取消注册(unregister)文件描述符。 - 使用
poll.register
函数将socket注册到polling对象上poll.register(fd[, eventmask])
,这样只要有新的连接或者新的数据过来都会触发一个事件。register函数带有两个参数,第一个参数是文件描述符(整数),第二个参数是一个关于事件比特掩码,可以由POLLIN、POLLOUT和POLLERR等常量表示,用来表示你要检查的感兴趣的事件,如果没有显示指定,那么默认会检查所有3种类型。 poll.poll([timeout])
函数返回由(fd, event)二元组组成的列表,包含了有事件或者错误需要报告的描述符。event是一个bitmask,该描述符需要被报告的事件(events)对应的位被设置为1。POLLIN
:读,等待输入;POLLOUT
:写,描述符可以被写入,依次类推。如果返回的是一个空的列表,那么表示调用超时并且没有文件描述符有任何事件需要报告。如果给出了timeout参数,那么系统会在返回之前等待timeout的时间,如果没有给出timeout参数,那么调用会阻塞直到这个poll对象有新的事件到来。poll.modify(fd, eventmask)
函数可以修改已经注册过的fd,这样我们就能利用这个方法更改某个fd的eventmask,如从读改为写。poll.unregister(fd)
从polling对象中移除fd。
一个示例程序:
import select
import socket
import sys
import Queue
# create a server socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server.bind(('localhost', 8888))
server.listen(5)
# queuses for message to send
message_queues = {}
# eventmask is an optional bitmask describing the type of events you want to
# check for
READ_ONLY = select.POLLIN | select.POLLPRI | select.POLLHUP | select.POLLERR
READ_WRITE = READ_ONLY | select.POLLOUT
# Returns a polling object, which supports registering and unregistering file
# descriptors, and then polling them for I/O events
poller = select.poll()
# Register a file descriptor with the polling object
poller.register(server, READ_ONLY)
# map file desriptor to socket
fd_to_socket = {server.fileno(): server, }
# timeout pass to poll(), ms
TIMEOUT = 1000
# in the loop calls poll() and process returned events
while True:
# wait for at least one socket be ready
print "waiting for the next event"
events = poller.poll(TIMEOUT)
for fd, flag in events:
# retrieve socket from fd
s = fd_to_socket[fd]
# handle read
if flag & (select.POLLIN | select.POLLPRI):
if s is server:
# a readable server is ready to accept a connection
conn, addr = s.accept()
print "new connection from {addr}".format(addr = addr)
conn.setblocking(0)
fd_to_socket[conn.fileno()] = conn
poller.register(conn, READ_ONLY)
# give the connection a queue for sending messages
message_queues[conn] = Queue.Queue()
# otherwise receive data waiting to be read
else:
request_data = s.recv(1024)
if request_data:
print "received {data} from {addr}".format(data = request_data, addr = s.getpeername())
message_queues[s].put(request_data)
# add to writable for response
poller.modify(s, READ_WRITE)
# get empty data means to close the connection
else:
print "closing {addr} after reading no data".format(addr = s.getpeername())
# stop listening for readable
poller.unregister(s)
s.close()
del message_queues[s]
# a client hang up the connection
elif flag & select.POLLHUP:
print "closing {addr} after receiving HUP".format(addr = s.getpeername())
# stop listening for readable poller.unregister(s)
s.close()
del message_queues[s]
# any event with POLLERR cause to close the socket
elif flag & select.POLLERR:
print "handling exception for {addr}".format(addr = s.getpeername())
poller.unregister(s)
s.close()
del message_queues[s]
其他
Python的select模块中提供的select()和poll()函数几乎可以被大多数操作系统支持,此外模块还提供了epoll()和kqueue()函数,但是仅仅限于一部分的操作系统,epoll()仅支持Linux 2.5.44及之后的系统,而kqueue()运行于BSD系统。
本文示例的完整代码,可在GitHub下载查看。
下面关于Python的文章您也可能喜欢,不妨看看:
Python:在指定目录下查找满足条件的文件 http://www.linuxidc.com/Linux/2015-08/121283.htm
Python2.7.7源码分析 http://www.linuxidc.com/Linux/2015-08/121168.htm
无需操作系统直接运行 Python 代码 http://www.linuxidc.com/Linux/2015-05/117357.htm
CentOS上源码安装Python3.4 http://www.linuxidc.com/Linux/2015-01/111870.htm
《Python核心编程 第二版》.(Wesley J. Chun ).[高清PDF中文版] http://www.linuxidc.com/Linux/2013-06/85425.htm
《Python开发技术详解》.( 周伟,宗杰).[高清PDF扫描版+随书视频+代码] http://www.linuxidc.com/Linux/2013-11/92693.htm
Python脚本获取Linux系统信息 http://www.linuxidc.com/Linux/2013-08/88531.htm
在Ubuntu下用Python搭建桌面算法交易研究环境 http://www.linuxidc.com/Linux/2013-11/92534.htm
Python 语言的发展简史 http://www.linuxidc.com/Linux/2014-09/107206.htm
Python 的详细介绍:请点这里
Python 的下载地址:请点这里
本文永久更新链接地址:http://www.linuxidc.com/Linux/2015-08/122481.htm