请多关注下我的个人博客，
测试下多线程和gevent在socket服务端的小包表现能力，测试的方法不太严谨，
有点属于自娱自乐，要是有问题之处，请大家喷之 ！
每个连接都特意堵塞了0.5秒钟 ！
在大批量tcp测试下，threading的开销越来越大，所以造成了在并发数加大的情况下，出现threading崩溃的情况 ！ gevent是 libevent和协程的融合，一个线程里面都可以跑超多的协程！ 利用libevent做io堵塞的调度 ，gevent体系下，同一时间只有一个任务在运行 ！
先来测试下多线程： 我们就不加线程池了
#!/usr/bin/env python# -*- coding: utf-8 -*-#xiaorui.ccimport sysimport socketimport timeimport threading#xiaorui.ccdef threads(port): s = socket.socket() s.bind(('0.0.0.0', port)) s.listen(500) while True: cli, addr = s.accept() t = threading.Thread(target=handle_request, args=(cli, time.sleep)) t.daemon = True t.start()def handle_request(s, sleep): try: s.recv(1024) sleep(0.5) s.send('''http/1.0 200 OK Hello World! ''') s.shutdown(socket.SHUT_WR) print '.', except Exception, ex: print ex finally: sys.stdout.flush() s.close()if __name__ == '__main__': threads(4444)
用threading跑socket，每个连接堵塞的时间是0.5
time ab -n 10000 -c 500 http://127.0.0.1:4444/This is ApacheBench, Version 2.3 <$Revision: 655654 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 127.0.0.1 (be patient)Completed 1000 requestsCompleted 2000 requestsCompleted 3000 requestsCompleted 4000 requestsCompleted 5000 requestsCompleted 6000 requestsCompleted 7000 requestsCompleted 8000 requestsCompleted 9000 requestsCompleted 10000 requestsFinished 10000 requestsServer Software:Server Hostname: 127.0.0.1Server Port: 4444Document Path: /Document Length: 0 bytesConcurrency Level: 500Time taken for tests: 11.123 secondsComplete requests: 10000Failed requests: 0Write errors: 0Total transferred: 470000 bytesHTML transferred: 0 bytesRequests per second: 899.01 [#/sec] (mean)Time per request: 556.166 [ms] (mean)Time per request: 1.112 [ms] (mean, across all concurrent requests)Transfer rate: 41.26 [Kbytes/sec] receivedConnection Times (ms) min mean[+/-sd] median maxConnect: 0 33 177.0 0 1000Processing: 500 508 33.9 501 1132Waiting: 500 508 33.9 501 1132Total: 500 541 201.8 501 2132Percentage of the requests served within a certain time (ms) 50% 501 66% 501 75% 502 80% 505 90% 522 95% 532 98% 1534 99% 1722 100% 2132 (longest request)real 0m11.145suser 0m0.210ssys 0m0.961s
加到800的时候~
gevent：
#xiaorui.ccimport sysimport socketimport timeimport geventfrom gevent import socketdef server(port): s = socket.socket() s.bind(('0.0.0.0', port)) s.listen(500) while True: cli, addr = s.accept() gevent.spawn(handle_request, cli, gevent.sleep)def handle_request(s, sleep): try: data=s.recv(1024) sleep(0.5) s.send('''http/1.0 200 OK Hello World! this is xiaorui.cc !!!''') print data request_string = "GET %s HTTP/1.1\r\nHost: %s\r\n\r\nServer: xiaorui.cc\n" %('index.html', '127.0.0.1') s.send(request_string) s.shutdown(socket.SHUT_WR) print '.',‘be killed’ except Exception, ex: print ex finally: s.close()if __name__ == '__main__': server(7777)
gevent跑socket服务：
并发数值是500的时候！
time ab -n 10000 -c 500 http://127.0.0.1:7777/This is ApacheBench, Version 2.3 <$Revision: 655654 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 127.0.0.1 (be patient)Completed 1000 requestsCompleted 2000 requestsCompleted 3000 requestsCompleted 4000 requestsCompleted 5000 requestsCompleted 6000 requestsCompleted 7000 requestsCompleted 8000 requestsCompleted 9000 requestsCompleted 10000 requestsFinished 10000 requestsServer Software:Server Hostname: 127.0.0.1Server Port: 7777Document Path: /Document Length: 0 bytesConcurrency Level: 500Time taken for tests: 11.312 secondsComplete requests: 10000Failed requests: 0Write errors: 0Total transferred: 20000 bytesHTML transferred: 0 bytesRequests per second: 884.04 [#/sec] (mean)Time per request: 565.584 [ms] (mean)Time per request: 1.131 [ms] (mean, across all concurrent requests)Transfer rate: 1.73 [Kbytes/sec] receivedConnection Times (ms) min mean[+/-sd] median maxConnect: 0 44 202.7 0 1001Processing: 500 513 10.1 511 707Waiting: 500 513 10.1 511 707Total: 500 557 204.1 512 1525Percentage of the requests served within a certain time (ms) 50% 512 66% 515 75% 517 80% 519 90% 531 95% 552 98% 1521 99% 1523 100% 1525 (longest request)real 0m11.334suser 0m0.159ssys 0m0.730s
服务端看到的信息都是正常的！
并发是1000的时候：
time ab -n 10000 -c 1000 http://127.0.0.1:7777/This is ApacheBench, Version 2.3 <$Revision: 655654 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 127.0.0.1 (be patient)Completed 1000 requestsCompleted 2000 requestsCompleted 3000 requestsCompleted 4000 requestsCompleted 5000 requestsCompleted 6000 requestsCompleted 7000 requestsCompleted 8000 requestsCompleted 9000 requestsCompleted 10000 requestsFinished 10000 requestsServer Software:Server Hostname: 127.0.0.1Server Port: 7777Document Path: /Document Length: 0 bytesConcurrency Level: 1000Time taken for tests: 7.406 secondsComplete requests: 10000Failed requests: 0Write errors: 0Total transferred: 20000 bytesHTML transferred: 0 bytesRequests per second: 1350.22 [#/sec] (mean)Time per request: 740.623 [ms] (mean)Time per request: 0.741 [ms] (mean, across all concurrent requests)Transfer rate: 2.64 [Kbytes/sec] receivedConnection Times (ms) min mean[+/-sd] median maxConnect: 0 175 491.7 0 3000Processing: 500 520 17.7 515 707Waiting: 500 520 17.7 515 707Total: 500 695 492.5 517 3521Percentage of the requests served within a certain time (ms) 50% 517 66% 523 75% 538 80% 569 90% 1515 95% 1530 98% 1539 99% 3514 100% 3521 (longest request)real 0m7.428suser 0m0.208ssys 0m0.741s
当并发到1500的时候：
time ab -n 10000 -c 1500 http://127.0.0.1:7777/This is ApacheBench, Version 2.3 <$Revision: 655654 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 127.0.0.1 (be patient)Completed 1000 requestsCompleted 2000 requestsCompleted 3000 requestsCompleted 4000 requestsCompleted 5000 requestsCompleted 6000 requestsCompleted 7000 requestsCompleted 8000 requestsCompleted 9000 requestsCompleted 10000 requestsFinished 10000 requestsServer Software:Server Hostname: 127.0.0.1Server Port: 7777Document Path: /Document Length: 0 bytesConcurrency Level: 1500Time taken for tests: 5.290 secondsComplete requests: 10000Failed requests: 0Write errors: 0Total transferred: 20000 bytesHTML transferred: 0 bytesRequests per second: 1890.27 [#/sec] (mean)Time per request: 793.536 [ms] (mean)Time per request: 0.529 [ms] (mean, across all concurrent requests)Transfer rate: 3.69 [Kbytes/sec] receivedConnection Times (ms) min mean[+/-sd] median maxConnect: 0 214 404.9 1 1003Processing: 500 522 23.0 514 716Waiting: 500 522 23.0 514 716Total: 500 736 406.7 520 1712Percentage of the requests served within a certain time (ms) 50% 520 66% 558 75% 602 80% 1506 90% 1526 95% 1531 98% 1535 99% 1548 100% 1712 (longest request)real 0m5.313suser 0m0.275ssys 0m0.763s
出现了少量的报错：
gevent 可以加个队列，来限制协程的数目，但是数目限制了，虽然稳定了，但是并发数上不去。
from gevent.pool import Poolpool = Pool(N)
这里测试有点简单，虽然已经安排了连接的堵塞，但是毕竟不符合业务。 有时间把后端的任务改成才mongodb取数据 ！