siong1987

siong1987

Teng Siong Ong  //  GraffitiGeo Co-founder, now, Working in Loopt.

Contact me at:
siong1987 at gmail dot com

Sep 11 / 8:46pm

EventMachine(Thin) vs Twisted vs Tornado

Notice: The results here is not accurate. But, you can get a rough idea how everything works. Refer to this: http://www.brianp.net/2008/10/03/changing-the-length-of-the-time_wait-state-on-mac-os-x/

The recent release of the realtime Tornado frameworb from Facebook(friendfeed) is really cool. But, the real question is: how is the performance of Tornado?

After reading the post from apparatus camparing Twisted with Tornado, I decided to do a performance test on Twisted equivalent in Ruby - EventMachine. I used Thin as the server and Rack middleware to test the speed of EventMachine. And, since there was someone in HackerNews curious about how MochiWeb performance on the test. So, I decided to include MochiWeb in my test.

I ran into problems of 'apr_socket_recv: Connection reset by peer (54)' when I tried to run the test on twisted(cocurrency > 75) and mochiweb(cocurrency > 50).

Results:
twisted:
  test1:
    Concurrency Level:      1
    Time taken for tests:   5.815 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1719.73 [#/sec] (mean)
    Time per request:       0.581 [ms] (mean)
    Time per request:       0.581 [ms] (mean, across all concurrent requests)
    Transfer rate:          233.44 [Kbytes/sec] received
  test2:
    Concurrency Level:      1
    Time taken for tests:   5.811 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1720.74 [#/sec] (mean)
    Time per request:       0.581 [ms] (mean)
    Time per request:       0.581 [ms] (mean, across all concurrent requests)
    Transfer rate:          233.58 [Kbytes/sec] received
  test3:
    Concurrency Level:      1
    Time taken for tests:   5.823 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1717.46 [#/sec] (mean)
    Time per request:       0.582 [ms] (mean)
    Time per request:       0.582 [ms] (mean, across all concurrent requests)
    Transfer rate:          233.13 [Kbytes/sec] received
tornado:
  test1:
    Concurrency Level:      1
    Time taken for tests:   4.122 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2425.79 [#/sec] (mean)
    Time per request:       0.412 [ms] (mean)
    Time per request:       0.412 [ms] (mean, across all concurrent requests)
    Transfer rate:          397.98 [Kbytes/sec] received
  test2:
    Concurrency Level:      1
    Time taken for tests:   4.174 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2395.56 [#/sec] (mean)
    Time per request:       0.417 [ms] (mean)
    Time per request:       0.417 [ms] (mean, across all concurrent requests)
    Transfer rate:          393.02 [Kbytes/sec] received
  test3:
    Concurrency Level:      1
    Time taken for tests:   4.163 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2402.26 [#/sec] (mean)
    Time per request:       0.416 [ms] (mean)
    Time per request:       0.416 [ms] (mean, across all concurrent requests)
    Transfer rate:          394.12 [Kbytes/sec] received
thin+rack(eventmachine):
  test1:
    Concurrency Level:      1
    Time taken for tests:   2.954 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    3385.44 [#/sec] (mean)
    Time per request:       0.295 [ms] (mean)
    Time per request:       0.295 [ms] (mean, across all concurrent requests)
    Transfer rate:          466.16 [Kbytes/sec] received
  test2:
    Concurrency Level:      1
    Time taken for tests:   2.915 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    3430.92 [#/sec] (mean)
    Time per request:       0.291 [ms] (mean)
    Time per request:       0.291 [ms] (mean, across all concurrent requests)
    Transfer rate:          472.42 [Kbytes/sec] received
  test3:
    Concurrency Level:      1
    Time taken for tests:   2.929 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    3414.41 [#/sec] (mean)
    Time per request:       0.293 [ms] (mean)
    Time per request:       0.293 [ms] (mean, across all concurrent requests)
    Transfer rate:          470.15 [Kbytes/sec] received
mochiweb:
  test1:
    Concurrency Level:      1
    Time taken for tests:   5.176 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1690000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1932.01 [#/sec] (mean)
    Time per request:       0.518 [ms] (mean)
    Time per request:       0.518 [ms] (mean, across all concurrent requests)
    Transfer rate:          318.86 [Kbytes/sec] received
  test2:  
    Concurrency Level:      1
    Time taken for tests:   14.485 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1690000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    690.36 [#/sec] (mean)
    Time per request:       1.449 [ms] (mean)
    Time per request:       1.449 [ms] (mean, across all concurrent requests)
    Transfer rate:          113.94 [Kbytes/sec] received
  test3:
    Concurrency Level:      1
    Time taken for tests:   6.173 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1690000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1620.07 [#/sec] (mean)
    Time per request:       0.617 [ms] (mean)
    Time per request:       0.617 [ms] (mean, across all concurrent requests)
    Transfer rate:          267.38 [Kbytes/sec] received

twisted:
  test1:
    Concurrency Level:      25
    Time taken for tests:   4.147 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2411.52 [#/sec] (mean)
    Time per request:       10.367 [ms] (mean)
    Time per request:       0.415 [ms] (mean, across all concurrent requests)
    Transfer rate:          327.34 [Kbytes/sec] received
  test2:
    Concurrency Level:      25
    Time taken for tests:   15.073 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    663.42 [#/sec] (mean)
    Time per request:       37.684 [ms] (mean)
    Time per request:       1.507 [ms] (mean, across all concurrent requests)
    Transfer rate:          90.05 [Kbytes/sec] received
  test3:
    Concurrency Level:      25
    Time taken for tests:   4.104 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2436.59 [#/sec] (mean)
    Time per request:       10.260 [ms] (mean)
    Time per request:       0.410 [ms] (mean, across all concurrent requests)
    Transfer rate:          330.75 [Kbytes/sec] received
tornado:
  test1:
    Concurrency Level:      25
    Time taken for tests:   2.362 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680336 bytes
    HTML transferred:       120024 bytes
    Requests per second:    4234.33 [#/sec] (mean)
    Time per request:       5.904 [ms] (mean)
    Time per request:       0.236 [ms] (mean, across all concurrent requests)
    Transfer rate:          694.83 [Kbytes/sec] received
  test2:
    Concurrency Level:      25
    Time taken for tests:   3.312 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680672 bytes
    HTML transferred:       120048 bytes
    Requests per second:    3018.97 [#/sec] (mean)
    Time per request:       8.281 [ms] (mean)
    Time per request:       0.331 [ms] (mean, across all concurrent requests)
    Transfer rate:          495.50 [Kbytes/sec] received
  test3:
    Concurrency Level:      25
    Time taken for tests:   2.364 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680336 bytes
    HTML transferred:       120024 bytes
    Requests per second:    4230.54 [#/sec] (mean)
    Time per request:       5.909 [ms] (mean)
    Time per request:       0.236 [ms] (mean, across all concurrent requests)
    Transfer rate:          694.21 [Kbytes/sec] received
thin+rack(eventmachine):
  test1:
    Concurrency Level:      25
    Time taken for tests:   1.912 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    5229.33 [#/sec] (mean)
    Time per request:       4.781 [ms] (mean)
    Time per request:       0.191 [ms] (mean, across all concurrent requests)
    Transfer rate:          720.05 [Kbytes/sec] received
  test2:
    Concurrency Level:      25
    Time taken for tests:   4.709 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410282 bytes
    HTML transferred:       120024 bytes
    Requests per second:    2123.55 [#/sec] (mean)
    Time per request:       11.773 [ms] (mean)
    Time per request:       0.471 [ms] (mean, across all concurrent requests)
    Transfer rate:          292.46 [Kbytes/sec] received
  test3:
    Concurrency Level:      25
    Time taken for tests:   12.924 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410423 bytes
    HTML transferred:       120036 bytes
    Requests per second:    773.77 [#/sec] (mean)
    Time per request:       32.309 [ms] (mean)
    Time per request:       1.292 [ms] (mean, across all concurrent requests)
    Transfer rate:          106.58 [Kbytes/sec] received
mochiweb:
  test1:
    Concurrency Level:      25
    Time taken for tests:   2.975 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1690000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    3360.99 [#/sec] (mean)
    Time per request:       7.438 [ms] (mean)
    Time per request:       0.298 [ms] (mean, across all concurrent requests)
    Transfer rate:          554.70 [Kbytes/sec] received
  test2:
    Concurrency Level:      25
    Time taken for tests:   21.965 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1690000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    455.28 [#/sec] (mean)
    Time per request:       54.912 [ms] (mean)
    Time per request:       2.196 [ms] (mean, across all concurrent requests)
    Transfer rate:          75.14 [Kbytes/sec] received
  test3:
    Concurrency Level:      25
    Time taken for tests:   7.974 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1690000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1254.08 [#/sec] (mean)
    Time per request:       19.935 [ms] (mean)
    Time per request:       0.797 [ms] (mean, across all concurrent requests)
    Transfer rate:          206.97 [Kbytes/sec] received

twisted:
  test1:
    Concurrency Level:      50
    Time taken for tests:   4.095 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2441.77 [#/sec] (mean)
    Time per request:       20.477 [ms] (mean)
    Time per request:       0.410 [ms] (mean, across all concurrent requests)
    Transfer rate:          331.45 [Kbytes/sec] received
  test2:
    Concurrency Level:      50
    Time taken for tests:   15.061 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    663.97 [#/sec] (mean)
    Time per request:       75.305 [ms] (mean)
    Time per request:       1.506 [ms] (mean, across all concurrent requests)
    Transfer rate:          90.13 [Kbytes/sec] received
  test3:
    Concurrency Level:      50
    Time taken for tests:   5.101 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    1960.27 [#/sec] (mean)
    Time per request:       25.507 [ms] (mean)
    Time per request:       0.510 [ms] (mean, across all concurrent requests)
    Transfer rate:          266.09 [Kbytes/sec] received
tornado:
  test1:
    Concurrency Level:      50
    Time taken for tests:   2.333 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680672 bytes
    HTML transferred:       120048 bytes
    Requests per second:    4286.57 [#/sec] (mean)
    Time per request:       11.664 [ms] (mean)
    Time per request:       0.233 [ms] (mean, across all concurrent requests)
    Transfer rate:          703.55 [Kbytes/sec] received
  test2:
    Concurrency Level:      50
    Time taken for tests:   21.458 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680504 bytes
    HTML transferred:       120036 bytes
    Requests per second:    466.03 [#/sec] (mean)
    Time per request:       107.290 [ms] (mean)
    Time per request:       2.146 [ms] (mean, across all concurrent requests)
    Transfer rate:          76.48 [Kbytes/sec] received
  test3:
    Concurrency Level:      50
    Time taken for tests:   2.338 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1681344 bytes
    HTML transferred:       120096 bytes
    Requests per second:    4277.70 [#/sec] (mean)
    Time per request:       11.689 [ms] (mean)
    Time per request:       0.234 [ms] (mean, across all concurrent requests)
    Transfer rate:          702.37 [Kbytes/sec] received
thin+rack(eventmachine):
  test1:
    Concurrency Level:      50
    Time taken for tests:   1.906 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410705 bytes
    HTML transferred:       120060 bytes
    Requests per second:    5246.96 [#/sec] (mean)
    Time per request:       9.529 [ms] (mean)
    Time per request:       0.191 [ms] (mean, across all concurrent requests)
    Transfer rate:          722.84 [Kbytes/sec] received
  test2:
    Concurrency Level:      50
    Time taken for tests:   12.911 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    774.54 [#/sec] (mean)
    Time per request:       64.554 [ms] (mean)
    Time per request:       1.291 [ms] (mean, across all concurrent requests)
    Transfer rate:          106.65 [Kbytes/sec] received
  test3:
    Concurrency Level:      50
    Time taken for tests:   1.907 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410564 bytes
    HTML transferred:       120048 bytes
    Requests per second:    5243.21 [#/sec] (mean)
    Time per request:       9.536 [ms] (mean)
    Time per request:       0.191 [ms] (mean, across all concurrent requests)
    Transfer rate:          722.25 [Kbytes/sec] received

twisted:
  test1:
    Concurrency Level:      75
    Time taken for tests:   4.106 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2435.40 [#/sec] (mean)
    Time per request:       30.796 [ms] (mean)
    Time per request:       0.411 [ms] (mean, across all concurrent requests)
    Transfer rate:          330.59 [Kbytes/sec] received
  test2:
    Concurrency Level:      75
    Time taken for tests:   6.309 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390278 bytes
    HTML transferred:       120024 bytes
    Requests per second:    1585.12 [#/sec] (mean)
    Time per request:       47.315 [ms] (mean)
    Time per request:       0.631 [ms] (mean, across all concurrent requests)
    Transfer rate:          215.21 [Kbytes/sec] received
  test3:
    Concurrency Level:      75
    Time taken for tests:   23.031 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    434.20 [#/sec] (mean)
    Time per request:       172.730 [ms] (mean)
    Time per request:       2.303 [ms] (mean, across all concurrent requests)
    Transfer rate:          58.94 [Kbytes/sec] received
tornado:
  test1:
    Concurrency Level:      75
    Time taken for tests:   9.284 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680168 bytes
    HTML transferred:       120012 bytes
    Requests per second:    1077.14 [#/sec] (mean)
    Time per request:       69.629 [ms] (mean)
    Time per request:       0.928 [ms] (mean, across all concurrent requests)
    Transfer rate:          176.74 [Kbytes/sec] received
  test2:
    Concurrency Level:      75
    Time taken for tests:   9.286 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680840 bytes
    HTML transferred:       120060 bytes
    Requests per second:    1076.94 [#/sec] (mean)
    Time per request:       69.642 [ms] (mean)
    Time per request:       0.929 [ms] (mean, across all concurrent requests)
    Transfer rate:          176.77 [Kbytes/sec] received
  test3:
    Concurrency Level:      75
    Time taken for tests:   9.328 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1680168 bytes
    HTML transferred:       120012 bytes
    Requests per second:    1072.03 [#/sec] (mean)
    Time per request:       69.961 [ms] (mean)
    Time per request:       0.933 [ms] (mean, across all concurrent requests)
    Transfer rate:          175.90 [Kbytes/sec] received
thin+rack(eventmachine):
  test1:
    Concurrency Level:      75
    Time taken for tests:   1.911 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    5233.07 [#/sec] (mean)
    Time per request:       14.332 [ms] (mean)
    Time per request:       0.191 [ms] (mean, across all concurrent requests)
    Transfer rate:          720.57 [Kbytes/sec] received
  test2:
    Concurrency Level:      75
    Time taken for tests:   20.880 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410423 bytes
    HTML transferred:       120036 bytes
    Requests per second:    478.92 [#/sec] (mean)
    Time per request:       156.601 [ms] (mean)
    Time per request:       2.088 [ms] (mean, across all concurrent requests)
    Transfer rate:          65.97 [Kbytes/sec] received
  test3:
    Concurrency Level:      75
    Time taken for tests:   1.926 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410564 bytes
    HTML transferred:       120048 bytes
    Requests per second:    5191.80 [#/sec] (mean)
    Time per request:       14.446 [ms] (mean)
    Time per request:       0.193 [ms] (mean, across all concurrent requests)
    Transfer rate:          715.17 [Kbytes/sec] received

twisted:
  test1:
    Concurrency Level:      100
    Time taken for tests:   4.151 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2409.15 [#/sec] (mean)
    Time per request:       41.508 [ms] (mean)
    Time per request:       0.415 [ms] (mean, across all concurrent requests)
    Transfer rate:          327.02 [Kbytes/sec] received
  test2:
    Concurrency Level:      100
    Time taken for tests:   4.151 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2409.24 [#/sec] (mean)
    Time per request:       41.507 [ms] (mean)
    Time per request:       0.415 [ms] (mean, across all concurrent requests)
    Transfer rate:          327.03 [Kbytes/sec] received
  test3:
    Concurrency Level:      100
    Time taken for tests:   4.176 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1390000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    2394.58 [#/sec] (mean)
    Time per request:       41.761 [ms] (mean)
    Time per request:       0.418 [ms] (mean, across all concurrent requests)
    Transfer rate:          325.05 [Kbytes/sec] received

tornado:
  test1:
    Concurrency Level:      100
    Time taken for tests:   2.412 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1681344 bytes
    HTML transferred:       120096 bytes
    Requests per second:    4146.64 [#/sec] (mean)
    Time per request:       24.116 [ms] (mean)
    Time per request:       0.241 [ms] (mean, across all concurrent requests)
    Transfer rate:          680.85 [Kbytes/sec] received
  test2:
    Concurrency Level:      100
    Time taken for tests:   9.366 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1682856 bytes
    HTML transferred:       120204 bytes
    Requests per second:    1067.73 [#/sec] (mean)
    Time per request:       93.657 [ms] (mean)
    Time per request:       0.937 [ms] (mean, across all concurrent requests)
    Transfer rate:          175.47 [Kbytes/sec] received
  test3:
    Concurrency Level:      100
    Time taken for tests:   2.414 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1681176 bytes
    HTML transferred:       120084 bytes
    Requests per second:    4142.38 [#/sec] (mean)
    Time per request:       24.141 [ms] (mean)
    Time per request:       0.241 [ms] (mean, across all concurrent requests)
    Transfer rate:          680.08 [Kbytes/sec] received
thin+rack(eventmachine):
  test1:
    Concurrency Level:      100
    Time taken for tests:   1.967 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    5083.27 [#/sec] (mean)
    Time per request:       19.672 [ms] (mean)
    Time per request:       0.197 [ms] (mean, across all concurrent requests)
    Transfer rate:          699.94 [Kbytes/sec] received
  test2:
    Concurrency Level:      100
    Time taken for tests:   20.919 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410987 bytes
    HTML transferred:       120084 bytes
    Requests per second:    478.03 [#/sec] (mean)
    Time per request:       209.191 [ms] (mean)
    Time per request:       2.092 [ms] (mean, across all concurrent requests)
    Transfer rate:          65.87 [Kbytes/sec] received
  test3:
    Concurrency Level:      100
    Time taken for tests:   1.945 seconds
    Complete requests:      10000
    Failed requests:        0
    Write errors:           0
    Total transferred:      1410000 bytes
    HTML transferred:       120000 bytes
    Requests per second:    5142.11 [#/sec] (mean)
    Time per request:       19.447 [ms] (mean)
    Time per request:       0.194 [ms] (mean, across all concurrent requests)
    Transfer rate:          708.04 [Kbytes/sec] received

Code:
Twisted:

Tornado:

Thin+Rack:

MochiWeb:

Feel free to comment on the mistakes I made.

Loading mentions Retweet

3 comments

Sep 19, 2009
Jean-Paul Calderone said...
Hi Teng,

You probably want to look for a better benchmarking tool than ab. The error it spat out at you in a couple of your tests is just one of its shortcomings. I'd also take a careful look at the systems you're using in these benchmarks. You seem to have an *extremely* large variability between your tests - much larger than the difference between any of the servers you tested. This makes the results highly suspicious at best, and useless at worst.

Oct 02, 2009
Nel said...
Given the variation between bench run I guess that you can just trash your benchmarks. You probably run the test on macOsX without monitoring the opened connection with netstat. It happens that you only have 16k port available and 15s timeout before a socket is closed => at more than 1k/s you will hit the limit and just benchmark the TIME_WAIT timeout time.

Just decrease TIME_WAIT timeout

Oct 02, 2009
Teng Siong Ong said...
Thank you for your reminder. In fact, a lot of people already told me the problem. But, I have no time to fix it. But, I will do it soon.

Leave a comment...

 
Got an account with one of these? Login here, or just enter your comment below.
Posterous-login    twitter