Le Wiki Koumbit
See also VarnishConfiguration, VarnishMaintenance and CachingService.
Contents
Testing
Once varnish is configured and a site has been added, some tests and benchmarks can be performed. It is preferable to do this after DNS has been configured to see the real situation, but this may not be desirable on a production site. Here's the general testing process:
- test a static page (e.g. an image)
- test the frontpage
- benchmark with 10 users
The following .siegerc will do a 2 minute benchmark with 10 concurrent users on http://example.com/ and hitting only cache0.koumbit.net, using the excellent siege software:
verbose = true fullurl = true concurrent = 10 time = 2M url = http://example.com/ delay = 1 internet = false benchmark = true proxy-host = 209.44.112.104 proxy-port = 80
Notice how the proxy-host directive is required so requests are not directed straight to example.com but to the varnish instance. This is necessary because DNS has not been configured yet.
Those tests should be performed from inside the cabinet, for example from shell.koumbit.net to avoid using unnecessary bandwidth. It will also ensure that there is no bottleneck associated with the upstream connexion.
Specific benchmarks
Popmtl
Benchmark with Drupal 6.10
Transactions: 3986 hits Availability: 100.00 % Elapsed time: 119.88 secs Data transferred: 113.37 MB Response time: 0.30 secs Transaction rate: 33.25 trans/sec Throughput: 0.95 MB/sec Concurrency: 9.98 Successful transactions: 3986 Failed transactions: 0 Longest transaction: 3.95 Shortest transaction: 0.05
Benchmark with Drupal 6.10 + Varnish
Transactions: 6639 hits Availability: 100.00 % Elapsed time: 120.31 secs Data transferred: 188.34 MB Response time: 0.18 secs Transaction rate: 55.18 trans/sec Throughput: 1.57 MB/sec Concurrency: 9.96 Successful transactions: 6616 Failed transactions: 0 Longest transaction: 5.10 Shortest transaction: 0.00
Benchmark with Pressflow 6.14.56
Transactions: 2924 hits Availability: 100.00 % Elapsed time: 119.90 secs Data transferred: 83.17 MB Response time: 0.41 secs Transaction rate: 24.39 trans/sec Throughput: 0.69 MB/sec Concurrency: 9.99 Successful transactions: 2924 Failed transactions: 0 Longest transaction: 3.69 Shortest transaction: 0.09
Benchmark with Pressflow 6.14.56 + Varnish
Transactions: 42360 hits Availability: 100.00 % Elapsed time: 119.71 secs Data transferred: 1251.84 MB Response time: 0.03 secs Transaction rate: 353.86 trans/sec Throughput: 10.46 MB/sec Concurrency: 9.98 Successful transactions: 42360 Failed transactions: 0 Longest transaction: 3.04 Shortest transaction: 0.00
QS
Benchmarks before cookie patch
-n 50 -c 2 -X cache.koumbit.net:80 (13 rps)
anarcat@shell:~$ /usr/sbin/ab -n 50 -c 2 -X cache.koumbit.net:80 http://example.net/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking example.net [through cache.koumbit.net:80] (be patient).....done
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26571 bytes
Concurrency Level: 2
Time taken for tests: 3.629120 seconds
Complete requests: 50
Failed requests: 49
(Connect: 0, Length: 49, Exceptions: 0)
Write errors: 0
Total transferred: 52494 bytes
HTML transferred: 26571 bytes
Requests per second: 13.78 [#/sec] (mean)
Time per request: 145.165 [ms] (mean)
Time per request: 72.582 [ms] (mean, across all concurrent requests)
Transfer rate: 14.05 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.8 0 9
Processing: 73 141 23.5 141 237
Waiting: 73 141 23.5 140 236
Total: 73 142 23.7 141 237
Percentage of the requests served within a certain time (ms)
50% 141
66% 142
75% 142
80% 144
90% 148
95% 170
98% 237
99% 237
100% 237 (longest request)
-n 50 -c 2 -X ques.koumbit.net:80 (10 rps)
anarcat@shell:~$ /usr/sbin/ab -n 50 -c 2 -X ques.koumbit.net:80 http://example.net/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking example.net [through ques.koumbit.net:80] (be patient).....done
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26571 bytes
Concurrency Level: 2
Time taken for tests: 4.946134 seconds
Complete requests: 50
Failed requests: 0
Write errors: 0
Total transferred: 1352126 bytes
HTML transferred: 1328550 bytes
Requests per second: 10.11 [#/sec] (mean)
Time per request: 197.845 [ms] (mean)
Time per request: 98.923 [ms] (mean, across all concurrent requests)
Transfer rate: 266.88 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 9.2 0 39
Processing: 74 192 184.3 142 987
Waiting: 66 171 174.5 130 919
Total: 74 195 190.9 145 1026
Percentage of the requests served within a certain time (ms)
50% 145
66% 160
75% 184
80% 199
90% 418
95% 490
98% 1026
99% 1026
100% 1026 (longest request)
-n 100 -c 10 -X cache.koumbit.net:80 (5rps)
anarcat@shell:~$ /usr/sbin/ab -n 100 -c 10 -X cache.koumbit.net:80 http://example.net/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking example.net [through cache.koumbit.net:80] (be patient).....done
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 0 bytes
Concurrency Level: 10
Time taken for tests: 20.976816 seconds
Complete requests: 100
Failed requests: 1
(Connect: 0, Length: 1, Exceptions: 0)
Write errors: 0
Total transferred: 79990 bytes
HTML transferred: 26571 bytes
Requests per second: 4.77 [#/sec] (mean)
Time per request: 2097.682 [ms] (mean)
Time per request: 209.768 [ms] (mean, across all concurrent requests)
Transfer rate: 3.72 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.9 0 19
Processing: 350 2075 2290.1 794 8712
Waiting: 350 2075 2290.1 793 8711
Total: 350 2075 2290.4 794 8712
Percentage of the requests served within a certain time (ms)
50% 794
66% 1528
75% 3409
80% 4059
90% 5032
95% 8049
98% 8369
99% 8712
100% 8712 (longest request)
-n 100 -c 10 -X ques.koumbit.net:80 (7rps)
anarcat@shell:~$ /usr/sbin/ab -n 100 -c 10 -X ques.koumbit.net:80 http://example.net/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking example.net [through ques.koumbit.net:80] (be patient).....done
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26571 bytes
Concurrency Level: 10
Time taken for tests: 14.552392 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 2704100 bytes
HTML transferred: 2657100 bytes
Requests per second: 6.87 [#/sec] (mean)
Time per request: 1455.239 [ms] (mean)
Time per request: 145.524 [ms] (mean, across all concurrent requests)
Transfer rate: 181.41 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 211 766.6 0 2992
Processing: 145 591 766.8 484 7525
Waiting: 96 528 759.6 442 7518
Total: 145 803 1079.2 626 7565
Percentage of the requests served within a certain time (ms)
50% 626
66% 761
75% 804
80% 822
90% 1318
95% 3476
98% 3773
99% 7565
100% 7565 (longest request)
-n 1000 -c 10 -X cache.koumbit.net:80 (11rps)
anarcat@shell:~$ /usr/sbin/ab -n 1000 -c 10 -X cache.koumbit.net:80 http://example.net/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
[...]
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26567 bytes
Concurrency Level: 10
Time taken for tests: 85.655765 seconds
Complete requests: 1000
Failed requests: 999
(Connect: 0, Length: 999, Exceptions: 0)
Write errors: 0
Non-2xx responses: 1
Total transferred: 544434 bytes
HTML transferred: 26567 bytes
Requests per second: 11.67 [#/sec] (mean)
Time per request: 856.558 [ms] (mean)
Time per request: 85.656 [ms] (mean, across all concurrent requests)
Transfer rate: 6.20 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 2
Processing: 86 852 1401.6 633 22470
Waiting: 83 851 1401.6 632 22470
Total: 86 852 1401.6 633 22470
Percentage of the requests served within a certain time (ms)
50% 633
66% 729
75% 777
80% 800
90% 988
95% 1219
98% 4549
99% 7994
100% 22470 (longest request)
-n 1000 -c 10 -X ques.koumbit.net:80 (11 rps)
anarcat@shell:~$ /usr/sbin/ab -n 1000 -c 10 -X ques.koumbit.net:80 http://example.net/
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26567 bytes
Concurrency Level: 10
Time taken for tests: 83.979262 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 27037304 bytes
HTML transferred: 26567000 bytes
Requests per second: 11.91 [#/sec] (mean)
Time per request: 839.793 [ms] (mean)
Time per request: 83.979 [ms] (mean, across all concurrent requests)
Transfer rate: 314.40 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 14 189.9 0 3015
Processing: 80 822 1669.1 562 23322
Waiting: 64 755 1606.3 506 23146
Total: 80 837 1679.2 565 23322
Percentage of the requests served within a certain time (ms)
50% 565
66% 632
75% 719
80% 777
90% 986
95% 1388
98% 3753
99% 8125
100% 23322 (longest request)
Benchmarks before after cookie patch
-n 1000 -c 10 -X cache.koumbit.net:80 (164 rps)
anarcat@shell:~$ /usr/sbin/ab -n 1000 -c 10 -X cache.koumbit.net:80 http://example.net/
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26567 bytes
Concurrency Level: 10
Time taken for tests: 6.65613 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 27236181 bytes
HTML transferred: 26641646 bytes
Requests per second: 164.86 [#/sec] (mean)
Time per request: 60.656 [ms] (mean)
Time per request: 6.066 [ms] (mean, across all concurrent requests)
Transfer rate: 4384.88 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 25 248.8 5 2988
Processing: 7 34 209.4 18 3698
Waiting: 0 21 208.8 5 3679
Total: 12 60 366.1 23 3703
Percentage of the requests served within a certain time (ms)
50% 23
66% 23
75% 24
80% 25
90% 27
95% 29
98% 43
99% 3697
100% 3703 (longest request)
-n 1000 -c 10 -X cache.koumbit.net:80 (400+rqs!)
anarcat@shell:~$ /usr/sbin/ab -n 1000 -c 10 -X cache.koumbit.net:80 http://example.net/
Server Software: Apache/1.3.34
Server Hostname: example.net
Server Port: 80
Document Path: /
Document Length: 26567 bytes
Concurrency Level: 10
Time taken for tests: 2.335725 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 27172032 bytes
HTML transferred: 26579440 bytes
Requests per second: 428.13 [#/sec] (mean)
Time per request: 23.357 [ms] (mean)
Time per request: 2.336 [ms] (mean, across all concurrent requests)
Transfer rate: 11360.50 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 4 1.6 4 11
Processing: 11 16 1.6 16 24
Waiting: 0 4 1.9 5 10
Total: 12 20 2.6 20 34
Percentage of the requests served within a certain time (ms)
50% 20
66% 21
75% 22
80% 22
90% 24
95% 24
98% 26
99% 26
100% 34 (longest request)