How to correctly use Siege to benchmark GET requests?

I use siege to benchmark a local HTTP service:

siege -q -b -t 30S -c 64 "http://localhost:8888/endpoint?params=abc"

Benchmark result gives ~500 transactions per second throughput.

However, when I benchmark the service manually by launching 8 simultaneous loops of curl, the same endpoint gives many thousands per second throughput.

So what am I doing wrong with siege?

Answer

There a few things that may be impacting your test. Check to see what you get if you set concurrent -c to 8 to match the number of concurrents you are testing with curl. Your site may not be able to handle a high number of concurrent users.

Also Siege by default does not do keep alive (I don’t think at least). By not using this option the server and client have to set up and tear down sockets which can be costly. You should be able to set the header and have it honored (siege -H "Connection: Keep-Alive").

As a side note, I find it useful to use several tools when benchmarking, once you get siege working, i’d play with AB (apache bench), httperf and keep your manual curl testing around as well.