Tyblog | 35 Million Hot Dogs: Benchmarking Caddy vs. Nginx

1

This blog post is about benchmarking Caddy against Nginx and their respective performance metrics as reverse proxies. Be forewarned: I was very thorough and there are tons of graphs and tables in here. I didn’t want to make any mistakes! Nobody is allowed to make mistakes on the Internet.

This is a companion discussion topic for the original entry at https://blog.tjll.net/reverse-proxy-hot-dog-eating-contest-caddy-vs-nginx/
1 Like
2

Really enjoyed reading it, thank for taking your time for the write up!

One question I have is how did you install Caddy? Did you build from source? As you correctly saw the spikes when Go’s garbage collector was running, when building with Go > 1.19 you have the option to specify GOMEMLIMIT, which is a soft limit of RAM for when the garbage collector should run. As your target server has 4GB of RAM, my assumption would be, this would improve the performance quite a bit in the more CPU intensive tests.
On the other hand, I am very impressed by nginx almost never overstepping 32MB of memory used.

1 Like
4

Thank you for taking the time to read it! :smile:

My system under test was a NixOS 22.05 machine, which builds Caddy using this package. Nix is sort of funky to read but it’s built with go 1.17. Maybe in the future I can tweak the build environment to see what that kind of change might cause, but I’d certainly be curious as well.

5

Thanks for sharing your benchmarks. In the past I’ve done some Nginx vs Caddy benchmarks and noticed as you add more response headers to requests, Caddy’s performance drops more than Nginx does (see links below). As as I’ve been always mindful to test with real world used response headers/equivalent for both web servers.

So I’m curious if in your proxy testing for latest Caddy versions, does the same performance overhead happen with Caddy still exist as you add more response headers?

Looking forward to more of your tests in future :slight_smile:

6

Performance measurements that include response headers is something I haven’t tested yet - since I published this research I’ve received lots of helpful input about some additional variables to test, and headers are one of those variables that I think would be helpful to include. I’ve recorded your suggestion here if and when I do another round of tests :slightly_smiling_face:

1 Like
7

Cheers much appreciated :slight_smile: