This is an awesome article but if your egress costs are so high that you're deciding which HTTP headers to exclude, you should probably be moving to an unmetered bandwidth provider, or at least one that charges a reasonable amount for egress.
Is there any such thing? I don't know of any cloud service provider that offers unlimited bandwidth. There are very few providers who could handle five billion connections per day in the first place, regardless of bandwidth.
5B requests/day is ~60k/second, that's big but nothing insane. There are numerous frameworks/setups that can do far more than that on a single machine [1]
popular unmetered options: he.net, ovh, hetzner - You generally lose a lot of the "cloud" capability with these options however.
cloud options: digital ocean egress is $0.01/GB ($0.005/GB if you buy it via droplets), linode is $0.02/GB, vultr is $0.01/GB, etc.
I'm talking about actual unmetered where you pay for a dedicated amount of bandwidth, e.g. 1 Gbps / 10 Gbps / 20 Gbps. 10 Gbps usually goes for about $1k-$2k/mo in the US. This is how colo facilities have operated for decades.
10 Gbps fully saturated delivers about 3300TB for that $1-2k/mo, versus the $22k/mo you'd pay AWS for the same.
I'm absolutely not talking about the "unlimited bandwdith" bullshit that discount hosts offer.
If your project gets featured on CNN and your bandwidth goes up 20x can these colo arrangements automatically scale up your dedicated bandwidth? I ask because having an outage when you get your first big break can cost you WAY more than your bandwidth bill ever would...
DI.FM uses cloudflare + bandwidth alliance [1] for their streaming audio network, so I'd model after that. Cloudflare isn't exactly transparent about their egress pricing, but most discussion seems to indicate once you start hitting about 50TB/mo, they'll strongly encourage you to upgrade to their $200/mo plan. But you can likely push tens of terabytes per month on their free or $20/plan.
