I love posts like this because, love it or hate it, it gives me a checkmark of thoughts to compare my choices to.
I see three possible approaches, all with their advantages and disadvantages. (Of course people may fit between them with a mixture of attributes.)
1. Monolithic - Build it all yourself, purpose built and high performance. This is why mailinator and plenty-of-fish are able to produce high thruput on a surprisingly small number of machines.
2. Confederated - Completely distributed. Each machine is its own monolithic platform with everything from DNS to database, including web server on that node, but a cluster of nodes gives you scalability, and workload is distributed across the cluster. (I'm not aware of any examples of this, which is why I'm building Nirvana.)
3. COGS: You build your cluster of machines by architecting a system whereby you minimize (but not eliminate) single points of failure. You have N web server machines and X database machines and you seek out really high performance open source cogs to keep the number of machines low (e.g.: Redis, MongoDB, etc.)
The COGS approach is often taken with the idea that we need something really fast. MongoDB being fast (and "SQL") are the reasons its often chosen. Redis being fast is given as a key advantage (which is relevant for an in memory database, sure.) Node.js is often chosen for similar reasons.
But the ends result of the COGS approach is a brittle architecture. You may have multiple redundant web servers but the thing that distributes loads is a SPF. More specifically the architecture is complicated- each machine has a different configuration, etc.
With Monolithic, you get performance, and save hosting costs, and you can probably scale pretty well because you know your system really well, and you've squeezed out a lot of the inefficiencies that come from being generic (in the cogs approach) such that you can interoperate.
What I think we should see more of is confederated- no machine is a unique snowflake. Every machine is identical to every other machine. This way configuration becomes dead simple-- just replicate your model node, bring it up and data and load starts going to it.
This can be done with cogs- but they have to be fully distributed cogs. An example is Riak (Which hit 1.1 yesterday) which is open source and written in erlang and probably loses to nodeDB in every single single node benchmark you can come up with (not that the Basho people have designed it to be slow, quite the contrary.) But where's the fully distributed web platform for such an architecture? (If you have an answer, please make this question non-rhetorical. I'm putting a lot of time into building one because I couldn't find one.)
An interesting thing about the confederated approach is, because each machine is identical, it could be built in a monolithic fashion. Thus super optimized for its purpose. I'm using a sorta cogs approach because there are many good erlang cogs to use in my project.
But I think the big mistake is to focus on single node performance these days. Servers are relatively cheap, and you need more than one anyway for redundancy, so might as well have a cluster and no single points of failure.
> 2. Confederated - Completely distributed. Each machine is its own monolithic platform with everything from DNS to database, including web server on that node, but a cluster of nodes gives you scalability, and workload is distributed across the cluster. (I'm not aware of any examples of this, which is why I'm building Nirvana.)
It looks like Tumblr is doing something like what you're describing, at least for their dashboard. [1]
If you go down to "Cell Design for Dashboard Inbox", it seems like the architecture is to have one (maybe a cluster) of machines function as an independent platform and map users into individual "cells". Their "Dashboard Inbox" is the true meat and potatoes of their product.
A fairly good example of the confederated system running website hosting using CPanel. By default every machine hosts its own mail servers, web servers, databases, ......
Surprisingly (or perhaps not) its actually very effective for cheap hosting. Machines can handle quite a lot and rather than having many points of failure and failovers, you have one, and only one, single point of failure for the system (excl network connectivity/power/etc.)
Every couple of years something goes horribly wrong, but it turns out most users can deal with (or rather -- have no choice and are willing to accept) a five-ten hour downtime every couple of years.
>2. Confederated - Completely distributed. Each machine is its own monolithic platform with everything from DNS to database, including web server on that node, but a cluster of nodes gives you scalability, and workload is distributed across the cluster. (I'm not aware of any examples of this, which is why I'm building Nirvana.)
A bit late to this conversation, but the Opa platform appears to be an attempt at this:
The front page talks about MongoDB integration, but when it was first announced it sounded like the DB was integrated into the framework. Maybe they changed it since, don't know. It's popped up on HN a few times last year in case you want to dig up those discussions.
I think the article was talking more about how not every server has to be a webserver. For example, linkedin's social graph is a specialized server that keeps the entire graph in local ram. There is value in building a specialpurpose server when scalability benefits by it.
For the life of me, I can't edit the above post. 5 attempts, all eaten by the HN monster.
When talking about benchmarks, I meant to say:
"and probably loses to nodeDB in every single one-machine benchmark you can come up with (not that the Basho people have designed it to be slow, quite the contrary.)"
Finally the conclusion of the post:
Further, the only inter-machine communication should be at the database layer. (very little needed elsewhere) so as a result:
1. Confederation of nodes that talk to each other to keep the databases consistent. With RIAK this means that one update only affects three nodes (with n=3).
2. The whole layer cake, DNS to Database, including web server, application services, queues, etc, is present on every machine. No single point of failure (if you use DNS round robin to spread load, otherwise your load balancer is the SPF).
3. Further, this layer cake, while composed of erlang cogs is mostly monolithic, as far as the programmer is concerned. You write your code in javascript or coffeescript and push it into the database. Each node is like a google App Engine, and you write handlers, and those handlers are run concurrently across the system.
4. Want more performance? Add servers, or upgrade some of your servers. Node went down in the middle of the night? Go back to sleep. (Or write a simple client test you can run from your iPhone to make sure there wasn't some sort of cascading failure and the service is still up.)
5. Almost zero operations, zero cluster architecture engineering work. No worrying about EBS being terribly slow (go dedicated machines for these nodes, btw) or any of that hassles. The engineer just writes javascript handlers and worries about CSS and javascript for the browser, etc.
For me, that idea is nirvana. Nirvana may not be right for you... but confederation has to be the future.
The systems I work on are pretty sizable in scale, right now we are doing about 120 million calls a day on one of core clusters. I don't claim to be the end all authority on scaling, but I do have some observations based on what I've dealt with.
First off, that type of architecture is brittle to logic changes. If you have a completely static architecture that you don't need to change it may be fine, but deploying those changes to every machine in the cluster is problematic.
Second, not all components have the same underlying machine requirements. For instance, our nginx servers don't need much ram, but the HAProxy load balancers with nginx that terminate SSL need a lot of Ram and a good chunk of CPU. Hadoop works better with JBOD (Just a bunch of disks), whereas cassandra seems to work better with a raid 0 configuration. Certain layers like the Nginx through certain paths have real time requirements which means ultra low latency. Other things need to operate against massive data sets and compute answers in a few hours.
So, not every machine in your architecture can have all of the services required by every other part of your architecture. A lot of it depends on workload types and what the underlying requirements are for your system. There are many more reasons, but I'll leave it at those for now. Ultimately the post is right that a single machine can work very well, but it's also misguided in just dismissing the HN commenter. There are many other cases where distributed architectures are required. Guaranteeing robustness and performance in the face of service and machine failures is very difficult, and is essentially impossible on a single machine.
Which approach you apply depends, it depends on the unique situation and the unique constraints you have. Using a single model to solve all problems seems to be worse than using no model. Learn multiple models, learn how and when to apply each (yes - for those of you in the class, I'm taking the model thinking class :) ).
Yeah, I don't see a reason why the entire stack has to be on one machine. It simplifies configuration a little but there's a trade off in the granularity of resources allocation that comes with the tiered approach.
1st, web servers are are distributable by design. If the web app in question can not be distributed, then a reread of Fielding's dissertation is in order:
So if the web tier is designed correctly and is simply a state-less, data transformation engine then what's left is making your data tier be distributable and elastic.
Even better we could put an elastic web service layer between the frontend web tier and the data storage tier to provide an even better separation of presentation and logic.
In this set-up I believe we have the best of both worlds, the elasticity of the confederated design because each tier is designed to be fault tolerant and elastic with the resource allocation granularity we get with the tiered clusters.
I see three possible approaches, all with their advantages and disadvantages. (Of course people may fit between them with a mixture of attributes.)
1. Monolithic - Build it all yourself, purpose built and high performance. This is why mailinator and plenty-of-fish are able to produce high thruput on a surprisingly small number of machines.
2. Confederated - Completely distributed. Each machine is its own monolithic platform with everything from DNS to database, including web server on that node, but a cluster of nodes gives you scalability, and workload is distributed across the cluster. (I'm not aware of any examples of this, which is why I'm building Nirvana.)
3. COGS: You build your cluster of machines by architecting a system whereby you minimize (but not eliminate) single points of failure. You have N web server machines and X database machines and you seek out really high performance open source cogs to keep the number of machines low (e.g.: Redis, MongoDB, etc.)
The COGS approach is often taken with the idea that we need something really fast. MongoDB being fast (and "SQL") are the reasons its often chosen. Redis being fast is given as a key advantage (which is relevant for an in memory database, sure.) Node.js is often chosen for similar reasons.
But the ends result of the COGS approach is a brittle architecture. You may have multiple redundant web servers but the thing that distributes loads is a SPF. More specifically the architecture is complicated- each machine has a different configuration, etc.
With Monolithic, you get performance, and save hosting costs, and you can probably scale pretty well because you know your system really well, and you've squeezed out a lot of the inefficiencies that come from being generic (in the cogs approach) such that you can interoperate.
What I think we should see more of is confederated- no machine is a unique snowflake. Every machine is identical to every other machine. This way configuration becomes dead simple-- just replicate your model node, bring it up and data and load starts going to it.
This can be done with cogs- but they have to be fully distributed cogs. An example is Riak (Which hit 1.1 yesterday) which is open source and written in erlang and probably loses to nodeDB in every single single node benchmark you can come up with (not that the Basho people have designed it to be slow, quite the contrary.) But where's the fully distributed web platform for such an architecture? (If you have an answer, please make this question non-rhetorical. I'm putting a lot of time into building one because I couldn't find one.)
An interesting thing about the confederated approach is, because each machine is identical, it could be built in a monolithic fashion. Thus super optimized for its purpose. I'm using a sorta cogs approach because there are many good erlang cogs to use in my project.
But I think the big mistake is to focus on single node performance these days. Servers are relatively cheap, and you need more than one anyway for redundancy, so might as well have a cluster and no single points of failure.