Vurtualization is not for production. Why to have this useless layer, which messes up your CPU caches even more, interfere with you IO and complicates memory model? What for?
Virtualization was build for server providers to make easy money, not for server owners to gain performance advantages.
Vistualization is not for production. Production servers need less code, not more.
It is the same kind of mistake as JVM - we need less code, integrated with OS, not more "isolated" crapware which needs networking, AIO and really quick access to the code from shared libraries.
And, of course, a setup without middle-ware (python-wsgi, etc) and several storage back-ends (redis, postgres) is meaningless.
Update:
Well, production is not about having a big server which is almost always 100% idle, and can be partitioned (with KVM, not a third-party product) to make a few semi-independent virtual servers 99% idle. This is virtual, imaginary advantage.
On the other side, your network card and your storage system cannot be partitioned efficiently, despite all they say in commercials. And that VM migration is also nonsense. You are running, say, a MySQL instance. Can you migrate it without a shutdown and then taking a snapshot of an FS? No. So, what migration you're talking about? It is all about your data, not about having a copy of a disk-image.
It is OK to partition development, or 100% idle machines - like almost all those Linode instances, which have a couple of page request in a day - this is what it was made for, same as old plain Apache virtual-hosting. But as long as one needs performance and low latency, all the middle-men must go away.
You have no idea how modern virtualisation works. Go read about hardware assisted virtualisation on x86/x86-64.
Most server operators don't care about performance. They have performance coming out of their ears. They care about redundancy and maintenance, or to put another way cost centres.
Your post is on the wrong side of history. Virtualisation is being rolled out in a massive scale right now. Essentially you can abstract your entire physical infrastructure away from your logical infrastructure.
You have a physical server die? The HV has already moved the image to a new node and started it. Before you even receive the e-mail notification the new server is already booting.
So now a hardware failure goes from being a massive panic, to being a small annoyance. You pull the dead hardware from the rack, and plug a new generic node in and that now becomes available for the HV to use.
You want to back up a server? Take a copy of the ENTIRE image in one go.
You want to deploy a template? Well that's trivial with images.
You want to do change management with the servers? Just put the images in GIT. Boom done.
What you're suggesting is essentially taking the cheapest bits of server management (i.e. the physical hardware) and acting like they're the most expensive bits (i.e. people, time, and flexibility).
Google doesn't need virtualisation because Google is already deploying a large number of identical nodes which can be easily pulled and replace (i.e. Google's infrastructure is identical to virtualisation, but without the need for it).
Most businesses don't have hundreds of identical servers and services. They have a few dozen very specific or niche ones which need high up-time. This is one area where virtualisation can play a great role.
Another example is someone like a host where they want to distribute resources without any human intervention (e.g. Virtual Private Servers, shared hosting, etc).
All in all you're now starting to see data centres turn into "dumb" hardware farms, with the logical design and deployment being handled up-stream. This even extends to things like networking (routes, switches, etc - all centrally controlled).
So, unless we are selling CPU hours, like owners of a mainframe or a hosting provider, we don't need any virtualization?
Is it possible that the inefficiency and added unnecessary complexity makes a virtualized servers unusable for the most common server's tasks, due to I/O interference and cache/memory access complications?)
I never said anything about selling CPU hours. It has nothing to do with that. In fact my example about organisations directly contradict that.
It has to do with organisation and about being able to abstract logical servers away from physical hardware.
There is very little inefficiency (see hardware assisted virtualisation point above) and very little overt complexity (go play with any modern HyerVisor solution).
As I said above you don't understand how virtualisation works. Your points about "I/O interference" and "cache/memory access complications" just make you sound ignorant.
Well, I'm really ignorant, when it comes to meaningless sentences like organisation and about being able to abstract logical servers away from physical hardware. You're probably right.
On the other hand, I'd been involved in a few projects, which includes optimization of a big centralized databases, so, I think, I know a bit about flows of data, access patterns and where the bottlenecks are (hint: around serializing and scheduling low-level I/O operations).
Try to look under the surface structure, which plain words are.)
You talk about edge cases. And you actually have yet to provide any data to back up your statement when it comes to penalty because of running it in a virtual environment.
>Go read about hardware assisted virtualisation on x86/x86-64.
To further on this excellent and valid point, very cheap virtualisation was available on a lot of other very solid and very productive architectures and has been for decades; it was just x86 finally catching-up late. There are a whole lot of reasons speaking FOR virtualisation and only a few very specific applications where it might be a bad idea. I have no idea what OP up there is all about and against it, they make no sense.
How is this the top comment? I'm sure some of your points may be salient, but practical experience says virtualization works just fine for production. Same with the JVM.
Nginx and jetty are not the same thing. Nginx is a web server optimized to deliver content. Jetty is an application server built to run the applications. Both can be used together.
Also...Is Amazon EC2 considered virtualization for you?
I agree with you. I don't understand how this comment got so high with such short-sighted arguments, I would expect people here to have a little more insight.
Use virtualization (or LXC containers) for ease of management and to better allocate complementary workloads according to the resources they require and to how redundant you need them to be.
Chances are your physical boxes will have one resource almost always 100% idle and having your workloads virtualized allows you to better use them.
Also, virtualization allows you to more easily fix problems - you just delete the box and rebuild it from a base image and the packages you need. Reimaging a physical box is a pain.
Obviously, if you are into HPC and very low-latency, you need physical hardware. You probably need some specialty networking hardware too and maybe less OS than most of us would like to have. But then we are no longer in the general purpose amd64 box world anymore.
I remember one instance where we were having some trouble with one batch-processing workload. The amd64 box we had for that was not enough, despite having much more memory and processing cores than its peers. I suggested we moved the workload to an almost abandoned Itanium box we had played with for some time. The HP machine crunched the workload more than 3 times as quickly. I noticed the working datasets could fit completely inside its (then) huge L3 cache (16MB, IIRC) but not in the 4MB of the amd64 machine, more or less eliminating the external memory bus penalties.
The first three paragraphs are just a marketing-speak.)
The last one is more interesting. Have you noticed that your solution was in a splitting workload, not in sharing it with other processes?
In general, splitting (dividing/partitioning/isolating) is a good idea, sharing (sharing resources) is a bad one.
Sharing, unless your data is read-only, like a txt segments of code, is a source of problems, not a solution. Partitioning, on the other hand, is a universal, natural way.
You cannot dismiss arguments as marketing-speak just because they don't fit your high-performance viewpoint, and "marketers" have made similar arguments in the past.
The first two paragraphs are about one benefit: using otherwise idle resources. At my lab, we have five powerful servers for data-processing experiments and non-time-critical analytics. Most of the time they sit completely unused. By installing Xen and running things in virtual machines, we have been able to also put tens of differently configured web servers for various small projects on each.
The third paragraph is also very valuable from experience. It has been very useful to keep documentation around how different servers are set up, and verifying that the actual setup still matches the documentation. This has been easiest to ensure by having scripts that create a new VM, install packages and make configuration changes - if in doubt, completely delete the VM and re-create it from scratch with a single command-line.
I think you overstate how much performance hit you get by running things virtually in ex. KVM or Xen. With the first, it's almost negligible, as you don't have to run a paravirtualized kernel in order to gain full access, so for Linux you are basically just limited by other factors, such as disk IO long before anything else start to kick in.
I don't buy your argument about latency one bit, do you have ANY data to back up your statement? You know - Google and many other big players run huge virtual machine clusters, you'd think they wouldn't if that almost unmeasurable difference in latency had an impact.
I would run KVM on top of a machine, even if there was only to be one VM on it, just because moving, backing up or scaling up or down can easily be done. You can also easily migrate the host to another physical machine if you are experiencing hardware issues, or just upgrading your rig. And yes, you can migrate a running server, I'm doing it all the time.
And yes, you can snapshot and take a backup of a running server, but what goes on inside of MySQL, or other applications for that matter, has nothing to do with the consistency of the disk image. When you snapshot a disk image, you naturally don't know whats in RAM.
but what goes on inside of MySQL, or other applications for that matter, has nothing to do with the consistency of the disk image.
All you can do is instruct MySQL to pause, flush all its write buffers, and then take a snapshot of FS, then move it on. But this procedure has nothing to do with whether or not it runs under, say, VmWare or not. It has anything to do with does this particular disk volume supports FS snapshoting.
The next question is - what use of that snapshot when your system crashes and you got lost all the changes made since the last snapshot? How a VMWare helps you here?
Now consider what a bottleneck naively virtualized (represented as a file in a host system) disk volumes become, when I/O operations on, say, your DB's physical (transaction) log got interfered by I/O operations of your syslog daemon, or whatever other activity is going on.
In a database world the solution is about decoupling, partitioning and avoiding any I/O sharing possible. So, virtualization is just another layer of complexity which makes everything less predictable and controllable.
Have you actually used any of the products you are talking about? You don't take the filesystem snapshots on the virtual machine, but of the disk image on the host, typically using LVM or something similar. What you want to do with this snapshot later on is up to you, normally people just export it to a remote location.
For total system failures you still need a proper backup solution on the machine. This is true even if it's dedicated hardware or a virtual machine.
You seem to always resort to talking about databases, and this might be true for huge centralized databases, but that is a pretty damn specific task. Also, I thought we were past the "put everything on one box"-model.
The complexity you talk about is just not there. It acts and feels just like a normal machine, and you have yet to provide any data that would support your claim, even for edge cases.
I might have exaggerated a little, I have no idea how huge the virtual machine park is, but I would argue that they wouldn't invest in Ganeti with full-time employees if they weren't using it internally. During a talk, one of the developers said that they were running pretty large clusters.
EC2 seems to be doing fine on virtualization. A service of that kind would be impossible to run without a VM layer inbetween. So yes, you pay a performance penalty, but you get something in return: Flexibility. And for some use-cases flexibility is far more valuable than raw performance.
This sort of feels like an argument with someone who prefers to optimize the hell out of things in the best possible way. Thats a fair position to have and to take in this day and age.
What virtualization allows you to do is optimize the organizational responsibility. If I am developing internally on my VM, when it gets certified by ops management, and rolled out onto the live VM stack, I can be very sure what I am being responsible for; the ops also can be sure (because they maintain a huge VM filesystem, mostly) that they don't have to deal with things at a very thin slice.
For web operations, look, its simple: VM is good enough because there is so much overhead all over the typical web stack that a few frames of difference are, largely, irrelevant to the use case. Okay, don't put your multiplayer live realtime universe hashes in a VM; those belong bare metal. But the web front ends that are serving the same old content, over and again .. these are well worth putting in a package which can be massively deployed at whim.
A typically well-packed VM, consisting of only the hand-optimized built image of a development team with this in mind, can be a very, very tight package. I've seen reflectors and mongodb proxies and so on, packed into 256meg boot image that can be replicated simply by giving it a new name .. deploy 2,000 copies of these VM's, and you have a massive solution to the front-door problem..
> And that VM migration is also nonsense. You are running, say, a MySQL instance. Can you migrate it without a shutdown and then taking a snapshot of an FS? No.
We have an entire 2 million GBP VMware estate which I inherited
I agree entirely with this comment. Between lun size limits, license management (which is hell), the general additional work and the fact that the benefits are suspect, I agree.
Processes and multiprogramming were invented to virtualise batch machines. That's as far as it needs to go.
Go to any big company's datacenter and you'll be sure to find some virtual machines that are in production. There is some performance penalty for sure, but the added flexibility and reliability covers that neatly. If your VM host seems to be failing, you can live migrate your VMs off that box and take it down for maintenance. That's not an option when running your services on bare metal. On flexibility side, it is very powerful to be able to allocate resources depending on need, not needing to buy specific hardware and hope that it's a good fit for the workload it runs.
Virtualization is most definitively not useless, as any sysadmin managing more than a couple servers will tell you. The advantages are so many that in fact it should be the default go-to option when provisioning a server (even a single-purpose one).
Many if not most of the production servers on the web run some sort of virtualization; Xen alone powers EC2, Rackspace, Linode, Rimuhosting and many others. In my experience it only adds about 3% of CPU overhead as a disadvantage.
This is a pretty bold statement to make considering the fact that there are a lot of companies who do use virtualization in production and do so quite successfully.
Not true, for a startup, virtualization is the most cost effective way of hosting applications/websites. We have a physical server with 32GB RAM and on it four virtual machines for different purposes. None of them breaks any sweat.
Virtualization was build for server providers to make easy money, not for server owners to gain performance advantages.
Vistualization is not for production. Production servers need less code, not more.
It is the same kind of mistake as JVM - we need less code, integrated with OS, not more "isolated" crapware which needs networking, AIO and really quick access to the code from shared libraries.
And, of course, a setup without middle-ware (python-wsgi, etc) and several storage back-ends (redis, postgres) is meaningless.
Update:
Well, production is not about having a big server which is almost always 100% idle, and can be partitioned (with KVM, not a third-party product) to make a few semi-independent virtual servers 99% idle. This is virtual, imaginary advantage.
On the other side, your network card and your storage system cannot be partitioned efficiently, despite all they say in commercials. And that VM migration is also nonsense. You are running, say, a MySQL instance. Can you migrate it without a shutdown and then taking a snapshot of an FS? No. So, what migration you're talking about? It is all about your data, not about having a copy of a disk-image.
It is OK to partition development, or 100% idle machines - like almost all those Linode instances, which have a couple of page request in a day - this is what it was made for, same as old plain Apache virtual-hosting. But as long as one needs performance and low latency, all the middle-men must go away.