> with Prefork and mod_php, every time someone was
> connecting I was spinning up a new Apache thread and a
> new PHP VM every single time (please correct this if
> wrong... this is how I understood it).
Since no one has yet... no, this isn't quite right. Apache in prefork mode maintains a pool of workers, each of which does contain essentially the full webserver and PHP system. These listen for requests and respond to 1 request per process per time. Once a process finishes responding to a request, it starts listening for a new one again. Apache tries to keep at least a few idle processes on hand all the time, so clients don't have to wait, and it will fork new processes to handle more simultaneous connections up to a configured limit. If you get a sudden spike in traffic which then dies down, Apache will start killing excess processes. Processes live until killed as unnecessary, or after serving N requests as configured.
That all sounds pretty reasonable, until you realize that every request, even for styleseets, scripts, images, favicon.ico or robots.txt, is served by one of those fat apache+php processes, which can easily reach 10MB of memory usage each (or a lot more sometimes, depending on your modules). And if Keep-Alive is enabled, or a client is a bit slow, or some part of the Internet gets dodgy, a process can get tied up for a long time, requiring Apache to start more, increasing your memory usage and leading to thrashing and/or hitting the max child processes limit.
Default max is 256 child processes, so easily enough to cause disk thrashing on a 1GB machine, and not enough to fully utilize a much larger machine.
Nginx can have workers, but the default number is one. nginx doesn't need a separate worker per time to handle each request. Anything that happens on a given connection is an event (connected, received some data, finished receiving data, got a response from the backend, closed the connection), and nginx just responds to events in the order received across all connections. It doesn't have to hang out and wait for the client to finish sending it's request, doesn't have to wait for PHP to process the request, just goes about handling the next event until those time-consuming things finish.
If you've looked at using PHP with nginx, you might be thinking of the fastcgi workers. Rather than including PHP in the web server process, fastcgi keeps a pool of PHP processes around, and whenever you get a PHP request, nginx passes it off to the fastcgi backend (then goes about other business until the backend responds). I don't know of any reason that PHP run this way should be faster than as an apache module, but the whole system ends up faster because the web server part is using a lot less memory and CPU. And depending on your site, there are some really low hanging optimizations you can do with nginx. E.g., just need a couple of configuration lines to cache all PHP GET requests for 5 min (unless you have cookie X, if you want). Since nginx seriously will not bat an eye over serving a redditing worth of static files, that means even your unoptimized word press site can stand up to the front page no problem. (Not verified from actually attempting it. But I did setup a simple PHP site on a 512 Linode that served 250k page views per day without excessive disk activity or the CPU usage crossing 10%.)
That all sounds pretty reasonable, until you realize that every request, even for styleseets, scripts, images, favicon.ico or robots.txt, is served by one of those fat apache+php processes, which can easily reach 10MB of memory usage each (or a lot more sometimes, depending on your modules). And if Keep-Alive is enabled, or a client is a bit slow, or some part of the Internet gets dodgy, a process can get tied up for a long time, requiring Apache to start more, increasing your memory usage and leading to thrashing and/or hitting the max child processes limit.
Default max is 256 child processes, so easily enough to cause disk thrashing on a 1GB machine, and not enough to fully utilize a much larger machine.