I wonder if you can't do a jolly good job by having emulators run under Linux with high RT priority and bypassing X by using the framebuffer? You can still have your other cores doing regular multitasking stuff in the background.
It would help even more, certainly. With really good setups, we're battling probably around ~100ms of latency when you combine everything I talked about together. If you were to do as you said, we could maybe drop another 10-20ms of that off.
