Real-world virtualization benchmarking: the best server CPUs compared
by Johan De Gelas on May 21, 2009 3:00 AM EST- Posted in
- IT Computing
vApus Mark I vs. VMmark
By now, it should be clear that vApus Mark I is not meant to replace VMmark or VConsolidate. The largest difference is that VMmark for example tries to mimic the "average" virtualized datacenter, while vApus Mark I clearly focuses on the heavier service oriented applications. vApus Mark I focuses on a smaller part of the market, while the creators of VMmark have invested a lot of thought into getting a nice mix of the typical applications found in a datacenter. We have listed the most important differences below.
| vApus Mark I compared to VMmark | ||
| vApus Mark I | VMmark | |
| Goal | Virtualization benchmarking across Guest OS, Hypervisor, and Hardware | Measuring what the best hardware is for ESX |
| Reproducible by third parties | No; for now it's only available to AnandTech and Sizing Server Lab | Yes |
| Modeling | "Harder to virtualize", "heavy duty" applications | A balanced mix of virtualized applications in the "typical" datacenter |
| VMs | Large "heavy duty" VMs; 4GB with 4 VCPUs | Small VMs 0.5-2GB, 1-2 VCPUs |
| Market coverage | Small but important part of the market | Large part of datacenter market |
| Relevance to the real-world | Uses real-world applications | Uses industry standard benchmarks |
The advantages of vApus Mark I are the fact that we use real-world applications and test them as if they are loaded by real people. The advantages of VMmark are that it is available to everyone and it has a mix of applications that is closer to what is found in the majority of datacenters. vApus Mark I focuses more on heavy duty applications.
There's one small difference between the existing benchmarks like VMmark and VConsolidate and our "vApus Mark I" virtual test. VMmark and VConsolidate add additional groups of VMs (called tiles or CSUs) until the benchmark score does not increase anymore, or until all the system processors are fully utilized. Our virtualization benchmark tries to get close to 100% CPU load much quicker. This is a result of the fact that our VMs require relatively large amounts of memory: each VM needs 4GB. If we used a throttled load such as VMmark or VConsolidate, we would require massive amounts of memory to measure servers with 16 cores and more. Six VMs that make up a tile in VMmark take only 5GB, while our four VMs require 16GB. Our current monitoring shows that this benchmark could run in 10-11GB, and thanks to VMware's shared memory technique probably less than 9GB. With four VMs we can test up to 12 physical CPUs, or 16 logical CPUs (8 Physical + 8 SMT). We need eight VMs (or two "tiles") to fully stress 16 to 24 physical cores.
Facebook
Twitter
RSS
66 Comments
View All Comments
JohanAnandtech - Friday, May 22, 2009 - link
Most of the time, the number of sessions on TS are limited by the amount of memory. Can you give some insight in what you are running inside a session? If it is light on CPU or I/O resources, sizing will be based on the amount of memory per session only.dragunover - Thursday, May 21, 2009 - link
would be interesting if this was done on desktop CPU's with price / performance ratiosjmke - Thursday, May 21, 2009 - link
nope, that would not be interesting at all. You don't want desktop motherboards, RAM or CPUs in your server room;nor do you run ESX at home. So there's no point to test performance of desktop CPUs.
simtex - Thursday, May 21, 2009 - link
Why so harsh, virtualization will eventually become a part of desktops users everyday life.Imagine, tabbing between different virtualization, like you do in your browser. You might have a secure virtualization for your webapplications, a fast virtualization for your games. Another for streaming music and maybe capturing television. All on one computer, which you seldom have to reboot because everything runs virtualized.
Azsen - Monday, May 25, 2009 - link
Why would you run all those applications on your desktop in VMs? Surely they would just be separate application processes running under the one OS.flipmode - Thursday, May 21, 2009 - link
Speaking from the perspective of how the article can be the most valuable, it is definitely better off to stick to true server hardware for the time being.For desktop users, it is a curiosity that "may eventually" impart some useful data. The tests are immediately valuable for servers and for current server hardware. They are merely of academic curiosity for desktop users on hardware that will be outdated by the time virtualization truly becomes a mainstream scenario on the desktop.
And I do not think he was being harsh, I think he was just being as brief as possible.