Muli Ben-Yehuda's journal

October 4, 2010

Filed under: Uncategorized — Muli Ben-Yehuda @ 8:57 AM

I was hurrying down the Newark airport terminal, wondering whether I
was going to make the connecting flight to Seattle, en-route to
Vancouver for the 9th USENIX
Symposium on Operating Systems Design and
. Suddenly, my cell phone rang. It
was Michael
, a long-time co-worker and mentor. “Have you seen the
email?” “No, I just landed in Newark and am on the way to catch a
connection to Seattle. Which email?” “Here, let me read you the

Dear Authors,

Your paper has been selected as one of two
winners of the OSDI Jay
Best Paper award.”

Receiving this award is a unique experience and a great honor. It is
doubly sweet because of all the research projects I’ve worked on, the
Turtles nested virtualization project is perhaps the one I am most
proud of. When Orit, Ben, and I started working on it in 2008, we set
out to do the impossible. Many colleagues claimed that efficient
nested x86 virtualization on the Intel platform could not be
done. Eventually, working long and hard, and with help from friends,
we showed that not only could it be done, it even performs well. I’ve
learned a lot in the process, about x86 virtualization, about leading
a team, and about the art and craft doing research, but the most
important lesson was to never lose hope, to always believe that
eventually, it will work. And guess what? It did!

If you want to know how we did it, and what we learned in the process,
check out The Turtles
Project: Design and Implementation of Nested Virtualization

In classical machine virtualization, a hypervisor runs multiple
operating systems simultaneously, each on its own virtual machine. In
nested virtualization a hypervisor can run multiple other
hypervisors with their associated virtual machines. As operating
systems gain hypervisor functionality—Microsoft Windows 7 already
runs Windows XP in a virtual machine—nested virtualization will
become necessary in hypervisors that wish to host them. We present the
design, implementation, analysis, and evaluation of high-performance
nested virtualization on Intel x86-based systems. The Turtles project,
which is part of the Linux/KVM hypervisor, runs multiple
unmodified hypervisors (e.g., KVM and VMware) and operating
systems (e.g., Linux and Windows). Despite the lack of architectural
support for nested virtualization in the x86 architecture, it can
achieve performance that is within 6-8\% of single-level (non-nested)
virtualization for common workloads, through multi-dimensional
for MMU virtualization and multi-level device
for I/O virtualization.

The scientist gave a superior smile before replying, “What
is the tortoise standing on?” “You’re very clever, young man, very
clever”, said the old lady. “But it’s turtles
all the way down!


  1. Congratulations
    Congratulations Muli…

    Comment by Anonymous — October 4, 2010 @ 4:49 PM | Reply

  2. Double congratulation

    Comment by Anonymous — October 4, 2010 @ 6:33 PM | Reply

  3. Congratulations! In fact, it does not totally surprise me, as I think it is a very impressive work 🙂

    Comment by aglik — October 5, 2010 @ 8:02 AM | Reply

  4. I’ve been reading/researching about things like the turtles project and hoping I could find a way for some time now.
    A job well done. Awesome work and Congratulations.
    Has the work(code) from the project been posted for consumption?

    Congratulations again.

    Comment by bk — April 21, 2011 @ 6:46 AM | Reply

  5. Hi there BK,

    The latest released code is available as a set of patches to kvm. See Nadav’s email to the kvm mailing list here for the details:

    Or just grab kvm from the github repository:

    git:// (branch “nvmx8”)

    Comment by mulix — April 21, 2011 @ 7:46 AM | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

%d bloggers like this: