Muli Ben-Yehuda's journal

When I was younger, I dreamed of going to MIT. Steven Levy’s Hackers fired my imagination and made me wish I could be there. Having just finished reading Pepper White’s The Idea Factory: Learning to Think at MIT, I’m glad it hasn’t happened.

This weekend I made the final changes to the OLS paper and pushed them to svn. It’s not that it’s done, it’s that I don’t have the energy to work on it any more and thus have pronounced it done.

In celebration, I then pronounced today “general cleanup day”. I threw away the two feet high pile of old papers that was keeping my desk from floating, applied every outstanding patch for various trees and finally wrote and submitted the trident fix for bugzilla 8172. This must be some kind of negative personal new record – almost two months from reported to fixed.

Plans for tomorrow: resume going for long walks in the morning (there, it is written, and thus I am now COMMITTED) and finally get the CalIOC2 and other outstanding Calgary patches in suitable shape for mainline submission. Also, start working on the “to-read” paper pile, which has grown to an alarming size.

I had a passion for the content of physics, but I was also possessed
by a hungry ambition for its earthly rewards. Both passion and hunger
persisted over the years, despite the inevitable disappointments. Ten
years later, as a postdoctoral researcher at Oxford in 1976, I
experienced a minor epiphany about ambition’s degradation. At age 16
or 17, I had wanted to be another Einstein; at 21, I would have been
happy to be another Feynman; at 24, a future T. D. Lee would have
sufficed. By 1976, sharing an office with other postdoctoral
researchers at Oxford, I realized that I had reached the point where I
merely envied the postdoc in the office next door because he had been
invited to give a seminar in France. In much the same way, by a
process options theorists call time-decay, financial stock options
lose their potential as they approach their own expiration.

From My
Life as a Quant: Reflections on Physics and Finance, by Emanuel
Derman.

Dear diary. Wow, its been a long time since I’ve updated you. I kept meaning to but … life got in the way. I’m writing this from the lounge at Ben-Gurion airport, waiting for the boarding call to the flight to NY. This is the first time I’m leaving home for travel since Yael was born. Everyone kept telling me it’s different, and I didn’t believe them, but it is… I haven’t even boarded the plane and I already miss Orna and Yael something fierce.

Then why am I leaving, you ask? I am headed for the Xen summit, taking place at the IBM T. J. Watson research center in upstate New York, where I will be giving a talk on IOMMU performance (or lack thereof…). What’s that? what’s so lacking in IOMMU performance? well, I could tell you, but the data is still preliminary and we’re still analyzing the results. At the summit I’ll be presenting preliminary results and some thoughts for the future — the real deal will be presented at OLS ’07.

There’s the boarding call. Later, diary dearest.

Maybe I’ve been dealing with IOMMUs for too long, but Joanna Rutkowska’s Black Hat DC 2007 presentation “Beyond the CPU: Defeating Hardware Based RAM Acquisition” seems to state the obvious in several different – sensationalist – ways. If you have a PCI device that does DMA, its memory accesses are controlled and potentially redirected by the various controllers between the device and memory. Well, duh.

The comments quoted in this blog entry are fairly amusing too.

Gaaarg, been sick. Yael caught a nasty bug at my sister’s wedding and passed it on to me and Orna. Better than the other way around!

Today’s paper is “Unmodified Device Driver Reuse and Improved System Dependability via Virtual Machines”, by Joshua LeVasseur, Volkmar Uhlig, Jan Stoess, and Stefan Gotz. The paper is a bit hard to decipher, but ultimately pretty interesting… it presents “Device Driver Operating Systems (DD/OS)” – “driver domains”, in Xen parlance – where the hardware drivers reside in virtual machines and “clients” connect to the driver domains through a “translation module” – the familiar Xen frontend/backend drivers scheme.

At first I was pretty excited to read this paper, as I thought it dealt with direct hardware access from *unmodified* fully operating systems (i.e., fully virtualized), which is something I’m very interested in. Once I continued reading however it became apparent that the authors make a distinction between unmodified drivers and unmodified operating systems, and limit their implementation and evaluation to the para-virtualized case, which is not as interesting.

Still, notable points:

– they presented a nice work-around for the case where no IOMMU is available (and thus no DMA remapping) – allocating DMA’able pages from a special pool of pages for which physical address is equal to the machine address, which may or may not be shared between driver domains.

– they present a nice hack for non-isolation-capable IOMMUs where the single IOMMU context is multiplexed between different driver domains. This has some implications related to PCI bus latencies – not every device will tolerate not being able to DMA for a prolonged period of time.

– the abstract claims “network performance within 3-8% of a native Linux system”, which is pretty amazing considering the numbers I’ve seen for Xen’s network performance. In the evaluation section we learn that *throughput* is within 3-8% of a native Linux system, but *CPU utilization* is 1.6 times to 2.22 times worse(!) than native. That makes a lot more sense…

Trying to get back in the habit of writing here, I decided to (re)start posting interesting articles and papers.

“Roadmap to a Failure-Resilient Operating System”, by Jorrit N. Herder, Herbert Bos, Ben Gras, Philip Homburg, and Andrew S. Tanenbaum. Interesting article, although not much new. MINIX 3. Drivers is major source of operating system bugs, therefore we move the drivers to a separate address space and contain the bugs. Mentions DMA, with IOMMUs as the solution, no discussion of how it fits in with the rest of the system. Performance Degradation of 5-10 percent when compared to MINIX 2 with in-kernel drivers, no comparison against Linux or other OS’s. See also Comparing Linux and MINIX on lwn.net.

Random thought: what if we ran drivers each in its own address space, but all with supervisor privileges? Is this a worthwhile point to explore in the performance/protection tradeoff?

Leendert Van Doorn, formerly a Senior Manager in IBM’s Research Dision, writes about the reasons he left IBM.

I had the good fortune to work with Leendert and I have a profound personal respect for him, so this is certainly thought provoking.

It’s amazing how time flies. I haven’t updated this thing in ages.

I’ve been working with Yaron Weinsberg on a paper for HotOS XI, which is just about done. I’ve also been working on the IOMMU perf paper for USENIX, which we’ve decided to target at OLS instead.

On the non-paper front, I’ve been taking a much needed break from Calgary, except for dealing with emergencies. I also spent a few days doing userspace stuff related to Xen live migration, which was a nice break from the rigors of kernel programming.

Next year I’ll be continuing the Calgary work for the LTC, while also working closely on a new and exciting project with the venerable Orran Krieger.