Muli Ben-Yehuda's journal

10/12/06 – the big news is that CalIOC2 DMAs are now going through!
the HW guys hooked up a logic analyzer and figured out that Linux is
misprogramming the bridge to have a base/limit of 0/0, which caused a
nasty interaction with the 0-1MB MMIO region on the bridge, since TCE
addresses are allocated from 0 and up. The
workaround is simple, start allocating TCEs from address 1MB and up,
and voila, DMAs start coming through.

I’m now trying to
understand why Linux is misprogramming the bridge. It only happens
when TCEs are enabled, and a diff of dmesg from the same kernel with
TCEs enabled and disabled has this to say:

+PCI: Ignore bogus resource 7 [0:0] of 0000:0a:00.0 
+PCI: Ignore bogus resource 8 [0:0] of 0000:0a:00.0
 PCI: Bridge: 0000:0a:00.0
-  IO window: disabled.
-  MEM window: disabled.
+  IO window: 0000-0000
+  MEM window: 00000000-00000000
   PREFETCH window: d4100000-d41fffff

After I figure this one out, I need to implement a new cache flush
routine for CalIOC2 and another workaround the designer recommends,
stress test it, and then if all goes well it can go in, probably for
2.6.21.

In other news, my hacker’s diet and workout regime is working very
well. I’m losing weight at a steady rate of 0.6kgs per week and
feeling better every day. Starting the day with a walk on the
sea-shore before the sun rises, masochistic as it may be, is also
highly enjoyable. The more I suffer, the more I enjoy it. I’m going to
add swimming too next week, albeit in a heated pool, not in the cold
Mediterranean.

In other other news, we decided to go ahead with the USENIX
paper. Deadlines, here we come!

05/12/06 – took half a day off and spent it pleasantly hacking the now-defunct Hacker’s Diet Tracker to run in my environment.

04/12/06 – finally got back to Xen-IOMMU hacking. Fixed the xenlinux tree to boot as domU, not just dom0, and cleaned up the IOMMU initialization code to allow multiple IOMMUs to be compiled in and used at run time (e.g., Calgary used by default but use swiotlb if iommu=soft is specified). Next step is assigning a PCI device to the xenlinux kernel that has translation enabled in the IOMMU and settling back to watch the fireworks. In the evening did a quick backport of the 2.6.20 Calgary tree to the SLES 10 SP1 kernel (based on 2.6.16.29). When I last dealt with SLES kernels a few years back they were crammed full of weird patches that made them look nothing like mainline. I was glad to see the situation has improved.

Summary of last week: continued working on CalIOC2 support. It’s pretty obvious that we’re seeing a HW problem and the ball is in the HW folks’s court. Switching back to xen-iommu for the time being.