I’m writing this on hydra, my laptop, which is running
muli@hydra:~$ uname -a Linux hydra 2.6.0-test1-shpte #2 SMP Tue Aug 5 16:25:42 IDT 2003 i686 unknown
Note the shpte, standing for Shared Page Tables. Neat!
TODO: port to 2.6.0-test2-latest, and verify that the changes I made to the code are correct.
Happy happy joy joy!
The OLS 2003 talk yesterday went reasonably well. I have a tough time explaining shared page tables in 30 minutes, partly because I don’t understand everything as well as I thought I did. Orna’s part of the talk was better, IMHO.
Yesterday morning I resigned from the board of directors of Hamakor, for reasons which can be summed up as “it was taking up too much time and mental energy from things I’d rather do, such as write free software”.
I am still a member and a firm believer in Hamakor. It’s the overhead and responsibility of being a board member that I can’t afford.
Yesterday was the long awaited August Penguin 2003.
I woke up at 0615 AM, after having gone to sleep very late the night before. There were a few last minute preparations to make, and then we were on our way. Just after we passed the turn to Natanya, Ira called me on the cellphone and asked if we could pick him up, since his car had some problems. We did a quick U-turn, picked him up, and then proceeded on our way to the convention.
When we got to the building where the convention was held at 0830 AM, there was already a line of people waiting to be registered. More than 40 people joined Hamakor during the day. I immediately started running around, seeing where I could help, setting up my laptop to work with the slides projector and otherwise setting stuff up. At 0930 we started, only half an hour late, with about 120 people present.
First up was Shachar, giving a “state of the NPO” address. I thought it was a bit long and a bit “business centric” for the crowd we had, mostly hobbyists, but it certainly got the points it was intended to accross. Then we split into two tracks, a beginners track and an advanced track. I was “in charge” of the advanced track, and thus didn’t get to experience the newbies track. Reports from the field indicate that the first part, a comparative flamewar^Wreview of linux distributions went quite well, and the second part, “how to connect to the net with Linux” didn’t go quite as well.
In the advanced track, the first speaker was Lior Amar, of the MOSIX team. He spoke about MOSIX with just the right blend of overview and technical details for the audience. Not much new there for me, but a very enjoyable talk. The second speaker was Mr. Bouncy, Moshe Zadka. moshez gave a great, energy filled, talk about pypy, implementing Python in Python.
After that, the two tracks joined together for a geek trivia contest, hosted by Gilad Ben-Yossef. I took part in the contest, and had loads of fun. We got points for the right answer, and you got points for a funny answer. Unfortunately, the other team, led by moshez, could name four more Linux distributions than us and thus won.
Next we had the gpg key signing party. Since we ran out of time in the hall, we had it in the lobby. It was hot, there was lots of noise and the inenvitable arguments about the best way to do the identification stage, but we carried it through.
Afterwards we went out for food, and then Orna and I met a couple of folks from Eddiea’s Linux Development workshop who are considering working on syscalltrack at a coffee place. To my great surprise and contradicting Murhpy’s law, a demonstration of syscalltrack actually worked. Wheee! 🙂
Other coverage of August Penguin:
Whatsup story. I even commented about why Lior’s MOSIX slide was given by powerpoint. In hebrew.
Tapuz Linux forum. In hebrew.
So, people have been asking me how OLS was, and I usually reply with “it was different than what I expected. Not good different and not bad different, just … different.”
OLS showed me a few things that needed showing, or at least reminding. It showed me that Orna and I are drifting on differnt paths. It showed me that talking about code is not enough, you have to actually write some, too (as if I didn’t know that… just needed reminding, I guess). It showed me that it’s all about what you know and what you do with it. Just knowledge is not enough, and just churning out code is not enough either. It also showed me how important people are, and that I badly need to work on my people skills. Funny conclusion from a hacker conference, isn’t it?
I’m very happy that I made it this year, and will definitely make it next year, regardless of who’s paying (this year, work did, and I highly appreciate it!). Long live OLS!
Sat 13:30
what we did on Friday night:
After rik’s talk, I sat and hacked on shared page tables for a few hours, since there were no interesting talks/bofs. Talked with Shawn Starr about adding sysfs support to ISA sound drivers, and with Orna and some other people and various other things. Eventually, people drifted off to the BOFs (Orna went to the Women in Linux BOF) and I went back to the room. Eventually, finished my book (Greg Eagan’s Diaspora) (ending kind of a let down) and went to sleep.
This morning I continued hacking on shared page tables, and talked with people, some in real life, most in IRC. I had a vivid demonstration of how much I learned in OLS when someone asked a question and I was able to answer, because this exact issue was discussed in the NUMA scheduler BOF. Hooray!
Around noon Orna wanted to get something to eat, and we went to the mall. Stopped in the music shop and bought tons of DVDs and some CDs, and then had fast food sushi. What a great invention! I hope someone imports it to .il.
Now listening to Kai talk about kbuild 2.5. Nothing new so far.
Disgressing: on the shpte front, I merged almost all of the code. I have three rejects in rmap.c, where the patch doesn’t fit any of the vanilla kernels from 2.5.60 to 2.5.75. Yes, I checked:
$ export prevnum=60; for num in `seq 61 75`; do mkdir linux-2.5.$num; \ cp -al linux-2.5.$prevnum/* linux-2.5.$num; (cd linux-2.5.$num && \ bzcat ../patch-2.5.$num.bz2 | patch -p1); prevnum=$num; done
$ for num in `seq 60 75`; do (cd linux-2.5.$num && patch -p1 $num.patchresults; done
$ grep -il rej *.patchresults
I need to find out what tree it’s against, or figure out from the patch snippets what the author meant. It’s code in the swap out path (try_to_unmap_one()), which I don’t know much about – yet.
Oh, Kai is talking about the interesting stuff now:
– make some/path/file.o object file – make some/path/file.i preprocessed – make some/path/file.s assembler – make some/path/file.lst annotated asm (cool!)
– make SUBDIRS=drivers/isdn/hisax modules
Makefile changes from 2.4 – 2.6: say any given thing just once. “descend into subdir to build those files / descend into subdir to link those files into an object file / link the object file into the kernel image” can be said once with “descend into this subdir”.
After the talk, stepped out to email a reminder about August Penguin’s key signing party and see if Gilad put up the book crossing page – he did, great.
A couple of gpg people snagged me to do the identification ritual, including Olaf Kirch, of SuSE fame, who actually remembered me from our discussion re syscalltrack. I was too amazed to remember to ask him if SuSE might be interested in it. Damn!
Now sitting in the bugzilla BOF, where people are discussing the scary query page (a usability nightmare) and a mail interface to bugzilla. Neat.
Fri, 15:00
Rik van Riel, previously of connectiva, now working for redhat, talking about toward an O(1) VM.
Machines get faster, but many operations get slower
page_launder: cleaning up pages (writing them to disk) in order to evict them from memory.
Split the inactive list in order to avoid scanning the entire list: – inactive dirty – pages might be clean or dirty – inactive laundry – ? – inactive clean – clean, just get rid of it
page aging: which pages from the active list to remove? 2.4-rmap uses LFRU approximation.
page aging: sort the active list based on level of activity of pages. many lists of pages, pageout moves each list one level down (toward inactivity), page_referenced bit moves a page up one list.
reverse mapping fundamentals:
regular reverse mapping (per page) simple, no corner cases set it up on fork, tear it down on exec – overhead!
object based rmap: rmaps non-existing ptes as well (because it’s per vma) needs to be searched in pageout path only works for linearly mapped file backed objects nasty interactions with truncate, remap_file_pages
conclusions: bottlenecks keep moving access patterns keep changing computers, processes keep growing
VM needs to be adjusted: more intelligent writeout of dirty pages better replacement algorithms beter search algorithms more scalable locking
O(1) VM is probably impossible, due to the problem space
Research required for a VM design that fits modern machines and workloads [maybe something I could work on…!]
Fri 10:05 AM
Greg KH is talking about “udev – devfs done right” (I wonder if rgooch [devfs maintainer] is here… No screams of rage so far, so I guess not)
problems with devfs: implementation (although hch cleaned it up in 2.5) no dynamic allocation
udev – combine /sbin/hotplug (notification whenever a device is added) and sysfs (information about devices) to do device addition/deletion to /dev and naming in user space.
very active crowd participation on this talk… I’m mostly tuned out, writing my notes. After the talk, sat outside, did the gpg identification ritual with a few more people, talked with Behdad, Orna and Pat Gaughen of IBM LTC in real life, and Con Kolivas on IRC. Freaky, the way IRC conversations and real life conversations blend in together.
Thu 16:30
Walked out of this one pretty soon after it started, due to the speaker’s indian accent and the light bulb that went up over my head – I can use xchg() to replace the syscall pointer atomically! Rushed out to finish my “how to port syscalltrack to 2.5” paper and email it to sct-hackers. Also Did the identity ritual with a few more people.
Today was the banquet dinner, sponsored by IBM. Had it with sarnold, mharris@redhat and another Redhat guy. The food was reasonable, the Alan Cox Stories were superb.
dinner speech was by Ian Stewart, of grid fame, and I spent most of it thinking about projects I could do independent research on:
– continue developing syscalltrack. – persistent scheduler. – storage intrusion detection.
Need to think more about all of them.
Muli Ben-Yehuda's journal 