Andrew Morton explains flex-mmap in “Re: ptrace single-stepping change breaks Wine” on lkml. Makes a lot of sense when you put it this way. The only downside is that some applications expect the old binary layout and break with the new one.
the ‘after’ layout guarantees that brk area (malloc()) can grow unlimited and mmap() can grow unlimited – they will meet somewhere inbetween when almost all of the VM is used up. [the ‘top’ of the mmaps in the ‘after’ layout is constrained by the stack ulimit – the stack must still fit and we never allocate into the stack’s yet unallocated and growable hole.]
with the ‘before’ layout we’ve got 900 MB for brk() and 1.9GB for mmaps() – a rigid limit.