Using a hash-table gives you pretty much the same end result
(usually you can refill the TLB with a single memory operation).
But locality is going to be worse, _and_ you have the overhead of
maintaining the hash table (which is not an issue for steady-state
things like long FORTRAN jobs, but it can be quite expensive if you
do lots of process creation or change your virtual memory maps
often).
Good point. The is reason #2 why I didn't use the TSB hash table
support provided by the UltraSparc mmu's. This means every time a
mapping changes, I would need to go and zap the TSB entry _as well_ as
the possible copy in the TLB hardware. No thanks, and this is why
Solaris is so damn slow on the Ultra.
Oh, and think about the issues of subsidiary caches like the TSB in
the presence of SMP. New synchronization problems, new locking
necessary, again no thanks.
Solaris does two L1 cache bypass loads in their TLB miss handlers
using the hardware assisted TSB. One of these loads gets at a truly
redundant piece of information (the TLB tag part of the entry, to
detect the TSB hit during miss processing).
I eat 3 L1 cache bypass loads, however none of what I look up is
redundant. And the clever way I have set things up makes the check
for "page not present" during the TLB miss wind up being one single
"branch on register value" instruction. The whole miss processing
fits in 1 L2 cache line (2 L1 Icache lines) for user TLB misses.
When solaris misses the TSB, it must go and search the large and
complex full HAT layer tables for this address space. I _NEVER_ eat
this overhead. This means that for all pure TLB misses, Solaris's TLB
miss processing overhead has no decidable upper bound, Linux's has a
perfect upper bound defined by:
1) Cost of missing L2 and Icache for tlb miss handler.
2) Cost of 3 L2 cache loads missing L2.
3) Approximately 6 or 7 actual integer operations to compute
offsets etc. during the page table walk.
So for a heavy data structure walking process, overall, the TLB miss
strategy of Linux on the Ultra should perform much better than the
Solaris one.
Oh, and also, kernel tlb misses for all physically mapped pages, costs
6 cycles, always ;-)
Later,
David "Sparc" Miller
davem@caip.rutgers.edu