No. In fact, it cannot be in the TLB, because the page is not present, and
the intel TLB doesn't cache non-present pages.
The reason the fix works is that the page fault logic on the CPU correctly
reset the CPU state, including the internal "lock" bit. This isn't too
surprising, because there are _real_ instructions with a lock prefix that
can fault with a page fault, so the CPU already has to have the logic to
not lock for those.
The thing that made the particular illegal instructions so nasty seems to
be that the instruction is _almost_ correct, so the illegal instruction
fault is noticed much later than usual, and obviously after the lock logic
has already gotten primed.
Essentially, there are a few kinds of illegal instructions:
- real illegal instructions
- legal instructions that are turned illegal at lower privilege levels or
non-protected mode. For example, some instructions are illegal in real
mode ("lar" and some other protected-mode-segmentation specific stuff).
- legal instructions that have a LOCK prefix, but that cannot be locked.
- instructions that are too long (redundant prefix bytes)
- legal instructions with illegal addressing modes
In this case, it was the last one: a legal locked instruction with an
illegal addressing mode. I suspect that what happens is that normally the
#UD fault is raised by the instruction decode logic, but that the specific
instruction (cmpxchg8b) is kind of special. It doesn't have a "normal"
memory access pattern for an integer instruction, so it is probably
special-cased in the decoding, and passes the checks into microcode.
The microcode notices the illegal addressing mode, and raises the fault,
but by that time the instruction has already been "accepted" by the decode
unit, and the LOCK prefix has already taken effect.
And as this cannot happen on previous intel CPU's (that didn't have the
instruction), the #UD logic probably doesn't bother with clearing the LOCK
state, and nobody noticed the bug...
In contrast, when a page fault (or a segment limit overrun, or..) happens,
it's quite normal for a locked instruction to have made it past all the
other checks, so the page fault logic has always had to deal with the case
of a lock prefix fault, and it correctly clears it before actually
faulting.
The above is pure speculation, and may have nothing at all to do with
reality. It explains the fix (and the reason for the bug), but I don't
know a transistor from a diode, so take anything I say with a ton of salt.
But maybe people should start looking for other complex instructions that
are lockable.. (the only other complex instruction I can think of is the
bitmap instructions - bt, bts, btc, btr - but they don't actually have an
illegal addressing mode).
Linus