I can see this being a useful compilation option for kernels meant for server
machines that are expected to run a fixed set of programs. Things like
routers and ppp servers. I don't think it should ever be used on a machine
where users log in and expect to run arbitrary programs, but not all machines
are like that.
Actually, the security mantra is to default to denying privileges, which in
this case would mean only granting execution privileges on the code segment
and library mappings. This would be a little extra work to make sure dlopen
still worked and such, and i could be missing something, but i think this
should work.
If this increases the number of high performance simulators (Executor comes to
mind) that fail then that's ok. In fact that sounds like a good, thing. If it
has more noticeable effects then people won't be tempted to turn it on for
machines where it is not appropriate. Perhaps even a printing a warning at
boot time indicating the kernel is a server kernel and not appropriate for
user machines would help.
I had a few comments though:
Is it possible to exploit the trampoline detector by first causing a spurious
RET then subsequently trying the exploit? spurious returns in many programs,
especially in robust server daemons are likely to be taken to indicate an
error condition and be handled reasonably.
Would it be possible to implement this in ld.so instead? Then you could be
more flexible and support things like turning the feature off and on at
runtime or supporting a flag in the executable to indicate to ld.so that the
executable really doesn't need the extra executable privileges.