1. Design

   The goal of EMUTRAMP is to emulate certain instruction sequences that are
   known to be generated at runtime in otherwise non-executable memory regions
   and hence would cause task termination when the non-executable feature is
   enforced (by PAGEEXEC or SEGMEXEC).

   While there are many sources of runtime generated code, PaX emulates only
   two of them: the kernel generated signal return stubs and the gcc nested
   function trampolines. We chose to emulate them and only them because they
   are the most common and still the shortest and best defined sequences. The
   other reason is that runtime code generation is by its nature incompatible
   with PaX's PAGEEXEC/SEGMEXEC and MPROTECT features, therefore the real
   solution is not in emulation but by designing a kernel API for runtime
   code generation and modifying userland to make use of it.

   Whenever a task attempts to execute code from a non-executable page, the
   PAGEEXEC or SEGMEXEC logic raises a page fault and therefore the page fault
   handler is the place where EMUTRAMP can carry out its job. Since the code
   to be emulated is architecture specific, EMUTRAMP is architecture dependent
   as well (for now it exists for IA-32 only).

   The kernel signal return stubs are generated by the kernel itself before
   a signal is dispatched to the userland signal handler. There are two stubs
   for returning from normal (old style) and real-time signal (new style)
   handlers, respectively. By default the kernel puts all information for the
   userland signal handler on the userland stack therefore the signal return
   stubs end up there as well, hence the need for emulating them. It is worth
   noting however that userland can tell the kernel to rely on userland
   supplied signal return code (the SA_RESTORER flag) and glibc 2.1+ actually
   makes use of this feature and therefore applications linking against it
   do not require this emulation.

   The gcc nested function trampolines implement a gcc specific C extension
   where one can define a function within another one (this is the nesting).
   Since the inner function has access to all the local variables of the
   outer one, the compiler has to pass an extra parameter (pointer to the
   outer function's stack frame) to such a nested function. This becomes a
   challenge when one takes the address of the nested function and calls it
   later via a function pointer (while the outer function's stack frame still
   exists of course). In this case the address of the function body does not
   fully identify the function since the inner function also needs access to
   the outer function's stack frame. The gcc solution is that whenever the
   address of a nested function is taken, gcc generates a small stub on the
   stack (in the stack frame of the outer function) and uses the stub's
   address as the function pointer. The stub then passes the current frame's
   address to the actual inner function body. While this is pretty much the
   only generic solution for implementing function pointers to nested
   functions, there is a better one for a special (but likely the most often
   encountered) situation: if the outer function is not called recursively
   then we know that there's at most one instance of the stack frame of the
   outer function and therefore its address can be stored in a normal (per
   thread in case of a multithreaded program) static variable and referenced
   from within the nested function's body - no need to generate the trampoline
   stub at runtime.


2. Implementation

   EMUTRAMP is implemented by the pax_handle_fetch_fault() function in
   arch/i386/mm/fault.c. This function is called during page fault processing
   when an instruction fetch attempt from a non-executable page is detected.
   Each signal return stub and nested function trampoline is recognized by
   checking for their instruction sequence signature and ensuring that some
   additional conditions (e.g., register contents) are met as well. Emulation
   itself is very simple, we only modify the userland register context which
   was saved by the kernel and will be restored when it returns to userland.

   The signal return stub emulation requires a bit of extra handling because
   it wants to invoke a system call at the end and the task has already entered
   the kernel (it is processing a page fault). Whenever EMUTRAMP is enabled,
   the low-level page fault handler stub in arch/i386/kernel/entry.S can be
   told to simulate a system call entry into the kernel by simply restoring
   the kernel stack to the point before the page fault processing began then
   jumping to the system call processing stub.