X86 virtualization

x86 virtualization is the use of hardware-assisted virtualization capabilities on an x86/x86-64 CPU.

In the late 1990s x86 virtualization was achieved by complex software techniques, necessary to compensate for the processor's lack of hardware-assisted virtualization capabilities while attaining reasonable performance. In 2005 and 2006, both Intel (VT-x) and AMD (AMD-V) introduced limited hardware virtualization support that allowed simpler virtualization software but offered very few speed benefits. Greater hardware support, which allowed substantial speed improvements, came with later processor models.

Software-based virtualization

The following discussion focuses only on virtualization of the x86 architecture protected mode.

In protected mode the operating system kernel runs in kernel space at the most privileged level (ring 0) that allows it to configure the MMU, manage physical memory, and directly control I/O peripherals, while applications run in user space at a lower privilege level (such as ring 3), where they are confined to their own virtual address spaces and must invoke system calls to request I/O operations or other privileged services from the kernel.

In software-based virtualization, a host OS has direct access to hardware while the guest operating systems have limited access to hardware, similarly to other user space application of the host OS. One approach used in x86 software-based virtualization to implement this mechanism is called ring deprivileging, which involves running the guest OS at a ring higher (lesser privileged) than 0, so that attempts to execute privileged instructions can be intercepted and handled by the hypervisor.

Three techniques made virtualization of protected mode possible:

Binary translation is used to rewrite certain ring 0 instructions in terms of ring 3 instructions, such as POPF, that would otherwise fail silently or behave differently when executed above ring 0, making the classic trap-and-emulate virtualization impossible. To improve performance, the translated basic blocks need to be cached in a coherent way that detects code patching (used in VxDs for instance), the reuse of pages by the guest OS, or even self-modifying code.

A number of key data structures used by a processor need to be shadowed. Because most operating systems use paged virtual memory, and granting the guest OS direct access to the MMU would mean loss of control by the virtualization manager, some of the work of the x86 MMU needs to be duplicated in software for the guest OS using a technique known as shadow page tables.