1 \input texinfo @c -*- texinfo -*-
3 @setfilename qemu-tech.info
6 @documentencoding UTF-8
8 @settitle QEMU Internals
15 * QEMU Internals: (qemu-tech). The QEMU Emulator Internals.
22 @center @titlefont{QEMU Internals}
44 * intro_x86_emulation:: x86 and x86-64 emulation
45 * intro_arm_emulation:: ARM emulation
46 * intro_mips_emulation:: MIPS emulation
47 * intro_ppc_emulation:: PowerPC emulation
48 * intro_sparc_emulation:: Sparc32 and Sparc64 emulation
49 * intro_xtensa_emulation:: Xtensa emulation
50 * intro_other_emulation:: Other CPU emulation
53 @node intro_x86_emulation
54 @section x86 and x86-64 emulation
56 QEMU x86 target features:
60 @item The virtual x86 CPU supports 16 bit and 32 bit addressing with segmentation.
61 LDT/GDT and IDT are emulated. VM86 mode is also supported to run
62 DOSEMU. There is some support for MMX/3DNow!, SSE, SSE2, SSE3, SSSE3,
63 and SSE4 as well as x86-64 SVM.
65 @item Support of host page sizes bigger than 4KB in user mode emulation.
67 @item QEMU can emulate itself on x86.
69 @item An extensive Linux x86 CPU test program is included @file{tests/test-i386}.
70 It can be used to test other x86 virtual CPUs.
74 Current QEMU limitations:
78 @item Limited x86-64 support.
80 @item IPC syscalls are missing.
82 @item The x86 segment limits and access rights are not tested at every
83 memory access (yet). Hopefully, very few OSes seem to rely on that for
88 @node intro_arm_emulation
89 @section ARM emulation
93 @item Full ARM 7 user emulation.
95 @item NWFPE FPU support included in user Linux emulation.
97 @item Can run most ARM Linux binaries.
101 @node intro_mips_emulation
102 @section MIPS emulation
106 @item The system emulation allows full MIPS32/MIPS64 Release 2 emulation,
107 including privileged instructions, FPU and MMU, in both little and big
110 @item The Linux userland emulation can run many 32 bit MIPS Linux binaries.
114 Current QEMU limitations:
118 @item Self-modifying code is not always handled correctly.
120 @item 64 bit userland emulation is not implemented.
122 @item The system emulation is not complete enough to run real firmware.
124 @item The watchpoint debug facility is not implemented.
128 @node intro_ppc_emulation
129 @section PowerPC emulation
133 @item Full PowerPC 32 bit emulation, including privileged instructions,
136 @item Can run most PowerPC Linux binaries.
140 @node intro_sparc_emulation
141 @section Sparc32 and Sparc64 emulation
145 @item Full SPARC V8 emulation, including privileged
146 instructions, FPU and MMU. SPARC V9 emulation includes most privileged
147 and VIS instructions, FPU and I/D MMU. Alignment is fully enforced.
149 @item Can run most 32-bit SPARC Linux binaries, SPARC32PLUS Linux binaries and
150 some 64-bit SPARC Linux binaries.
154 Current QEMU limitations:
158 @item IPC syscalls are missing.
160 @item Floating point exception support is buggy.
162 @item Atomic instructions are not correctly implemented.
164 @item There are still some problems with Sparc64 emulators.
168 @node intro_xtensa_emulation
169 @section Xtensa emulation
173 @item Core Xtensa ISA emulation, including most options: code density,
174 loop, extended L32R, 16- and 32-bit multiplication, 32-bit division,
175 MAC16, miscellaneous operations, boolean, FP coprocessor, coprocessor
176 context, debug, multiprocessor synchronization,
177 conditional store, exceptions, relocatable vectors, unaligned exception,
178 interrupts (including high priority and timer), hardware alignment,
179 region protection, region translation, MMU, windowed registers, thread
180 pointer, processor ID.
182 @item Not implemented options: data/instruction cache (including cache
183 prefetch and locking), XLMI, processor interface. Also options not
184 covered by the core ISA (e.g. FLIX, wide branches) are not implemented.
186 @item Can run most Xtensa Linux binaries.
188 @item New core configuration that requires no additional instructions
189 may be created from overlay with minimal amount of hand-written code.
193 @node intro_other_emulation
194 @section Other CPU emulation
196 In addition to the above, QEMU supports emulation of other CPUs with
197 varying levels of success. These are:
212 @chapter QEMU Internals
215 * QEMU compared to other emulators::
216 * Portable dynamic translation::
217 * Condition code optimisations::
218 * CPU state optimisations::
219 * Translation cache::
220 * Direct block chaining::
221 * Self-modifying code and translated code invalidation::
222 * Exception support::
225 * Hardware interrupts::
226 * User emulation specific details::
230 @node QEMU compared to other emulators
231 @section QEMU compared to other emulators
233 Like bochs [1], QEMU emulates an x86 CPU. But QEMU is much faster than
234 bochs as it uses dynamic compilation. Bochs is closely tied to x86 PC
235 emulation while QEMU can emulate several processors.
237 Like Valgrind [2], QEMU does user space emulation and dynamic
238 translation. Valgrind is mainly a memory debugger while QEMU has no
239 support for it (QEMU could be used to detect out of bound memory
240 accesses as Valgrind, but it has no support to track uninitialised data
241 as Valgrind does). The Valgrind dynamic translator generates better code
242 than QEMU (in particular it does register allocation) but it is closely
243 tied to an x86 host and target and has no support for precise exceptions
244 and system emulation.
246 EM86 [3] is the closest project to user space QEMU (and QEMU still uses
247 some of its code, in particular the ELF file loader). EM86 was limited
248 to an alpha host and used a proprietary and slow interpreter (the
249 interpreter part of the FX!32 Digital Win32 code translator [4]).
251 TWIN from Willows Software was a Windows API emulator like Wine. It is less
252 accurate than Wine but includes a protected mode x86 interpreter to launch
253 x86 Windows executables. Such an approach has greater potential because most
254 of the Windows API is executed natively but it is far more difficult to
255 develop because all the data structures and function parameters exchanged
256 between the API and the x86 code must be converted.
258 User mode Linux [5] was the only solution before QEMU to launch a
259 Linux kernel as a process while not needing any host kernel
260 patches. However, user mode Linux requires heavy kernel patches while
261 QEMU accepts unpatched Linux kernels. The price to pay is that QEMU is
264 The Plex86 [6] PC virtualizer is done in the same spirit as the now
265 obsolete qemu-fast system emulator. It requires a patched Linux kernel
266 to work (you cannot launch the same kernel on your PC), but the
267 patches are really small. As it is a PC virtualizer (no emulation is
268 done except for some privileged instructions), it has the potential of
269 being faster than QEMU. The downside is that a complicated (and
270 potentially unsafe) host kernel patch is needed.
272 The commercial PC Virtualizers (VMWare [7], VirtualPC [8]) are faster
273 than QEMU (without virtualization), but they all need specific, proprietary
274 and potentially unsafe host drivers. Moreover, they are unable to
275 provide cycle exact simulation as an emulator can.
277 VirtualBox [9], Xen [10] and KVM [11] are based on QEMU. QEMU-SystemC
278 [12] uses QEMU to simulate a system where some hardware devices are
279 developed in SystemC.
281 @node Portable dynamic translation
282 @section Portable dynamic translation
284 QEMU is a dynamic translator. When it first encounters a piece of code,
285 it converts it to the host instruction set. Usually dynamic translators
286 are very complicated and highly CPU dependent. QEMU uses some tricks
287 which make it relatively easily portable and simple while achieving good
290 QEMU's dynamic translation backend is called TCG, for "Tiny Code
291 Generator". For more information, please take a look at @code{tcg/README}.
293 @node Condition code optimisations
294 @section Condition code optimisations
296 Lazy evaluation of CPU condition codes (@code{EFLAGS} register on x86)
297 is important for CPUs where every instruction sets the condition
298 codes. It tends to be less important on conventional RISC systems
299 where condition codes are only updated when explicitly requested. On
300 Sparc64, costly update of both 32 and 64 bit condition codes can be
301 avoided with lazy evaluation.
303 Instead of computing the condition codes after each x86 instruction,
304 QEMU just stores one operand (called @code{CC_SRC}), the result
305 (called @code{CC_DST}) and the type of operation (called
306 @code{CC_OP}). When the condition codes are needed, the condition
307 codes can be calculated using this information. In addition, an
308 optimized calculation can be performed for some instruction types like
309 conditional branches.
311 @code{CC_OP} is almost never explicitly set in the generated code
312 because it is known at translation time.
314 The lazy condition code evaluation is used on x86, m68k, cris and
315 Sparc. ARM uses a simplified variant for the N and Z flags.
317 @node CPU state optimisations
318 @section CPU state optimisations
320 The target CPUs have many internal states which change the way it
321 evaluates instructions. In order to achieve a good speed, the
322 translation phase considers that some state information of the virtual
323 CPU cannot change in it. The state is recorded in the Translation
324 Block (TB). If the state changes (e.g. privilege level), a new TB will
325 be generated and the previous TB won't be used anymore until the state
326 matches the state recorded in the previous TB. For example, if the SS,
327 DS and ES segments have a zero base, then the translator does not even
328 generate an addition for the segment base.
330 [The FPU stack pointer register is not handled that way yet].
332 @node Translation cache
333 @section Translation cache
335 A 32 MByte cache holds the most recently used translations. For
336 simplicity, it is completely flushed when it is full. A translation unit
337 contains just a single basic block (a block of x86 instructions
338 terminated by a jump or by a virtual CPU state change which the
339 translator cannot deduce statically).
341 @node Direct block chaining
342 @section Direct block chaining
344 After each translated basic block is executed, QEMU uses the simulated
345 Program Counter (PC) and other cpu state information (such as the CS
346 segment base value) to find the next basic block.
348 In order to accelerate the most common cases where the new simulated PC
349 is known, QEMU can patch a basic block so that it jumps directly to the
352 The most portable code uses an indirect jump. An indirect jump makes
353 it easier to make the jump target modification atomic. On some host
354 architectures (such as x86 or PowerPC), the @code{JUMP} opcode is
355 directly patched so that the block chaining has no overhead.
357 @node Self-modifying code and translated code invalidation
358 @section Self-modifying code and translated code invalidation
360 Self-modifying code is a special challenge in x86 emulation because no
361 instruction cache invalidation is signaled by the application when code
364 When translated code is generated for a basic block, the corresponding
365 host page is write protected if it is not already read-only. Then, if
366 a write access is done to the page, Linux raises a SEGV signal. QEMU
367 then invalidates all the translated code in the page and enables write
368 accesses to the page.
370 Correct translated code invalidation is done efficiently by maintaining
371 a linked list of every translated block contained in a given page. Other
372 linked lists are also maintained to undo direct block chaining.
374 On RISC targets, correctly written software uses memory barriers and
375 cache flushes, so some of the protection above would not be
376 necessary. However, QEMU still requires that the generated code always
377 matches the target instructions in memory in order to handle
378 exceptions correctly.
380 @node Exception support
381 @section Exception support
383 longjmp() is used when an exception such as division by zero is
386 The host SIGSEGV and SIGBUS signal handlers are used to get invalid
387 memory accesses. The simulated program counter is found by
388 retranslating the corresponding basic block and by looking where the
389 host program counter was at the exception point.
391 The virtual CPU cannot retrieve the exact @code{EFLAGS} register because
392 in some cases it is not computed because of condition code
393 optimisations. It is not a big concern because the emulated code can
394 still be restarted in any cases.
397 @section MMU emulation
399 For system emulation QEMU supports a soft MMU. In that mode, the MMU
400 virtual to physical address translation is done at every memory
401 access. QEMU uses an address translation cache to speed up the
404 In order to avoid flushing the translated code each time the MMU
405 mappings change, QEMU uses a physically indexed translation cache. It
406 means that each basic block is indexed with its physical address.
408 When MMU mappings change, only the chaining of the basic blocks is
409 reset (i.e. a basic block can no longer jump directly to another one).
411 @node Device emulation
412 @section Device emulation
414 Systems emulated by QEMU are organized by boards. At initialization
415 phase, each board instantiates a number of CPUs, devices, RAM and
416 ROM. Each device in turn can assign I/O ports or memory areas (for
417 MMIO) to its handlers. When the emulation starts, an access to the
418 ports or MMIO memory areas assigned to the device causes the
419 corresponding handler to be called.
421 RAM and ROM are handled more optimally, only the offset to the host
422 memory needs to be added to the guest address.
424 The video RAM of VGA and other display cards is special: it can be
425 read or written directly like RAM, but write accesses cause the memory
426 to be marked with VGA_DIRTY flag as well.
428 QEMU supports some device classes like serial and parallel ports, USB,
429 drives and network devices, by providing APIs for easier connection to
430 the generic, higher level implementations. The API hides the
431 implementation details from the devices, like native device use or
432 advanced block device formats like QCOW.
434 Usually the devices implement a reset method and register support for
435 saving and loading of the device state. The devices can also use
436 timers, especially together with the use of bottom halves (BHs).
438 @node Hardware interrupts
439 @section Hardware interrupts
441 In order to be faster, QEMU does not check at every basic block if a
442 hardware interrupt is pending. Instead, the user must asynchronously
443 call a specific function to tell that an interrupt is pending. This
444 function resets the chaining of the currently executing basic
445 block. It ensures that the execution will return soon in the main loop
446 of the CPU emulator. Then the main loop can test if the interrupt is
447 pending and handle it.
449 @node User emulation specific details
450 @section User emulation specific details
452 @subsection Linux system call translation
454 QEMU includes a generic system call translator for Linux. It means that
455 the parameters of the system calls can be converted to fix the
456 endianness and 32/64 bit issues. The IOCTLs are converted with a generic
457 type description system (see @file{ioctls.h} and @file{thunk.c}).
459 QEMU supports host CPUs which have pages bigger than 4KB. It records all
460 the mappings the process does and try to emulated the @code{mmap()}
461 system calls in cases where the host @code{mmap()} call would fail
462 because of bad page alignment.
464 @subsection Linux signals
466 Normal and real-time signals are queued along with their information
467 (@code{siginfo_t}) as it is done in the Linux kernel. Then an interrupt
468 request is done to the virtual CPU. When it is interrupted, one queued
469 signal is handled by generating a stack frame in the virtual CPU as the
470 Linux kernel does. The @code{sigreturn()} system call is emulated to return
471 from the virtual signal handler.
473 Some signals (such as SIGALRM) directly come from the host. Other
474 signals are synthesized from the virtual CPU exceptions such as SIGFPE
475 when a division by zero is done (see @code{main.c:cpu_loop()}).
477 The blocked signal mask is still handled by the host Linux kernel so
478 that most signal system calls can be redirected directly to the host
479 Linux kernel. Only the @code{sigaction()} and @code{sigreturn()} system
480 calls need to be fully emulated (see @file{signal.c}).
482 @subsection clone() system call and threads
484 The Linux clone() system call is usually used to create a thread. QEMU
485 uses the host clone() system call so that real host threads are created
486 for each emulated thread. One virtual CPU instance is created for each
489 The virtual x86 CPU atomic operations are emulated with a global lock so
490 that their semantic is preserved.
492 Note that currently there are still some locking issues in QEMU. In
493 particular, the translated cache flush is not protected yet against
496 @subsection Self-virtualization
498 QEMU was conceived so that ultimately it can emulate itself. Although
499 it is not very useful, it is an important test to show the power of the
502 Achieving self-virtualization is not easy because there may be address
503 space conflicts. QEMU user emulators solve this problem by being an
504 executable ELF shared object as the ld-linux.so ELF interpreter. That
505 way, it can be relocated at load time.
508 @section Bibliography
513 @url{http://bochs.sourceforge.net/}, the Bochs IA-32 Emulator Project,
514 by Kevin Lawton et al.
517 @url{http://www.valgrind.org/}, Valgrind, an open-source memory debugger
521 @url{http://ftp.dreamtime.org/pub/linux/Linux-Alpha/em86/v0.2/docs/em86.html},
522 the EM86 x86 emulator on Alpha-Linux.
525 @url{http://www.usenix.org/publications/library/proceedings/usenix-nt97/@/full_papers/chernoff/chernoff.pdf},
526 DIGITAL FX!32: Running 32-Bit x86 Applications on Alpha NT, by Anton
527 Chernoff and Ray Hookway.
530 @url{http://user-mode-linux.sourceforge.net/},
531 The User-mode Linux Kernel.
534 @url{http://www.plex86.org/},
535 The new Plex86 project.
538 @url{http://www.vmware.com/},
539 The VMWare PC virtualizer.
542 @url{https://www.microsoft.com/download/details.aspx?id=3702},
543 The VirtualPC PC virtualizer.
546 @url{http://virtualbox.org/},
547 The VirtualBox PC virtualizer.
550 @url{http://www.xen.org/},
554 @url{http://www.linux-kvm.org/},
555 Kernel Based Virtual Machine (KVM).
558 @url{http://www.greensocs.com/projects/QEMUSystemC},
559 QEMU-SystemC, a hardware co-simulator.
563 @node Regression Tests
564 @chapter Regression Tests
566 In the directory @file{tests/}, various interesting testing programs
567 are available. They are used for regression testing.
575 @section @file{test-i386}
577 This program executes most of the 16 bit and 32 bit x86 instructions and
578 generates a text output. It can be compared with the output obtained with
579 a real CPU or another emulator. The target @code{make test} runs this
580 program and a @code{diff} on the generated output.
582 The Linux system call @code{modify_ldt()} is used to create x86 selectors
583 to test some 16 bit addressing and 32 bit with segmentation cases.
585 The Linux system call @code{vm86()} is used to test vm86 emulation.
587 Various exceptions are raised to test most of the x86 user space
591 @section @file{linux-test}
593 This program tests various Linux system calls. It is used to verify
594 that the system call parameters are correctly converted between target