The XCP-ng Developer's Handbook

The simplest way to allocate resources (CPU, RAM, storage, network bandwidth and more generally any device) to a service is to dedicate a physical machine to it. Although this has the appeal of simplicity (the service has exclusive access to all the resources of the machine), it comes with an intrinsic inefficiency that we will now explain.

A machine must be capable of handling peak load. As brief as the peak may be, the machine must be sized to handle it, which means that for the rest of the time, a fraction of its CPU or memory remains unused. This is not a failure of planning: it is a structural property of dedicating a machine to a single service.

Beyond this waste of resources, a dedicated machine carries operational costs that are easy to underestimate. On the hardware side, components fail and must be repaired or replaced, and when the service grows and the machine can no longer keep up, capacity must be added, for instance by adding RAM, replacing disks with larger ones, or adding network interfaces. On the software side, the operating system and all installed software must be kept up to date. And when the machine reaches the end of its life, it must be decommissioned: any remaining data must be migrated, the machine must be wiped, and then either recycled or donated.

But an organization running tens or hundreds of services on dedicated machines would face these costs for each machine in the fleet, multiplying both the inefficiency and the operational burden.

And so does the environmental cost. Indeed, each step of each machine's life cycle does have an environmental impact. Manufacturing consumes raw materials, including rare earth elements whose extraction carries its own increasing environmental cost. Running consumes electricity, and so do the cooling systems that keep datacenters from overheating, which in many cases also consume significant quantities of water. Finally, recycling itself consumes energy and cannot even be complete: some components must be discarded, representing an irreversible loss of the physical resources that went into making them. At the scale of tens or hundreds of machines, these costs add up to an environmental impact that is no longer negligible.

As established in the previous section, running one service per machine is not viable. The only practical approach is thus to run multiple services on the same machine, which implies sharing its resources. This in turn translates into two constraints.

The first is the equitable sharing of resources. No service should be able to lessen the others in terms of access to the machine's resources.

The second is the isolation of data and privileges. A service should not be able to read the data of another service running on the same machine, nor benefit from its privileges.

From a user's perspective, a service is either available or it is not. When it is not, whatever the reason, the user cannot do what they came to do. Availability is therefore not a nice-to-have: for any service that people depend on, it is a hard requirement.

Yet a service running on a single machine cannot fulfill this requirement: any failure, whether a hardware fault, a software crash, or a necessary maintenance operation, interrupts it, which requires redundancy: running multiple instances of the service on multiple machines, so that if one machine fails, the others take over without interruption.

Redundancy, however, introduces a new constraint. Running multiple instances of the same service on different machines only works if those instances are consistent with one another. A service deployed differently on each machine, with different configurations or different versions of its dependencies, will behave unpredictably. The ability to deploy a service reliably and consistently across multiple machines is therefore a prerequisite for redundancy.

This need extends beyond production. Development and testing environments must be close enough to the production environment for tests to be meaningful, yet distinct from it to avoid any risk of disrupting the service. This again requires the ability to deploy the service consistently in multiple contexts.

Finally, keeping a service available over time raises the question of updates. Any service in production will inevitably need to be updated, whether to add new features, fix bugs, or address security vulnerabilities. But updating a service, whether the service itself or its dependencies, risks introducing regressions or, if done carelessly, causing the very interruption that redundancy was designed to prevent. How to update a service running on multiple machines, progressively and without interruption, is a problem that takes us directly to the question of orchestration.

Most of what has been discussed so far has concerned a single service. In practice, however, an organization runs tens or hundreds of services, each with its own life cycle: initial deployment, successive updates, monitoring, restarts when it fails, and eventual decommissioning. Managing these life cycles manually is already demanding for a handful of services, and even then it is error-prone. At the scale of tens or hundreds of services, each running on multiple machines, it is simply not an option.

The situation is further complicated by the fact that services often depend on one another. A service that relies on another cannot be deployed or updated independently of it. Coordinating deployments and updates across interdependent services is exponentially more complex than managing each service in isolation.

The same applies to development. Developing and testing a service requires an environment that is close enough to production for the results to be meaningful, which adds yet another context in which the full set of dependencies must be deployed coherently.

Orchestration is the name given to this problem: how to coordinate the development, testing, deployment, monitoring, and life cycle of a large number of services running on a large number of machines, reliably and without constant manual intervention.

As has been seen, running one service per machine is not an option. Beyond its economic and environmental costs, it is simply not viable in practice. The only remaining option is thus to run multiple services on the same machine, which raises the need for both equitable access to its resources and isolation between services (no service should be able to access the data or privileges of another).

Moreover, keeping services available requires redundancy, which in turn requires the ability to deploy services consistently across multiple machines and in multiple environments. And as the number of (often interdependent) services grows, so does the need to coordinate their deployment, monitoring, and life cycles without constant manual intervention.

Throughout this chapter, we have identified a set of constraints that arise when running services at scale. As announced in the introduction, it is against this set of constraints that we will situate each of the approaches presented in the next chapter. To help the reader keep them in mind while reading, here they are in synthesized form:

1. Run multiple services on the same machine, with equitable access to its resources: CPU, memory, storage, network bandwidth, and devices.
2. Prevent any service from accessing the data or privileges of another.
3. Deploy services across multiple machines so that they remain available despite hardware failures and can be updated consistently and without interruption.
4. Keep development, testing, and production environments consistent with one another while keeping them distinct.
5. Coordinate the deployment, monitoring, and life cycles of many often interdependent services without constant manual intervention.

We have not yet asked how a service is installed on a machine. The natural starting point is the system package manager: apt on Debian-based systems, dnf on Red Hat-based systems, and their equivalents elsewhere. A package manager not only installs a package but also, before doing so, identifies and installs a list of packages it depends on. For each of these dependencies, there may be an additional constraint on the version: a package may require another at version 2.0 or above, or between versions 1.4 and 2.0, or exactly at version 3.1. The package manager's job is to find a set of package versions that satisfies all these constraints simultaneously and to install them.

Such a system of constraints typically has many solutions. The package manager picks one, and the result is a coherent set of packages whose components are expected to work together. From a user's perspective, this is a desirable property: the system stays current as packages are updated and constraints evolve, the package manager may find a new consistent set, and coherence is maintained at each step.

For a developer working across multiple projects, though, this model shows its limits. Two projects may require incompatible versions of the same library: no single version can satisfy the constraints of both. Since the system package manager allows only one version of a given library to be present at a time, no solution exists that satisfies both projects simultaneously. There is also a second pressure: language ecosystems tend to move faster than distributions, with new package versions published directly by their authors rather than going through the slower process of distribution packaging. Both factors contributed to the emergence of language-specific tools that address the problem within their own scope.

These tools emerged pragmatically, within specific communities, to solve immediate needs. Their language-specificity is both a strength and a limit: a tool designed for a given ecosystem can integrate intimately with its conventions, package metadata, and build tooling, in a way that a language-agnostic tool could not. But it can only ever solve the problem within that ecosystem.

The approach these tools share is straightforward: a dedicated directory that holds its own copy of the packages installed for a given project, combined with a set of environment variable modifications that redirect the language's tools toward that directory instead of the global one. A program launched with the appropriate environment will find its dependencies in that directory rather than in the system-wide store. The packages installed in one environment are invisible to another, and conflicts between projects become impossible in principle, within the bounds of what the language's tooling manages.

Like system package managers, these tools express dependencies as constraints on version numbers and must find a solution that satisfies them all. Unlike system package managers, many of them record that solution in a lock file: a precise, complete record of the exact version of every dependency as resolved at a given point in time. A lock file transforms a set of constraints, which may have many solutions, into a single fixed solution that can be reproduced exactly across machines and over time. Tools like poetry, stack, and uv generate such lock files as a matter of course.

These tools differ in what they expose to the user. An environment can be local to a single project, or shared across several; some tools offer one mode, some the other, and some both. npm in JavaScript and stack in Haskell treat the environment as a property of the project, as does Cargo in Rust. nvm in JavaScript and rbenv in Ruby offer named environments that can span multiple projects. venv and virtualenv in Python leave the choice to the user. opam switch in OCaml also supports both modes: switches can be global and shared across projects, or local to a specific project directory.

The conda Python tool occupies a distinct position in this landscape. Unlike the tools described above, it can manage dependencies written in other languages, including C libraries that Python packages may depend on. This makes it more general than the other tools described here, but it remains an ecosystem-specific solution. As we will see in the next section, Nix and Guix generalize this further in a principled way, handling dependencies of any kind, for any project, within a single unified framework.

Tools such as Snap, Flatpak, and AppImage take a different approach: rather than isolating dependencies in a separate directory, they bundle them directly inside the package. Unlike the ecosystem-specific tools described above, they are not tied to any particular language. This eliminates conflicts with system libraries for the bundled application, but these tools are designed primarily for deploying desktop applications, not services, and they fall outside the scope of what we examine in this chapter.

In terms of the constraints identified in the previous chapter, these ecosystem-specific tools address a narrow slice. The first constraint asks that multiple services be able to run on the same machine with equitable access to its resources: these tools do not control resource usage in any way. The second constraint asks that no service be able to access the data or privileges of another. These tools offer little here: two services running in different environments share the same operating system and the same file system outside their respective directories. More fundamentally, the isolation they provide rests entirely on a cooperative assumption: every tool in the ecosystem must agree to respect the environment variables that redirect package lookups. A package that hardcodes a path to a system library, or a build script that bypasses the package manager entirely, will silently break out of the isolation with no external mechanism to prevent it. The boundary exists only because every participant agrees to observe it. As we will see, this assumption persists in different forms through the approaches that follow, and addressing it fully requires mechanisms of a different kind entirely.

The third constraint concerns deployment across multiple machines and availability despite hardware failures. These tools are silent on availability and fault tolerance, though the use of lock files does facilitate consistent deployment across machines, which is a partial contribution to this constraint. The fifth constraint, concerning the coordination of many interdependent services, is likewise not addressed.

The fourth constraint is where these tools have something genuine to offer: the requirement that development, testing, and production environments remain consistent with one another. By isolating dependencies on a per-project basis and, where lock files are used, fixing an exact solution to the dependency constraints, these tools make it possible to reproduce a known package configuration across machines and over time. This reproducibility, however, extends only to the packages the tool manages: dependencies at the system level remain outside its reach. That boundary is precisely what Nix and Guix set out to eliminate.

As has been seen, system package managers have been designed to maintain a coherent set of packages across installations and updates: they have not been designed to address reproducibility concerns. Neither do they permit the coexistence of two versions of the same package. The tools examined in the previous section address these issues only partially: they are mostly confined to a single ecosystem, which means they are not general enough to cover all dependencies in a uniform way. And even for the dependencies they do cover, nothing prevents a package from relying on anything other than its declared dependencies.

The Nix package manager was designed to overcome these limitations. Its central idea is that each package is identified by a hash derived from all of its inputs: its source code, build instructions, and build-time and run-time dependencies. Readers familiar with Git will recognize the structure: just as a commit hash uniquely identifies a point in a repository's history by capturing both the state of the tree and its ancestry, a Nix package hash uniquely identifies a build result by capturing everything that went into producing it. Nix installs each package in its own directory, as ecosystem-specific tools do; what distinguishes it is that the directory name is derived from the package's hash, so any difference in inputs, however slight, results in a distinct package in a distinct location.

This more complete and rigorous management of dependencies does not only allow packages to be distinguished more precisely from one another: because each package is fully determined by its inputs, the build process becomes much more deterministic, producing the same output from the same inputs on any machine. This property is what we call reproducibility. Since this is a primary concern for Nix, it ensures that the package's build process can only access what has been explicitly declared as an input, so that only its declared inputs can influence the result. This isolation is enforced by mechanisms provided by the operating system, which we will examine later in this chapter. It is worth pausing on this point. In the previous section, isolation relied on a cooperative assumption: every package had to agree to respect the conventions that made isolation possible. Here, for the first time, we encounter a constraint that is enforced mechanically rather than observed by convention. A package's build process cannot reach outside its declared inputs, not because it is expected to refrain from doing so, but because it is made impossible. This isolation applies only during the package's build stage: once a package has been built, the resulting binaries run without any restriction imposed by Nix.

This approach, too, has its limits: not all dependencies can be expressed as Nix inputs. The kernel is the most significant example: it is provided by the host system and lies outside what Nix can describe or control. A build that depends on kernel-specific values, like the output of the uname command, will therefore produce different results on machines with different kernel versions, even if all its other inputs are identical. NixOS addresses this by extending Nix's declarative approach to the configuration of an entire system, including its kernel: it thus becomes possible to express all aspects of a machine's configuration as Nix inputs, and to reproduce it exactly on another machine. Guix and Guix System are the GNU project's equivalents of Nix and NixOS respectively. Where Nix uses a dedicated language designed specifically for package description, Guix uses Guile, a general-purpose Scheme implementation: package descriptions are Scheme values that can be computed, which means the full range of Scheme's factorisation and reuse mechanisms are available to facilitate the development and maintenance of package descriptions and build recipes.

Against the constraints identified in the previous chapter, Nix and Guix have little to say about the first: they exercise no control over how services share CPU time, memory, or any other resource.

The second constraint asks that no service be able to access the data or privileges of another. The cooperative assumption is not eliminated here but reduced in scope. At the package build stage, isolation is enforced: a package cannot silently depend on anything other than its declared inputs. Once the package has been built, though, the resulting binaries run without any restriction imposed by Nix: nothing prevents them from reading files outside their environment, opening network connections, or interacting with other running programs. The activation of a Nix environment, like that of an ecosystem-specific tool, relies on environment variable adjustments that any program is free to ignore. The boundary that Nix draws is precise and reliable at the package build stage, but once the resulting binaries are running, the cooperative assumption remains.

The third constraint concerns deployment across multiple machines and continued availability despite hardware failures. NixOS and Guix System address the first part: because a system configuration is fully declared, it can be reproduced exactly on another machine, which makes consistent deployment easier. The second part, availability despite hardware failures, however remains outside their scope.

The fourth constraint is where Nix and Guix offer their most substantial contribution. Because every dependency is a declared input with a fixed identity, the reproducibility they provide extends to the entire user-space environment. NixOS and Guix System extend this further by bringing the kernel configuration within the scope of what is declared, reducing though not eliminating the remaining sources of variance: even with a fully declared system configuration, factors such as differences in underlying hardware or non-deterministic behaviour in the kernel itself remain outside the reach of any declarative tool.

The fifth constraint, concerning the coordination of many interdependent services, is not addressed by these tools either.

Operating systems have long provided mechanisms to address the first two constraints identified in the previous chapter, those that arise when sharing a machine between multiple services. The first is ensuring equitable access to its resources. The second is the need to prevent any service from accessing the data or privileges of another.

In response to the first constraint, the main mechanism provided by operating systems is scheduling, which governs equitable access to the CPU. Equitable access to other resources such as memory, storage, and network bandwidth is not addressed at the level of operating systems. The following section will introduce a more complete mechanism: containers.

In response to the second constraint, operating systems provide two distinct mechanisms. The first is memory isolation, which prevents a service from reading or writing the memory of another. The second is access control through users, groups, and permissions, which governs access to files and devices. Access to network resources is not addressed at this level either. Here again, it is containers that will fill in the gaps.

This section does not follow the evaluative structure used everywhere else in this chapter: rather than presenting a single approach and then situating it against all five constraints, it examines a set of related mechanisms that together constitute the earliest systematic response to constraints 1 and 2, and that also provide the conceptual foundations for the approaches examined in the following sections. It is also the first time in this book where we move beyond pure software: some of the mechanisms we examine here (scheduling and memory isolation) rely on features provided by the processor itself.

We now introduce containers and evaluate them against the five constraints identified in the previous chapter. Containers rely on three mechanisms (namespaces, cgroups, and images) that we describe in turn, before explaining how containers are built on top of them.

Namespaces. The previous section introduced chroot, which confines a process to a subtree of the filesystem. A process confined in such a way cannot name files outside that subtree, and therefore cannot access them. However, chroot only isolates the filesystem. Other aspects of a process' execution environment, such as process identifiers, network interfaces, and users, remain shared with the rest of the system. A process confined by chroot can still see other processes and send them signals, access any network interface of the host, and share its view of users and groups. Namespaces generalize the principle of chroot to many aspects of a process' execution environment.

Here are a few examples of namespaces available on Linux. A mount namespace gives a process its own table of mounted filesystems, independently of the host's. (In particular, a process can mount any filesystem as its root, which means that chroot can be seen as a special case of what a mount namespace provides.) A PID namespace gives a process its own set of process identifiers: processes outside the namespace are invisible to it, and even the identifiers it sees bear no relation to those used on the host. A network namespace gives a process its own set of network interfaces and routing tables. In each case, the isolation is structural rather than based on explicit access checks: the entities a process cannot access are simply absent from its view of the system. This is the same principle we observed with virtual memory, where a process cannot reach the memory of another simply because that memory does not appear in its address space.

Cgroups. Namespaces control what a process can see. However, they provide no mechanism to express or enforce limits on what it can consume. Except CPU time which is covered by scheduling, nothing prevents a process running in its own set of namespaces from exhausting most of the resources available to the host, starving other processes of them. This is what cgroups, short for control groups, address. They make it possible to limit and account for the resource usage of a process or a group of processes: CPU time (complementing the scheduling discussed earlier with precise quotas and usage accounting), memory, network bandwidth, and others. Where namespaces provide isolation, cgroups provide containment. Like scheduling and virtual memory, both namespaces and cgroups are implemented as kernel mechanisms; unlike them, though, they require no dedicated hardware support.

Images. Namespaces and cgroups together define the boundaries of a process' execution environment. It still remains to be specified what will run in that execution environment. Just as a program is an executable that can be run by an operating system, an image is an executable that can be run inside an execution environment. Such an image consists of both a read-only filesystem that embeds the service to be run and its dependencies, and an entry point: the program that will actually be executed when the image is run. Because an image is self-contained, it can be run on any host, independently of what is installed on it.

Containers. Launching an image involves creating a mount namespace and mounting its filesystem as the root of that namespace, creating additional namespaces to isolate various aspects of its execution environment, such as its view of processes, network interfaces, and users, and creating cgroups to bound its resource consumption. Its entry point is then executed: it runs as a process in that confined environment. Just as running a program yields a process, running an image yields a container. This is not merely an analogy: at the system level, running an image amounts to running its entry point, a program, which effectively yields a process. Just as a program can be run several times, each run yielding a distinct process, likewise an image can be run several times, each run yielding a distinct container. And, as processes can be started, stopped, resumed, and terminated, so can containers: they, too, have a life cycle.

The software that made containers popular is Docker, and it remains the most widely used containerisation platform. It makes two design choices in the name of simplicity: it centers on a daemon, and it runs that daemon as root. The first choice creates a single point of failure: if the daemon stops, all containers become unreachable. The second has a cost in terms of security: any user authorized to communicate with the daemon thus has effective root access to the host, and can, for instance, ask it to mount the host's root filesystem into a container started from an otherwise ordinary image, hence gaining full access to the host's whole filesystem.

An alternative that addresses these drawbacks is Podman. At the cost of a more complex implementation, it departs from both of Docker's choices: it requires no daemon, and containers can be run without any root privileges, a mode known as rootless containers.

Now, turning to the constraints of Chapter 1: containers address constraint 1 through cgroups. Where the scheduling discussed in the previous section already protected the CPU, cgroups extend this protection to memory, network bandwidth, and other resources, preventing any service from monopolizing them.

Constraint 2 is addressed through namespaces: since the resources of a container are simply not visible from others, it becomes impossible for any service to access either the data or the privileges of that container.

Constraint 3 is only partially addressed: images ensure that a service can be deployed consistently across multiple hosts, which is a necessary condition for distributed deployment, but keeping services available despite hardware failures, and updating them without user-visible interruption, requires coordinating containers across a cluster of machines, which is beyond what containers themselves provide; it is the role of cluster management tools such as Kubernetes or Docker Swarm.

Constraint 4 is addressed by images: because an image is immutable and self-contained, images provide the same degree of environment consistency as the declarative approaches examined in Section 2 of this chapter. Here too, the main remaining source of variation is the host's kernel.

Finally, constraint 5 is not addressed at all by containers themselves. Coordinating the deployment, monitoring, and life cycles of many interdependent services requires tooling that operates above the level of individual containers. Tools such as Docker Compose handle this at small scale, while tools such as Helm address it at the scale of a cluster of machines.

All the approaches examined so far were developed with the constraints of the previous chapter as a central concern. Emulators were not: they were designed to serve different needs, and whatever they have to say about those constraints is incidental. We examine them here because they occupy an important place in the landscape of systems software, and because some of the concepts they rely on, in particular the distinction between host and guest, are the same ones we will encounter again when we turn to virtualization.

The first need is compatibility, understood broadly. Software written for a system that no longer exists must somehow be made to run on current hardware. But compatibility has a forward-looking face as well: a developer may want to write software that is independent of any particular platform, meaning any combination of hardware and operating system, so that it can run anywhere without modification. These two motivations are different, but they call for the same mechanism.

The second need is control. Running software in an environment that the developer fully controls, where execution can be observed, interrupted, and replayed, is valuable both for debugging and for testing. This need overlaps more directly with some of the constraints of the previous chapter, though it was not originally framed in those terms.

An emulator addresses these needs by reproducing the behavior of a target platform on a host platform. The guest program runs as though it were executing on the target platform; the host platform runs the emulator, which decodes and executes each guest instruction on the guest program's behalf. Familiar examples include FCEUX, which emulates the Nintendo Entertainment System and runs on Linux, Windows, and macOS; RetroArch, a frontend that bundles emulators for a wide range of consoles and runs on the same platforms; and DOSBox, which emulates a complete x86 PC including its processor, memory, and peripherals, and runs on Linux, Windows, and macOS. In each case the emulated system is the guest and the machine running the emulator is the host. These emulators serve the compatibility need: they make it possible to run software written for hardware that no longer exists or is no longer practical to use.

QEMU, in its emulation mode, serves both needs. It can emulate a wide range of processor architectures, making it possible to run software compiled for one architecture on a machine with a different architecture. It can also emulate a given architecture on itself, which is useful for testing and debugging: a program running under QEMU has no direct access to the host hardware, and its execution can be observed and interrupted at will.

The Java Virtual Machine follows a related but distinct logic. Rather than reproducing the behavior of an existing physical system, the JVM defines an abstract machine: a set of instructions, called bytecode, that correspond to no real processor, for which Java programs are compiled. Any conforming JVM implementation will execute them identically, regardless of the underlying host platform. This is what "virtual" means in "Java Virtual Machine": the machine exists only as a specification, not as physical hardware. The JVM was introduced in 1995 with the slogan "write once, run anywhere," and it serves the portability face of the compatibility need: write for the abstract machine, and the implementation takes care of the rest. This sense of "virtual machine" is different from the one we will encounter in the next section, where a virtual machine is a reproduction of a real physical hardware environment rather than an abstract specification.

The idea behind the JVM is older than it might seem. In 1979, Infocom designed the Z-machine, a specification for an abstract processor for which all of their interactive fiction games were compiled. A Z-machine interpreter was then written for each platform Infocom wanted to support, making the games themselves fully portable. Any conforming Z-machine interpreter will produce the same results, because the guarantee rests not on a particular implementation but on the rigor of the specification. The JVM applied the same principle to general-purpose programming sixteen years later.

Not all systems that model another system's behavior aim for this kind of faithful reproduction. A simulator models some aspects of a system's behavior for the purpose of observation or analysis, without attempting to reproduce the system completely or execute its actual code. The iOS simulator included in Xcode is a clear example. An iPhone has sensors, an accelerometer, a gyroscope, a compass, that have no physical counterpart on a Mac. The simulator does not reproduce this hardware; instead, it provides a software interface through which the developer can inject artificial sensor data, for instance simulating a rotation or a change of orientation using controls in the Xcode interface. The application under development interacts with this interface as though it were talking to real hardware. The goal here is not faithful reproduction but a controlled environment in which the application can be developed and tested without requiring a physical device.

Finally, Wine occupies a different position in this landscape. It allows programs compiled for Windows to run on Linux, but it does not emulate the Windows hardware environment or interpret Windows executable code. Instead, it provides reimplementations of the Windows API libraries, such as kernel32.dll and user32.dll. When a Windows program calls a function from one of these libraries, Wine's implementation of that function translates the call into the equivalent Linux system call. The Windows program's own code runs natively on the host processor throughout. This is why Wine's name is a recursive acronym for "Wine Is Not an Emulator": compatibility is achieved not by reproducing a hardware environment but by reimplementing the software interface that the program depends on. The result is that Windows programs run at native speed under Wine, but only if Wine implements the API calls they make.

As noted at the outset, the tools examined in this section were not designed as responses to the constraints of the previous chapter, and it would be misleading to evaluate them as if they were. Nevertheless, some observations are worth making. On constraint 1, running multiple services on the same machine with equitable access to resources, emulators bring nothing specific: multiple emulators can run concurrently on the same host, but this is simply because they are ordinary processes managed by the host operating system's scheduler. On constraint 2, preventing any service from accessing the data or privileges of another, emulators offer a form of isolation as a byproduct of how they work: a guest program can only access the resources that the emulator exposes to it. The quality of this isolation depends entirely on the emulator's implementation and on whatever isolation mechanisms the host operating system makes available; it is not guaranteed by construction. On constraint 3, deploying services across multiple machines so that they remain available despite hardware failures, emulators have nothing to offer regarding availability or seamless updates; they do, however, contribute to portability, as discussed above. On constraint 4, keeping development, testing, and production environments consistent, emulators are genuinely useful: because an emulator reproduces a target platform deterministically, a program that runs correctly under the emulator in development can be expected to behave the same way in production, regardless of differences in the host hardware. On constraint 5, coordinating the deployment, monitoring, and life cycles of many interdependent services, emulators have nothing to offer: this constraint is simply outside their scope.

This section occupies a particular place in this chapter. The approaches examined so far were chosen because they address, each in its own way, the constraints identified in the previous chapter. Machine virtualization is examined here for the same reason, but also because it is the central subject of this book: XCP-ng is a virtualization platform, and the concepts introduced in this section will be used throughout the rest of this book.

We start by defining what machine virtualization is, then turn to introducing hypervisors, before evaluating machine virtualization against the constraints identified in the previous chapter.