Firecracker is an open source Virtual Machine Monitor (VMM) that enables secure, multi-tenant, minimal-overhead execution of container and function workloads.

Firecracker was built by developers at Amazon Web Services to enable services such as AWS Lambda and AWS Fargate to improve resource utilization and customer experience, while providing the security and isolation required of public cloud infrastructure. Firecracker started from Chromium OS's Virtual Machine Monitor, crosvm, an open source VMM written in Rust. Today, crosvm and Firecracker have diverged to serve very different customer needs. Rust-vmm is an open source community where we collaborate with the crosvm maintainers and other groups and individuals to build and share quality Rust virtualization components.

When we launched Lambda in November of 2014, we were focused on providing a secure serverless experience. At launch we used per-customer EC2 instances to provide strong security and isolation between customers. As Lambda grew, we saw the need for technology to provide a highly secure, flexible, and efficient runtime environment for services like Lambda and Fargate. Using our experience building isolated EC2 instances with hardware virtualization technology, we started an effort to build a VMM that was tailored to integrate with container ecosystems.

The Firecracker VMM is built to be processor agnostic. Intel, AMD and 64 bit ARM processors are supported for production workloads.

You can find more details here.

Yes. Firecracker is integrated with Kata Containers, Weave FireKube (via Weave Ignite), and containerd via firecracker-containerd. We welcome contributions that enable Firecracker to integrate naturally with the container ecosystem and provide more choices in how container workloads are isolated.

Firecracker is an alternative to QEMU that is purpose-built for running serverless functions and containers safely and efficiently, and nothing more. Firecracker is written in Rust, provides a minimal required device model to the guest operating system while excluding non-essential functionality (only 6 emulated devices are available: virtio-net, virtio-balloon, virtio-block, virtio-vsock, serial console, and a minimal keyboard controller used only to stop the microVM). This, along with a streamlined kernel loading process enables a < 125 ms startup time and a < 5 MiB memory footprint. The Firecracker process also provides a RESTful control API, handles resource rate limiting for microVMs, and provides a microVM metadata service to enable the sharing of configuration data between the host and guest.

Firecracker supports Linux host and guest operating systems as well as OSv guests. Currently supported host/guest kernel versions can be found in the kernel support policy.

Firecracker is licensed under the Apache License, version 2.0, allowing you to freely use, copy, and distribute your changes under the terms of your choice. Read more about Apache 2.0. Crosvm code sections are licensed under a BSD-3-Clause license that also allows you to use, copy, and distribute your changes under the terms of your choice.

Firecracker is an AWS open source project that encourages contributions from customers and the developer community. Any contribution is welcome as long as it aligns with our charter. You can learn more about how to contribute in CONTRIBUTING.md. You can chat with others in the community on the Firecracker Slack workspace.

The Firecracker team at Amazon Web Services owns project maintainer responsibilities, permissions to merge pull requests, and the ability to create new Firecracker releases.

Guest operating systems must be built for the same CPU architecture as the host on which it will run. Firecracker does not support running microVMs on any architecture other than the one the host is running on. In other words, running an OS built for a x86_64 on an aarch64 system will not work, and vice versa.

Initrds are only recently supported in Firecracker. If your release predates issue #228 being resolved, please update.

In order to debug the issue, check the response of the InstanceStart API request. Possible responses:

• Error: Submit a new issue with the label "Support: Failure".
• Success: If the boot was successful, you should get a response with 204 as the status code.

If you have no output in the console, most likely you will have to update the kernel command line. By default, Firecracker starts with the serial console disabled for boot time performance reasons.

Example of a kernel valid command line that enables the serial console (which goes in the boot_args field of the /boot-source Firecracker API resource):

console=ttyS0 reboot=k panic=1 pci=off nomodule

The ip= boot param in the linux kernel only actually supports configuring a single interface. Multiple interfaces can be set up in Firecracker using the API, but guest IP configuration at boot time through boot arguments can only be done for a single interface.

The canonical solution is to use NTP in your guests.

However, if you want to run Firecracker at scale, we suggest using a PTP emulated device as the guest's NTP time source so as to minimize network traffic and resource overhead. With this solution the guests will constantly update time to stay in sync with host wall-clock. They do so using cheap para-virtualized calls into kvm ptp instead of actual network NTP traffic.

To be able to do this you need to have a guest kernel compiled with KVM_PTP support:

CONFIG_PTP_1588_CLOCK=y
CONFIG_PTP_1588_CLOCK_KVM=y

Our recommended x86_64 guest kernel config already has these included.

Now /dev/ptp0 should be available in the guest. Next you need to configure /dev/ptp0 as a NTP time source.

For example when using chrony:

1. Add refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0 to the chrony conf file (/etc/chrony/chrony.conf)
2. Restart the chrony daemon.

You can see more info about the refclock parameters here. Adjust them according to your needs.

The relatively high FD usage is expected and correct. Firecracker heavily relies on event file descriptors to drive device emulation.

There is no relation between the numbering of the /network-interface API calls and the number of the network interface in the guest. Rather, it is usually the order of network interface creation that determines the number in the guest (but this depends on the distribution).

For example, when you create two network interfaces by calling /network-interfaces/1 and then /network-interfaces/0, it may result in this mapping:

/network-interfaces/1 -> eth0
/network-interfaces/0 -> eth1

Firecracker does not implement ACPI and PM devices, therefore operations like gracefully rebooting or powering off the guest are supported in unconventional ways.

Running the poweroff or halt commands inside a Linux guest will bring it down but Firecracker process remains unaware of the guest shutdown so it lives on.

Running the reboot command in a Linux guest will gracefully bring down the guest system and also bring a graceful end to the Firecracker process.

On x86_64 systems, issuing a SendCtrlAltDel action command through the Firecracker API will generate a Ctrl Alt Del keyboard event in the guest which triggers a behavior identical to running the reboot command. This is, however, not supported on aarch64 systems.

Check out our rootfs and kernel image creation guide.

If you see errors like ...

[<TIMESTAMP>] fc_vmm: page allocation failure: order:6, mode:0x140c0c0
(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null)
[<TIMESTAMP>] fc_vmm cpuset=<GUID> mems_allowed=0

... then your host is running out of memory. KVM is attempting to do an allocation of 2^order bytes (in this case, 6) and there aren't sufficient contiguous pages.

Possible mitigations are:

• Reduce memory pressure on the host.
• Maybe the host has memory but it's too fragmented for the kernel to use. The allocation above of order 6 means the kernel could not find 2^6 consecutive pages. One way to mitigate memory fragmentation is to set a higher value for vm.min_free_kbytes
  • Or investigate other mitigations

Passing an optional command line parameter, --config-file, to the Firecracker process allows this type of configuration. This parameter must be the path to a file that contains the JSON specification that will be used to configure and start the microVM. One example of such file can be found at tests/framework/vm_config.json.

If the Firecracker process exits with 12 exit code (Out of memory error), the root cause is that there is not enough memory on the host to be used by the Firecracker microVM.

If the microVM was not configured in terms of memory size through an API request, the host needs to meet the minimum requirement in terms of free memory size, namely 128 MB of free memory which the microVM defaults to.

This may be related to "We are seeing page allocation failures ..." above. To validate, run this:

sudo dmesg | grep "page allocation failure"

If another hypervisor like VMware or VirtualBox is running on the host and locks /dev/kvm, Firecracker process will fail to start with "Resource busy" error.

This issue can be resolved by terminating the other hypervisor running on the host, and allowing Firecracker to start.