libguestfs Frequently Asked Questions (FAQ)

<<< Back to the libguestfs home page

1. About libguestfs

   • What is libguestfs?
   • What are the virt tools?
   • Does libguestfs need libvirt / KVM / Red Hat / Fedora?
   • How does libguestfs compare to other tools?
   • How do I know what version I'm using?
   • How can I get help? What mailing lists/IRC are available?
   • How can I report bugs?
   • How do I debug libguestfs problems?
   • Where can I get the latest binaries for ...?

2. Frequently asked questions

   • upload or write seems very slow
   • libguestfs is too slow to start!
   • libguestfs uses too much disk space!
   • libguestfs uses too much memory!
   • libguestfs hangs or fails in the run / launch / wait_ready commands
   • How do I package libguestfs for distro X?
   • Do you use virtio in the appliance?
   • The API has hundreds of methods, where do I start?
   • Why don't you do everything through the FUSE / filesystem interface?
   • Why don't you do everything through GVFS?
   • Can I use guestfish --ro as a way to backup my virtual machines?
   • Is there a binding for Python or (insert other language here)?
   • What's the difference between guestfish and virt-rescue?
   • Why can't I type full shell commands in guestfish?
   • What's the deal with guestfish -i? Why does virt-cat only work on a real VM image, but virt-df works on any disk image? What does "no root device found in this operating system image" mean?

3. Issues compiling libguestfs from source

   • libguestfs has a really long list of dependencies!
   • How do I speed up libguestfs builds?

libguestfs is a C library (hence "lib-"), and a set of tools built on this library, and a set of bindings in many different scripting languages.

For more information about what libguestfs can do read the introduction on the home page. To find out about how libguestfs works, see the libguestfs architecture page.

Virt tools (website) are a whole set of virtualization management tools aimed at system administrators. Some of them come from libguestfs, some from libvirt and many others from other open source projects. So virt tools is a superset of libguestfs. However libguestfs comes with many important tools. See the main page for a full list.

Red Hat sponsors (ie. pays for) development of libguestfs and a huge number of other open source projects. But you can run libguestfs and the virt tools on many different Linux distros and Mac OS X. Some virt tools have been ported to Windows.

vs. kpartx: Libguestfs takes a different approach from kpartx. kpartx needs root, and mounts filesystems on the host kernel (which can be insecure). Libguestfs isolates your host kernel from guests, is more flexible, scriptable, supports LVM, doesn't require root, is isolated from other processes, and cleans up after itself. Libguestfs is more than just file access.

vs. vdfuse: vdfuse is like kpartx but for VirtualBox images. See the kpartx comparison above. You can use libguestfs on the partition files exposed by vdfuse, although it's not necessary since libguestfs can access VirtualBox images directly.

vs. qemu-nbd: nbd is like kpartx but for qcow2 images. See the kpartx comparison above. You can use libguestfs and qemu-nbd together, eg. for access to block devices over the network.

vs. mounting filesystems in the host: Mounting guest filesystems in the host is insecure and should be avoided completely for untrusted guests. Use libguestfs to provide a layer of protection against filesystem exploits.

vs. parted: Libguestfs supports LVM. Libguestfs uses parted and provides most parted features through the libguestfs API.

guestfish --version

Libguestfs development happens along an unstable branch and we periodically create a stable branch which we backport stable patches to. To find out more, read about libguestfs version numbers.

If you are a Red Hat customer using Red Hat Enterprise Linux, please contact Red Hat Support.

For upstream versions we have a mailing list. You can also talk to us on IRC channel #libguestfs on FreeNode.

For upstream bugs please use this link to enter a bug in Bugzilla. Include as much detail as you can and a way to reproduce the problem. Include the full output of libguestfs-test-tool.

There are two LIBGUESTFS_* environment variables you can set in order to get more information from libguestfs.

• LIBGUESTFS_TRACE — Set this to 1 and libguestfs will print out each command / API call in a format which is similar to guestfish commands.
• LIBGUESTFS_DEBUG — Set this to 1 in order to enable massive amounts of debug messages. If you think there is some problem inside the libguestfs appliance, then you should use this option.

To set these from the shell, do this before running the program:

export LIBGUESTFS_DEBUG=1

For csh/tcsh the equivalent command would be:

setenv LIBGUESTFS_DEBUG 1

When using guestfish

You can use the same environment variables above. Alternatively use the guestfish options -x (to trace commands) or -v (to get the full debug output), or both.

For programmers using the API

Call guestfs_set_trace to enable command traces, and/or guestfs_set_verbose to enable debug messages. For best results, call these functions as early as possible, just after creating the guestfs handle if you can, and definitely before calling launch.

Digging deeper into the appliance boot process

Enable debugging and then read this documentation on the appliance boot process.

In libguestfs < 1.13.16, the mount command enabled option -o sync implicitly. This causes very poor write performance, and was one of the main gotchas for new libguestfs users.

For libguestfs < 1.13.16, replace mount with mount-options, leaving the first parameter as an empty string. You can also do this with more recent versions of libguestfs, but if you know that you are using libguestfs ≥ 1.13.16 then it's safe to use plain mount.

If the underlying disk is not fully allocated (eg. sparse raw or qcow2) then writes can be slow because the host operating system has to do costly disk allocations while you are writing. The solution is to use a fully allocated format instead, ie. non-sparse raw, or qcow2 with the preallocation=metadata option.

chmod o+rw /dev/kvm

libguestfs works by booting an appliance (like a mini-operating system), and this does take a few seconds. In libguestfs ≥ 1.5.4 (and febootstrap ≥ 2.8) we made some changes which dramatically improve boot times, down to under 5 seconds on bare metal with KVM. So you should make sure if possible to be running those or newer versions.

Linux kernel 2.6.35 is known to be much slower at starting qemu than earlier versions. Using libguestfs ≥ 1.5.4 and febootstrap ≥ 2.8 will help to workaround this problem.

If you find yourself restarting guestfish over and over again, you may find it better to use the remote control feature of guestfish to avoid the start-up costs.

If your distro packager enabled the so-called supermin appliance variant of libguestfs, then libguestfs will use quite a lot of disk space from /tmp for each libguestfs handle. Probably around 200MB per UID (but shared between handles with the same user ID).

This is unavoidable, but you can place the temporary files somewhere else by setting the TMPDIR environment variable. (Since libguestfs 1.0.63, see this bug).

Since febootstrap ≥ 2.8, the appliance is cached in TMPDIR and not deleted when libguestfs programs exit. This improves boot times because libguestfs (with the same UID) just reuses the old appliance instead of rebuilding it. You can delete the cached appliance whenever you want, or use a "/tmp cleaner".

Nevertheless you can change the amount of memory allocated to the appliance by setting the LIBGUESTFS_MEMSIZE environment variable. Be careful about setting this too small.

export LIBGUESTFS_DEBUG=1

$ guestfish -v

$ libguestfs-test-tool

Usually these problems are caused by the combination of host, qemu and appliance kernel, and are nothing to do with libguestfs. There are various ways you can try to work around these problems, which come down to trying different host, qemu and appliance combinations. Use LIBGUESTFS_QEMU to select other versions of qemu.

See also How do I debug libguestfs problems? and documentation on the appliance boot process.

To understand the answer, you need to read about the architecture of libguestfs.

Specifically we include an appliance (like a mini-operating system) with libguestfs when it is packaged for Fedora and Debian. On Fedora and Debian, this appliance gets built on the fly when the libguestfs package itself is built. The tool we use for this is called febootstrap.

You need to add a new package manager back-end to febootstrap. See this email for more details.

If you see a serious performance regression (only for certain operations) when using the virtio block driver then you can try switching to using the IDE (libata) driver:

./configure --with-drive-if=ide

Note that you will still be able to use names like /dev/sda even with the virtio driver enabled. See the section on block device naming in the manpage.

We recommend you start with the API overview.

Now although this overview covers the C API, it's still worth reading, because all the other languages use the same API, with simple logical changes to the names of the calls:

Once you're familiar with the API overview, you should look at this list of starting points for other language bindings.

We offer a command called guestmount which lets you mount guest filesystems on the host. This is implemented as a FUSE module. Why don't we just implement the whole of libguestfs using this mechanism, instead of having the large and rather complicated API?

The reasons are twofold. Firstly, libguestfs offers API calls for doing things like creating and deleting partitions and logical volumes, which don't fit into a filesystem model very easily. Or rather, you could fit them in: for example, creating a partition could be mapped to mkdir /fs/hda1 but then you'd have to specify some method to choose the size of the partition (maybe echo 100M > /fs/hda1/.size), and the partition type, start and end sectors etc., but once you've done that the filesystem-based API starts to look more complicated than the call-based API we currently have.

The second reason is for efficiency. FUSE itself is reasonably efficient, but it does make lots of small, independent calls into the FUSE module. In guestmount these have to be translated into messages to the libguestfs appliance which has a big overhead (in time and round trips). For example, reading a file in 64 KB chunks is inefficient because each chunk would turn into a single round trip. In the libguestfs API it is much more efficient to download an entire file or directory through one of the streaming calls like download or tar_out.

The problems are similar to the problems with FUSE.

GVFS is a better abstraction than POSIX/FUSE. There is an FTP backend for GVFS, which is encouraging because FTP is conceptually similar to the libguestfs API. However the GVFS FTP backend makes multiple simultaneous connections in order to keep interactivity, which we can't easily do with libguestfs.

Usually this is not a good idea. The question is answered in more detail in this mailing list posting.

We have language bindings for many common programming languages. A full list is on the front page and this link describes how to get started in each language.

One thing you should note about libguestfs is that we prefer to autogenerate as much boilerplate code as possible, and that includes language bindings. Therefore if you wish to add a new programming language to the mix (which is something we welcome) it's best to get that language added to the generator.

Which language did we use to write the generator? OCaml, which is a modern take on ML, a language which has been around since 1974 and was originally designed in order to write compilers, and therefore is very suitable for writing code generation. So you'll need to learn a very little bit of OCaml in order to add your code to the generator.

$ guestfish --ro -a guest.img
><fs> run
><fs> fsck /dev/sda1

$ virt-rescue --ro guest.img
><rescue> /sbin/fsck /dev/sda1

guestfish is a program providing structured access to the guestfs API. It happens to be a nice interactive shell too, but its primary purpose is structured access from shell scripts. Think of it more like a language binding, like Python and other bindings, but for shell. The key differentiating factor of guestfish (and the libguestfs API in general) is the ability to automate changes.

virt-rescue is a free-for-all freeform way to boot the libguestfs appliance and make arbitrary changes to your VM. It's not structured, you can't automate it, but for making quick ad-hoc fixes to your guests, it can be quite useful.

See the question above.

But, libguestfs also has a "backdoor" into the appliance allowing you to send arbitrary shell commands. It's not as flexible as virt-rescue, because you can't interact with the shell commands, but here it is anyway:

><fs> debug sh "cmd arg1 arg2 ..."

Note that you should not rely on this. It could be removed or changed in future. If your program needs some operation, please add it to the libguestfs API instead.

At the libguestfs API and guestfish level, a "disk image" is just a pile of partitions and filesystems.

In contrast, when the virtual machine boots, it mounts those filesystems into a consistent hierarchy such as:

 /          (/dev/sda2)
 |
 +-- /boot  (/dev/sda1)
 |
 +-- /home  (/dev/vg_external/Homes)
 |
 +-- /usr   (/dev/vg_os/lv_usr)
 |
 +-- /var   (/dev/vg_os/lv_var)

(or drive letters on Windows).

The libguestfs API however knows nothing about how filesystems are mounted. In general, a disk image that we just made up might not contain an operating system or "mountpoints" at all. The previous statement was true up to libguestfs 1.5.3, but now the API contains a way to inspect disk images and work out how the mountpoints are arranged. So while libguestfs usually treats a disk image as a mere "pile of partitions" it is also possible to heuristically determine mountpoints for many guest operating systems.

Users expect some tools (like virt-cat) to work with VM paths:

virt-cat fedora.img /var/log/messages

How does virt-cat know that /var is a separate partition?

The trick is a large external program called virt-inspector. Virt-inspector contains all sorts of tests and heuristics, so it can take a disk image about which nothing is known beforehand and work out both what it is, and how filesystems would be mounted. Some of the functionality of virt-inspector is now available in the core API.

Some tools, such as virt-cat, virt-edit, virt-ls and virt-tar use the same virt-inspector code to map VM paths. Other tools, such as virt-df and virt-filesystems operate entirely at the raw "big pile of filesystems" level of the libguestfs API, and don't use virt-inspector code.

guestfish is in an interesting middle ground. If you use the -a and -m command line options, then you have to tell guestfish exactly how to add disk images and where to mount partitions. This is the raw API level.

However there is also a guestfish -i (for inspector) mode. In this mode, guestfish runs virt-inspector, and then (if successful) virt-inspector runs guestfish a second time with the correct -a and -m options. This is much slower, but usually simpler for the user. Since libguestfs 1.5.3, guestfish -i uses the new guestfs_inspect_os API and thus can work out mountpoints much more quickly than before.

The error no root device found in this operating system image is related to this. It means virt-inspector was unable to locate an operating system within the disk image you gave it. You might see this from programs like virt-cat if you try to run them on something which is just a disk image, not an actual VM disk image.

If you're on Fedora and want to build libguestfs from source, a good place to start is to look at all of the BuildRequires lines in the libguestfs spec file. Once you've installed all of those packages, you should be good.

A very good place to start with Squid configuration is here: Using Squid to Speed Up Mock package downloads.

Make sure Squid is running, and that the environment variables $http_proxy and $ftp_proxy are pointing to it.

I have a file /etc/profile.d/local.sh that contains:

export http_proxy=http://192.168.2.1:3128/
export ftp_proxy=$http_proxy

where 192.168.2.1:3128 is the address and port number of my Squid proxy.

With that, the appliance building step should be reduced to a few minutes.

$Id: FAQ.html,v 1.31 2011/10/01 09:39:49 rjones Exp $