A distributed file system can be used on the local hard disks of a KVM host. In connection with KVM Glusterfs is a useful candidate.

Using a distributed file system can have the following advantages:

• The preconditions for a live migration are met.
• With the storage strategy of replicated volumes the reliability is increased because the disk images are replicated across several KVM hosts.

The live migration based on Glusterfs was already successfully tested.

For a distributed file system the use of a dedicated storage network is also recommended. In contradiction to iSCSI one instead of two additional network interfaces per host is sufficient.

The use of a distributed file system can have the following disadvantages:

• Storage capacity can decrease by the redundancy: A group of e.g. three identical KVM hosts can have the storage capacity of a single host all-together.
• The loss of storage capacity possibly has to be compensated by additional hard disks for all tree hosts.
• When using a storage strategy for increasing the redundancy, the performance of the hard disks in the best case does not decrease.
• Using a distributed file system increases the complexity of the virtualization platform.
• For the additional storage network, the administration of the distributed file system and troubleshooting additional knowledge and experience is necessary which is not always available.
• Distributed file systems can be rendered unusable if only one of the involved hosts becomes inaccessible. This increases the risk of loosing virtual hosts dramatically.
• Restarting services or rebooting KVM hosts gets more complicated while a distributed file system is used.

Especially because of the additionally needed storage capacity and the complexity a distributed file system is often not used.

An alternative for a distributed file system is iSCSI. With iSCSI several KVM hosts access a common storage via a storage network.

Without an additional distributed file system it is of course also not possible that several KVM hosts write to the same file on the same iSCSI storage. But for the aim of the use of distributed storage - the live migration of virtual hosts - common write access is not necessary.

It is only necessary that several KVM hosts can write to the same storage consecutively. This works best on iSCSI in combination with LVM: LVM can block and unblock the access to virtual host images and makes sure that only one KVM host can write to the same image at a time. This works perfectly without distributed file system.

The prerequisites for a live migration using iSCSI are:

• The storage must be accessible via the same path on all involved KVM hosts.
• The cache of the virtual disks has to be disabled in the settings of the virtual host.

We dedicated a complete chapter to the topic KVM and iSCSI.

previous chapter | contents