===== No communication with a virtual host =====
==== Error description ====
The first hint to the error are often monitoring error messages stating that the vhost is down. Trials afterwards to access the vhost via network (e.g. ''ping'') fail. After login to the vhost (e.g. from the serial console) it can be found that no outgoing network communication from the vhost is possible. 
==== Failure cause ====
The libvirtd service of the KVM host cannot communicate with the vhost process '/usr/bin/kvm' of the vhost.

The error is caused by a [[https://bugzilla.redhat.com/show_bug.cgi?id=676205|bug]]. It is caused by a race condition during the access to the vhost process.

According to [[https://bugs.launchpad.net/nova/+bug/1254872|this source]] the error should be fixed with libvirtd-bin version v1.0.3 because beginning with this version the access to the vhost process is done with more granularity.

The error has indeed not been observed since libvirtd-bin versions v1.1 and above are used.
==== Solution ====
There are several solution opportunities. They are sorted by the severity of influence on the KVM host and the possible risks for the stability of the KVM host.
=== First solution: restart the vhost ===
The easiest solution is the restart of the vhost. There are three opportunities to do this.
== 1. Restart from the command line of the vhost ==
  root@vhost:~# shutdown –r now
Restart the vhost from the KVM host afterwards:
  root@kvm:~# virsh start vhost
== 2. Restart from the command line of the KVM host ==
This is only the second best option because the vhost is not properly shut down:
  root@kvm:~# virsh destroy vhost
  root@kvm:~# virsh start vhost
== 3. Restart of the vhost from the virtual machine manager (VMM) ==
  * Start the VMM
  * Select the KVM host and connect to it
  * From the opening list of vhosts choose the affected vhost
  * Start the VNC window of the vhost by double click
  * Stop the vhost by the menu item 'Virtual Machine' -> 'Shut Down' -> 'Force Off'
  * Start the vhost by the menu item 'Virtual Machine' -> 'Run'

If the libvirtd service cannot communicate with the kvm process all three trials will fail. On trial 1.) the vhost will not shut down, on trial 3.) nothimg will happen at all and on trial 2.) after some time (30-60s) the following error messages will occur:
  root@kvm:~# virsh destroy vhost
  error: Timed out during operation: cannot acquire state change lock
The error message is a secure indication that the libvirtd service cannot communicate with the kvm process. Another secure indication to the problem is the fact that the affected vhost is displayed as running in the VMM, but the window for the 'CPU usage' of the vhost does not show any activity.
=== Second solution: restart the libvirtd service ===
As a next step it can be tried to restart the libvirtd service to get the service to communicate with the vhost again:
  root@kvm:~# /etc/init.d/libvirt-bin restart
Normally all running virtual hosts will keep running without interrupt when the libvirtd service is restarted, because the libvirtd service in fact monitors the vhost execution but the vhosts are running in their own processes.

In many cases the communication with the affected vhost or at least the restart of the vhost will be possible
again after the restart of libvirtd. But in some cases the restart of libvirtd did not help.

Please note:\\
There exist no completely reliable statements, that the restart of the libvirtd service under all circumstances - e.g. also after an update of libvirtd - will certainly not interrupt the execution of vhosts. In order to be 100% secure, possibly move all other running vhosts to other KVM hosts before the restart of libvirtd. After move of all vhosts the third solution is unnecessary because the KVM host also could be rebooted.
=== Third solution: redefine the vhost ===
As a last resort create a new vhost using the virtual hard disk of the failed vhost and abandon the failed vhost to his fate till the next reboot of the KVM host.

At first, save the definition of the old host:
  root@kvm:~# virsh dumpxml vhost >vhost_new.xml
Afterwards the saved xml file has to be edited to change the following information:
  * The name of the vhost (tags <name>...</name>)
  * The UUID of the vhost (tags <uuid>...</uuid>) 
A new and valid UUID can be generated with the command ''uuid'' from the software package 'uuid'.

In a next step the new vhost is defined and started:
  root@kvm:~# virsh define vhost_new.xml
  root@kvm:~# virsh start vhost_new
  
[[public:trouble:cache|previous chapter]] | [[public:start|contents]] | [[public:trouble:checksum|next chapter]]