From: Miroslav Rezanina <mrezanin@redhat.com>
Date: Mon, 11 May 2009 08:37:37 -0400
Subject: [xen] deadlock between libvirt and xentop
Message-id: 1074031810.137691242045457218.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com
O-Subject: Re: [PATCH RHEL 5.4] BZ 499013 Deadlock between libvirt and xentop
Bugzilla: 499013
RH-Acked-by: Chris Lalancette <clalance@redhat.com>
RH-Acked-by: Rik van Riel <riel@redhat.com>
RH-Acked-by: Justin M. Forbes <jforbes@redhat.com>
RH-Acked-by: Don Dutile <ddutile@redhat.com>

Problem in BZ 499013 (https://bugzilla.redhat.com/show_bug.cgi?id=499013) has relation to xen-unstable c/s 19519 mentioned in BZ 494873 (https://bugzilla.redhat.com/show_bug.cgi?id=494873). In our code, deadlock was only on domctl_lock spinlock, that encapsulate code that try to pause vcpu. When this vcpu try to execute do_domctl() code while request for pause was waiting for vcpu to be paused, it can't pause as it try to obtain domctl_lock that was taken by process trying to pause this vcpu.
Running command set described in BZ 499013 caused machine to freeze. After patch was applied, both commands can be executed simultaneously for at least 10 minutes without freezing.

--
Miroslav Rezanina
Software Engineer - Virtualization Team - XEN kernel

diff --git a/common/domctl.c b/common/domctl.c
index 5a61bfc..2b44179 100644
--- a/common/domctl.c
+++ b/common/domctl.c
@@ -194,7 +194,8 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domctl_t) u_domctl)
     if ( op->interface_version != XEN_DOMCTL_INTERFACE_VERSION )
         return -EACCES;
 
-    spin_lock(&domctl_lock);
+    if ( !spin_trylock(&domctl_lock) )
+      return hypercall_create_continuation(__HYPERVISOR_domctl, "h", u_domctl);
 
     switch ( op->cmd )
     {