From: Don Dutile <ddutile@redhat.com>
Date: Tue, 25 Nov 2008 18:03:37 -0500
Subject: [xen] build xen-platform-pci as a module
Message-id: 492C8449.3090609@redhat.com
O-Subject: [RHEL5.3 PATCH] [BZ472504] - build xen-platform-pci as a module [V2]
Bugzilla: 472504
RH-Acked-by: Jon Masters <jcm@redhat.com>
RH-Acked-by: Rik van Riel <riel@redhat.com>
RH-Acked-by: Prarit Bhargava <prarit@redhat.com>
RH-Acked-by: Mark McLoughlin <markmc@redhat.com>

BZ 472504

Version 2: with changes due to reviewer feedback and extensive testing/shakedown
           to ensure no other regressions, lost functionality.

RHEL5.2 shipped an external driver update package, kmod-xenpv, containing
four loadable modules: xen-platform-pci.ko, xen-balloon.ko, xen-vbd.ko, &
xen-vnif.ko.
When this functionality was included in RHEL5.3, the module configuation
was changed to include the xen-platform-pci.ko into the built-in kernel module
to mimic the equivalent functionality in the kernel-xen package for xen driver support.
This causes havoc with module dependencies when there are older external
modules installed on a 5.2 system and then an upgraded 5.3 kernel is added, because depmod is
unable to determine the proper dependencies between the older modules and in-kernel symbols.

Thus, if a RHEL5.2 fully-virtualized (FV) guest installation has the
kmod-xenpv package installed, and then upgrades to RHEL5.3 (without this patch),
the guest kernel will fail to load the xen-vbd.ko module, breaking
the guest configuration.
 -- a regression if rhel5.3 installed on rhel5.2+kmod-xenpv FV guest.
Removing the kmod-xenpv prior to a RHEL5.3 upgrade would break older
kernels, if the customer wants to keep both kernel versions in the guest,
and is considered improper to break older kernels when a new kernel is installed.

The following patch pulls the xen-platform-pci.ko functionality
back out of the built-in kernel module, makes it a separate module as before,
and depmod now creates the proper module dependencies, and xen-vbd.ko auto-loads
correctly.

Minor changes to other files were needed beyond just changing the Makefiles:
 (a) drivers/xen/core/gnttab.c
       -- changed gnttab_init() from __init to __devinit, so
          it is guaranteed to be available during dynamic load of
          xen-platform-pci.ko; without this fix, Section mismatch build warnings are generated.
 (b) execve() is not exported, and call_usermodehelper() was designed
     to do the equivalent for loadable modules, so update reboot.c to match upstream.
 (c) xenbus_probe.c
     there was code to remove unconnected xvd devices under /sys/devices/xen
     to prevent an anaconda install failure.
     that code was wired to the non-MODULE code path.
     so, the addition to xenbus_probe() does the same thing (once) at
     xen-platform-pci.ko load time to ensure.
     The code is wrapped around with ifdef HVM & ifdef MODULE's so
     kernel-xenU (PV guests) are not affected by this functionality
     (coding to prevent regression on PV guests).
 (d) platform-compat.c
     ctrl_alt_del() functionality is not exported from the kernel.
     when needed by the xenpv shutdown code, it has to provide an equivalent
     version.  The only problem is the kernel/sys.c version has an ioctl that
     can disable ctrl_alt_del.  That functionality is lost (for a FV guest) once
     xen-platform-pci is made a loadable vs built-in to the main kernel module.
     This is what upstream does and the current kmod-xenpv if loaded on a FV (5.2) guest,
     so no regression with this change.

Testing:
(a) tested a FV x86_64 guest; none of the modified modules are 32/64-bit dependent.
(b) ran kudzu -p -c HD to ensure only IDE shown, no xvd's;
    xen tools will present both IDE & xvda for IDE attachment;
    if the xvd is left in the system, it'll appear under /sys/devices/xen/vbd-xxxx
    and that'll blow up anaconda at install time when it tries to open that
    device as an optional boot device.
(c) booted a PV x86_64 guest to ensure it still works
(d) ran xm block-attach against FV guest -- can mount & traverse
    the xvd-attached file system.
    *** NOTE ***: Could not do an xm block-detach;
                 -- after much struggling, proved that an old 5.2+kmod-xenpv
                    block-detach failed as well (and it worked previously),
                    so either my xen tools are broken, or a recent update broke them;
                    -- outstanding email & possible BZ on this lost functionality.
(e) rebuilt initrd to include xen-vbd & xen-vnif, and able to boot
    a FV guest using xvd as boot device, xen-vnif as network device.
(f) xm block-attach done to PV guest to ensure nothing gone a foul there as well.

Full brew build on-going:
https://brewweb.devel.redhat.com/taskinfo?taskID=1588682

Please review & ACK.

- Don

diff --git a/drivers/xen/core/gnttab.c b/drivers/xen/core/gnttab.c
index 0b165b8..9ed6b50 100644
--- a/drivers/xen/core/gnttab.c
+++ b/drivers/xen/core/gnttab.c
@@ -579,7 +579,7 @@ static int gnttab_expand(unsigned int req_entries)
 	return rc;
 }
  
-int __init gnttab_init(void)
+int __devinit gnttab_init(void)
 {
 	int i;
 	unsigned int max_nr_glist_frames, nr_glist_frames;
diff --git a/drivers/xen/core/reboot.c b/drivers/xen/core/reboot.c
index 7c22041..929c590 100644
--- a/drivers/xen/core/reboot.c
+++ b/drivers/xen/core/reboot.c
@@ -240,7 +240,8 @@ static int shutdown_process(void *__unused)
 
 	if ((shutting_down == SHUTDOWN_POWEROFF) ||
 	    (shutting_down == SHUTDOWN_HALT)) {
-		if (execve("/sbin/poweroff", poweroff_argv, envp) < 0) {
+		if (call_usermodehelper("/sbin/poweroff", poweroff_argv,
+					envp, 0) < 0) {
 #ifdef CONFIG_XEN
 			sys_reboot(LINUX_REBOOT_MAGIC1,
 				   LINUX_REBOOT_MAGIC2,
diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
index 1142c8b..40cf81f 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -90,6 +90,8 @@ static int xenbus_probe_backend(const char *type, const char *domid);
 static int xenbus_dev_probe(struct device *_dev);
 static int xenbus_dev_remove(struct device *_dev);
 static void xenbus_dev_shutdown(struct device *_dev);
+static int print_device_status(struct device *dev, void *data);
+static void xvd_dev_shutdown(void);
 
 /* If something in array of ids matches this device, return it. */
 static const struct xenbus_device_id *
@@ -1037,6 +1039,11 @@ EXPORT_SYMBOL_GPL(unregister_xenstore_notifier);
 
 void xenbus_probe(void *unused)
 {
+#ifdef CONFIG_XEN_PV_ON_HVM
+#ifdef MODULE
+	struct device_driver *drv = NULL;
+#endif
+#endif
 	BUG_ON((xenstored_ready <= 0));
 
 	/* Enumerate devices in xenstore. */
@@ -1050,6 +1057,16 @@ void xenbus_probe(void *unused)
 	register_xenbus_watch(&be_watch);
 #endif
 
+#ifdef CONFIG_XEN_PV_ON_HVM
+#ifdef MODULE /* to mark unused xen devices after auto-load */
+	bus_for_each_dev(&xenbus_frontend.bus, NULL, drv,
+			 print_device_status);
+#endif
+	/* now see about removing unused xvd's */
+	xvd_dev_shutdown();
+#endif
+
+
 	/* Notify others that xenstore is up */
 	blocking_notifier_call_chain(&xenstore_notifier_list, 0, NULL);
 }
@@ -1273,6 +1290,11 @@ static void xendev_rem_add(struct xenbus_device *dev)
 	}
 	new->xendev = dev;
 
+	/* need to init if not done so already; for xen-platform-pci build */
+	if (xendev_rem_hd.list.next == NULL) {
+		INIT_LIST_HEAD(&xendev_rem_hd.list);
+	}
+
 	spin_lock(&xendev_rem_lock);
 	/* make sure xendev not already on the list */
 	list_for_each_entry(ptr, &xendev_rem_hd.list, list) {
diff --git a/drivers/xenpv_hvm/balloon/Makefile b/drivers/xenpv_hvm/balloon/Makefile
index a80677f..ed7faca 100644
--- a/drivers/xenpv_hvm/balloon/Makefile
+++ b/drivers/xenpv_hvm/balloon/Makefile
@@ -1,6 +1,6 @@
 include $(src)/../overrides.mk
 
-obj-y  = xen-balloon.o
+obj-m  = xen-balloon.o
 
 EXTRA_CFLAGS += -I$(src)/../platform-pci
 
diff --git a/drivers/xenpv_hvm/platform-pci/Makefile b/drivers/xenpv_hvm/platform-pci/Makefile
index 901a4e0..1907af9 100644
--- a/drivers/xenpv_hvm/platform-pci/Makefile
+++ b/drivers/xenpv_hvm/platform-pci/Makefile
@@ -2,7 +2,7 @@ include $(src)/../overrides.mk
 
 EXTRA_CFLAGS += -I$(src)/../platform-pci
 
-obj-y += xen-platform-pci.o
+obj-m += xen-platform-pci.o
 
 xen-platform-pci-objs := evtchn.o platform-compat.o platform-pci.o xen_support.o
 xen-platform-pci-objs += ../../xen/core/gnttab.o
diff --git a/drivers/xenpv_hvm/platform-pci/platform-compat.c b/drivers/xenpv_hvm/platform-pci/platform-compat.c
index 2e77cec..abe5df9 100644
--- a/drivers/xenpv_hvm/platform-pci/platform-compat.c
+++ b/drivers/xenpv_hvm/platform-pci/platform-compat.c
@@ -12,7 +12,8 @@ static int system_state = 1;
 EXPORT_SYMBOL(system_state);
 #endif
 
-#if 0
+#ifdef MODULE
+/* Need to include this for modular build; in kernel/sys.c if built into kernel */
 void ctrl_alt_del(void)
 {
 	kill_proc(1, SIGINT, 1); /* interrupt init */