From: Bhavna Sarathy <bnagendr@redhat.com>
Date: Fri, 20 Nov 2009 15:47:52 -0500
Subject: [xen] fix numa on magny-cours systems
Message-id: <4B06BA28.8070100@redhat.com>
Patchwork-id: 21451
O-Subject: Re: [RHEL5.5 PATCH] Fix Xen NUMA on Magny-cours systems
Bugzilla: 526051
RH-Acked-by: Prarit Bhargava <prarit@redhat.com>
RH-Acked-by: Christopher Lalancette <clalance@redhat.com>

Resolves BZ 526051

Here is the final patch that boots Xen with NUMA on Magny-cours.   The patch takes care to 
not mess anything up for ia64 and respect the dma_ bitsize entered by the user on both x86 
and ia64.  Here's the brew build to prove that.

https://brewweb.devel.redhat.com/taskinfo?taskID=2085397

Chris and I worked on the patch together, we agreed to retain dma_emergency_pool as there 
is no gain following the upstream version of the patch, and it might make the RHEL5 Xen 
unstable.

When NUMA is enabled on Magny-cours system, the system crashes during boot.
The reason was an overlap of Xen's DMA region (everything below 4GB per
default) and the memory on the first node.

Upstream link:
http://xenbits.xen.org/xen-3.3-testing.hg?rev/445681d122c0

Tested successfully on a couple of different Magny-cours 2P systems.

Bhavna

diff --git a/common/page_alloc.c b/common/page_alloc.c
index 8dd18e2..5eb9238 100644
--- a/common/page_alloc.c
+++ b/common/page_alloc.c
@@ -54,7 +54,7 @@ boolean_param("bootscrub", opt_bootscrub);
 /*
  * Bit width of the DMA heap.
  */
-static unsigned int dma_bitsize = CONFIG_DMA_BITSIZE;
+static unsigned int dma_bitsize = 0;
 static void __init parse_dma_bits(char *s)
 {
     unsigned int v = simple_strtol(s, NULL, 0);
@@ -604,6 +604,20 @@ void __init end_boot_allocator(void)
             init_heap_pages(pfn_dom_zone_type(i), mfn_to_page(i), 1);
     }
 
+    if (dma_bitsize == 0)
+    {
+#ifdef CONFIG_X86
+	if (num_online_nodes() > 1)
+	    dma_bitsize = min_t(unsigned int,
+		fls(NODE_DATA(0)->node_spanned_pages) - 1 + PAGE_SHIFT - 2,
+			32);
+	else
+	    dma_bitsize = CONFIG_DMA_BITSIZE;
+#else
+	dma_bitsize = CONFIG_DMA_BITSIZE;
+#endif
+    }
+
     printk("Domain heap initialised: DMA width %u bits\n", dma_bitsize);
 }
 #undef avail_for_domheap