From: Larry Woodman <lwoodman@redhat.com>
Date: Tue, 15 Sep 2009 16:24:32 -0400
Subject: [x86_64] PCI space below 4GB forces mem remap above 1TB
Message-id: 1253046272.25569.105.camel@dhcp-100-19-198.bos.redhat.com
O-Subject: [RHEL5-U5 patch] Prevent boot-time panic when system has 1T RAM and the PCI space below 4GB forces memory remapping above 1TB
Bugzilla: 523522
RH-Acked-by: Rik van Riel <riel@redhat.com>
RH-Acked-by: Anton Arapov <anton@redhat.com>
RH-Acked-by: Stefan Assmann <sassmann@redhat.com>
RH-Acked-by: Prarit Bhargava <prarit@redhat.com>

When holes in RAM push end of memory above 1TB system panics on
boot freeing the bootmem because end of RAM is higher than
2^40(MAX_PHYSMEM_BITS).

This e820 map has 1TB but since PCI re-mapped end_pfn is above 1TB:
----------------------------------------------------------------
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000010000 - 0000000000095c00 (usable)
 BIOS-e820: 0000000000095c00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000007f750000 (usable)
 BIOS-e820: 000000007f75e000 - 000000007f760000 type 9
 BIOS-e820: 000000007f760000 - 000000007f76e000 (ACPI data)
 BIOS-e820: 000000007f76e000 - 000000007f7d0000 (ACPI NVS)
 BIOS-e820: 000000007f7d0000 - 000000007f7e0000 (reserved)
 BIOS-e820: 000000007f7ec000 - 0000000090000000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000010080000000 (usable)
-----------------------------------------------------------------
Unable to handle kernel paging request at 000000038000001c RIP:
 [<ffffffff8041050c>] __free_pages_bootmem+0x72/0xc0
PGD 0
Oops: 0002 [1] SMP
last sysfs file:
CPU 0
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.18-164.el5 #1
RIP: 0010:[<ffffffff8041050c>]  [<ffffffff8041050c>]
__free_pages_bootmem+0x72/0xc0
RSP: 0018:ffffffff803f1ee0  EFLAGS: 00010206
RAX: 0000000380000054 RBX: ffff81e081c369c8 RCX: 0000000000000001
RDX: 000000038000001c RSI: 0000000000000006 RDI: 000000038000001c
RBP: ffffffffffffffff R08: 0000000000000000 R09: ffffffff803f1ea8
R10: ffffffff803f1ea8 R11: 0000000000001000 R12: 0000000010000000
R13: 0000000000000000 R14: 0000000001f80000 R15: ffffffff80467a40
FS:  0000000000000000(0000) GS:ffffffff803c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 000000038000001c CR3: 0000000000201000 CR4: 00000000000006a0
Process swapper (pid: 0, threadinfo ffffffff803f0000, task ffffffff802ffae0)
Stack:  ffffffff8040fb88 ffff81e080006000 0000000001f1179a 0000000002000000
 0000000100000000 0000000380000000 0000000000000007 000000000dc88a04
 00000000000000d0 0000000000000000 0000000000000000 ffff810000000000
Call Trace:
 [<ffffffff8040fb88>] free_all_bootmem_core+0xf9/0x221
 [<ffffffff8040c920>] numa_free_all_bootmem+0x3b/0x74
 [<ffffffff8027cd02>] _etext+0x0/0x17e2fe
 [<ffffffff8040ba4a>] mem_init+0x78/0x19b
 [<ffffffff804113c4>] inode_init_early+0x53/0x71
 [<ffffffff803fb75d>] start_kernel+0x180/0x225
 [<ffffffff803fb22f>] _sinittext+0x22f/0x236
------------------------------------------------------------------------------

The attached patch fixes this problem by limiting the end_pfn of physical memory
to 1TB.

Fixes BZ 523522

diff --git a/arch/x86_64/kernel/e820.c b/arch/x86_64/kernel/e820.c
index f6702b2..6505e3b 100644
--- a/arch/x86_64/kernel/e820.c
+++ b/arch/x86_64/kernel/e820.c
@@ -194,7 +194,7 @@ void __init e820_bootmem_free(pg_data_t *pgdat, unsigned long start,unsigned lon
 			free_bootmem_node(pgdat, addr, last-addr);
 	}
 }
-
+#define TB_PFN (1UL << (40 - PAGE_SHIFT))
 /*
  * Find the highest page frame number we have available
  */
@@ -219,7 +219,9 @@ unsigned long __init e820_end_of_ram(void)
 				end_pfn_map = end>>PAGE_SHIFT;
 		} 
 	}
-
+	/* cap last pfn at 1TB */
+	if (end_pfn > TB_PFN)
+		end_pfn = TB_PFN;
 	if (end_pfn > end_pfn_map) 
 		end_pfn_map = end_pfn;
 	if (end_pfn_map > MAXMEM>>PAGE_SHIFT)