From: Bhavna Sarathy <bnagendr@redhat.com>
Date: Wed, 28 Oct 2009 19:59:00 -0400
Subject: [x86_64] amd: iommu system management erratum 63 fix
Message-id: <4AE8A284.8050900@redhat.com>
Patchwork-id: 21247
O-Subject: Re: [RHEL5 PATCH] Add AMD IOMMU erratum fix
Bugzilla: 531469
RH-Acked-by: Prarit Bhargava <prarit@redhat.com>
RH-Acked-by: Don Dutile <ddutile@redhat.com>

Resolves BZ 531469

This patch fixes an AMD erratum that results in an IO PAGE FAULT on
Magny-cours while  booting up RHEL5.x.   A recent BIOS enabled access
to the system management bits. Linux marks all enries in device table
as valid at driver attach time, but leaves
the SysMgt bits at zero, which is defines as target abort all system
management requests.   Those sysmgt bits are filled according to IVHD
only at the time when device driver makes memory bind request.
The problem is that SMBus that generates those system management
accesses never asks to bind memory (i.e. it never does regular DMA).
Therefore those bits are never set. This results in IO PAGE FAULT
messages at boot time on the M-C system.

AMD Erratum 63:
    IOMMU Blocks Writes in the HT System Management Address Range when
    SysMgt=0x1

    Write requests to the HT system management address range may be
    blocked by the IOMMU when the associated device table entry has the
    SysMgt field set to 01b (pass through writes) with V=1, TV=1 and IW=0.

    Potential Effect on System: Unpredictable system behaviour.

    This issue has only been observed in the simulation environment.
    Suggested Workaround Only set DTE.IW=1 for the south bridge ACPI
    controller as this is the only device in the system that should
    issue system management requests.

The function iommu_feature_disable is required on system shutdown to
disable the IOMMU but it is marked as __init.  This may result in a
panic if the memory is reused.  I have added this one line fix
to the erratum fix for the AMD IOMMU driver.

Testing:
Successfully tested on Dinar system, and confirmed that with patch
the issue is resolves.

Upstream: The patch will be submitted upstream for inclusion in
2.6.32.   The patch was literally developed yesterday.

diff --git a/arch/x86_64/kernel/amd_iommu.c b/arch/x86_64/kernel/amd_iommu.c
index 945fbd3..6e19734 100644
--- a/arch/x86_64/kernel/amd_iommu.c
+++ b/arch/x86_64/kernel/amd_iommu.c
@@ -969,6 +969,8 @@ static void __detach_device(struct protection_domain *domain, u16 devid)
 	amd_iommu_dev_table[devid].data[1] = 0;
 	amd_iommu_dev_table[devid].data[2] = 0;
 
+	amd_iommu_apply_erratum_63(devid);
+
 	/* decrease reference counter */
 	domain->dev_cnt -= 1;
 
diff --git a/arch/x86_64/kernel/amd_iommu_init.c b/arch/x86_64/kernel/amd_iommu_init.c
index 34dc114..043efd1 100644
--- a/arch/x86_64/kernel/amd_iommu_init.c
+++ b/arch/x86_64/kernel/amd_iommu_init.c
@@ -232,7 +232,7 @@ static void __init iommu_feature_enable(struct amd_iommu *iommu, u8 bit)
 	writel(ctrl, iommu->mmio_base + MMIO_CONTROL_OFFSET);
 }
 
-static void __init iommu_feature_disable(struct amd_iommu *iommu, u8 bit)
+static void iommu_feature_disable(struct amd_iommu *iommu, u8 bit)
 {
 	u32 ctrl;
 
@@ -476,6 +476,25 @@ static void set_dev_entry_bit(u16 devid, u8 bit)
 	amd_iommu_dev_table[devid].data[i] |= (1 << _bit);
 }
 
+static int get_dev_entry_bit(u16 devid, u8 bit)
+{
+	int i = (bit >> 5) & 0x07;
+	int _bit = bit & 0x1f;
+
+	return (amd_iommu_dev_table[devid].data[i] & (1 << _bit)) >> _bit;
+}
+
+void amd_iommu_apply_erratum_63(u16 devid)
+{
+	int sysmgt;
+
+	sysmgt = get_dev_entry_bit(devid, DEV_ENTRY_SYSMGT1) |
+		 (get_dev_entry_bit(devid, DEV_ENTRY_SYSMGT2) << 1);
+
+	if (sysmgt == 0x01)
+		set_dev_entry_bit(devid, DEV_ENTRY_IW);
+}
+
 /* Writes the specific IOMMU for a device into the rlookup table */
 static void __init set_iommu_for_device(struct amd_iommu *iommu, u16 devid)
 {
@@ -504,6 +523,8 @@ static void __init set_dev_entry_from_acpi(struct amd_iommu *iommu,
 	if (flags & ACPI_DEVFLAG_LINT1)
 		set_dev_entry_bit(devid, DEV_ENTRY_LINT1_PASS);
 
+	amd_iommu_apply_erratum_63(devid);
+
 	set_iommu_for_device(iommu, devid);
 }
 
diff --git a/include/asm-x86_64/amd_iommu.h b/include/asm-x86_64/amd_iommu.h
index 1e689c1..cbf6829 100644
--- a/include/asm-x86_64/amd_iommu.h
+++ b/include/asm-x86_64/amd_iommu.h
@@ -28,6 +28,7 @@ extern int amd_iommu_init_dma_ops(void);
 extern void amd_iommu_detect(void);
 extern irqreturn_t amd_iommu_int_handler(int irq, void *data,
 					 struct pt_regs *regs);
+extern void amd_iommu_apply_erratum_63(u16 devid);
 #else
 static inline int amd_iommu_init(void) { return -ENODEV; }
 static inline void amd_iommu_detect(void) { }