From: Bhavna Sarathy <bnagendr@redhat.com>
Date: Fri, 12 Jun 2009 11:27:24 -0400
Subject: [x86_64] AMD IOMMU: fix GLX issue in bare metal
Message-id: 20090612152832.14598.67622.sendpatchset@localhost.localdomain
O-Subject: [RHEL5.4 PATCH] V2 Fix GLX issue in bare metal AMD IOMMU
Bugzilla: 504010
RH-Acked-by: Don Dutile <ddutile@redhat.com>
RH-Acked-by: Chris Wright <chrisw@redhat.com>
RH-Acked-by: Rik van Riel <riel@redhat.com>

Resolves BZ504010

This patch implements a workaround into the AMD IOMMU code to make
all graphics cards work when IOMMU is enabled (as in 5.4).  A similar
workaround exists for Intel IOMMU hardware.

Issues the patch fixes:
The graphics issue (not using DMA API for GPU accessible memory) is
not specific to one adapter, and it occurs with open source graphics
drivers as well. The older graphics cards that work in RHEL5.3 will
just hang, and the newer ones have a memory  limitation.  i.e. default
64M per-PCI device limitation imposed by the current IOMMU driver.
Graphics devices need much more memory than this limitation.

Brew build:
http://brewweb.devel.redhat.com/brew/taskinfo?taskID=1830208

The patch has been tested with proprietary graphics cards such as
rv610 and the patch also fixes the issue with open source graphics
drivers. Also, note that this is a regression as the graphics support
worked in 5.3 (where AMD IOMMU was disabled by default).

The issue was first fixed in RHEL and then same fix submitted upstream.
This submission link (below) is bit botched due to mailer difficulties,
I have resubmitted using send mail script (I don't have a new link in
the IOMMU archives I check just yet)

https://lists.linux-foundation.org/pipermail/iommu/2009-June/001553.html

DonZ: dependency note: Please apply this patch after the corner cases patch
set (5 patches), and the kdump patch in that order, to avoid patch application
issues, if any.

Please ACK for RHEL5.4 beta.

diff --git a/arch/x86_64/kernel/amd_iommu.c b/arch/x86_64/kernel/amd_iommu.c
index e17b779..945fbd3 100644
--- a/arch/x86_64/kernel/amd_iommu.c
+++ b/arch/x86_64/kernel/amd_iommu.c
@@ -32,6 +32,8 @@
 
 #define EXIT_LOOP_COUNT 10000000
 
+#define IS_GFX_DEVICE(pdev) ((pdev->class >> 16) == PCI_BASE_CLASS_DISPLAY)
+
 static DEFINE_RWLOCK(amd_iommu_devtable_lock);
 
 /* A list of preallocated protection domains */
@@ -1630,6 +1632,30 @@ static int amd_iommu_dma_supported(struct device *dev, u64 mask)
 	return 1;
 }
 
+static void remove_pdev_from_iommu(struct pci_dev *pdev)
+{
+	u16 devid = calc_devid(pdev->bus->number, pdev->devfn);
+
+	amd_iommu_pd_table[devid] = NULL;
+	amd_iommu_rlookup_table[devid] = NULL;
+
+	amd_iommu_dev_table[devid].data[0] = 0;
+}
+
+static void init_gfx_workaround(void)
+{
+	struct pci_dev *dev = NULL;
+
+	while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
+		if (!IS_GFX_DEVICE(dev))
+			continue;
+		printk(KERN_INFO "AMD IOMMU: enabling GFX workaround for "
+				 "PCI device ");
+		print_devid(calc_devid(dev->bus->number, dev->devfn), 1);
+		remove_pdev_from_iommu(dev);
+	}
+}
+
 /*
  * The function for pre-allocating protection domains.
  *
@@ -1699,6 +1725,9 @@ int __init amd_iommu_init_dma_ops(void)
 			goto free_domains;
 	}
 
+	/* Initialize GFX workaround */
+	init_gfx_workaround();
+
 	/*
 	 * If device isolation is enabled, pre-allocate the protection
 	 * domains for each device.