From: Bhavna Sarathy <bnagendr@redhat.com>
Date: Tue, 31 Aug 2010 17:01:30 -0400
Subject: [misc] amd_iommu: change default to passthrough mode
Message-id: <20100831170728.9076.35540.sendpatchset@localhost.localdomain>
Patchwork-id: 27972
O-Subject: [RHEL5.6 PATCH] Change default to passthrough mode
Bugzilla: 628018
RH-Acked-by: Chris Wright <chrisw@redhat.com>
RH-Acked-by: Jarod Wilson <jarod@redhat.com>

Resolves BZ  628018

This patch makes the passthrough mode (iommu=pt mode) the
default for the AMD IOMMU driver. Additionally the kernel
parameter amd_iommu=isolate|share is used to disable
passthrough mode again if requested by the user.

The changes for this need to be done in the amd_iommu_detect()
function, which runs on every hardware, because there is the
first check for the passthrough mode.  Therefore this patch
also moves some code in this function to the part where the
IOMMU is surely detected to not affect systems with other
types of IOMMUs.

These changes were discussed in depth with Chris Wright, Chris
Lalancette during RHEL6 and the same patch is being added into
RHEL5.

Jarod, please apply this after applying the pasthrough mode patch
for BZ 561127.   This patch applies cleanly to 2.6.18-212.el5, and
this has been our development tree for the set of IOMMU patches.

Please note that the kdump issue has been fixed and will be submitted
using BZ 627663.

Testing:
[ PASS ] IOMMU passthrough mode is set by default, SWIOTLB is turned on
[ PASS ] KVM PCI device passthrough with IOMMU passthrough mode works
[ PASS ] Force IOMMU to be enabled without device isolation
[ PASS ] Force IOMMU to be enabled with device isolation

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 81bd2b3..2814c1d 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -232,13 +232,16 @@ running once the system is up.
 			Possible values are:
 			isolate - enable device isolation (each device, as far
 				  as possible, will get its own protection
-				  domain) [default]
+				  domain). This parameter also forces the
+				  AMD IOMMU driver out of passthrough mode.
 			share - put every device behind one IOMMU into the
-				same protection domain
+				same protection domain. This parameter also
+				forces the AMD IOMMU driver to not use
+				passthrough mode.
 			fullflush - enable flushing of IO/TLB entries when
 				    they are unmapped. Otherwise they are
 				    flushed before they will be reused, which
-				    is a lot of faster
+				    is a lot faster
 
 	amd_iommu_size= [HW,X86-64]
 			Define the size of the aperture for the AMD IOMMU
diff --git a/arch/x86_64/kernel/amd_iommu_init.c b/arch/x86_64/kernel/amd_iommu_init.c
index fb40704..9ddfa5d 100644
--- a/arch/x86_64/kernel/amd_iommu_init.c
+++ b/arch/x86_64/kernel/amd_iommu_init.c
@@ -122,6 +122,9 @@ LIST_HEAD(amd_iommu_unity_map);		/* a list of required unity mappings
 unsigned amd_iommu_aperture_order = 26; /* size of aperture in power of 2 */
 bool amd_iommu_isolate = true;		/* if true, device isolation is
 					   enabled */
+static bool amd_iommu_force_nopt;	/* force isolation (no-pt) mode
+					   of iommu */
+
 bool amd_iommu_unmap_flush;		/* if true, flush on every unmap */
 
 LIST_HEAD(amd_iommu_list);		/* list of all AMD IOMMUs in the
@@ -1185,9 +1188,6 @@ void __init amd_iommu_detect(void)
 	if (swiotlb || no_iommu || (iommu_detected && !iommu_aperture))
 		return;
 
-	if (iommu_pass_through)
-		swiotlb = 1;
-
 	if (acpi_table_parse(ACPI_IVRS, early_amd_iommu_detect) != 0) {
 		iommu_detected = 1;
 		amd_iommu_detected = 1;
@@ -1197,6 +1197,17 @@ void __init amd_iommu_detect(void)
 #endif
 		/* Make sure ACS will be enabled */
 		pci_request_acs();
+	
+		/*
+		 * The AMD IOMMU driver forces the passthrough mode by default
+		 * for performance reasons. Since we need a way to switch back
+		 * to the isolation mode we reuse amd_iommu={isolate|share} for
+		 * that.
+		 */
+		iommu_pass_through = amd_iommu_force_nopt ? 0 : 1;
+	
+		if (iommu_pass_through)
+			swiotlb = 1;
 	}
 }
 
@@ -1210,10 +1221,14 @@ void __init amd_iommu_detect(void)
 static int __init parse_amd_iommu_options(char *str)
 {
 	for (; *str; ++str) {
-		if (strncmp(str, "isolate", 7) == 0)
+		if (strncmp(str, "isolate", 7) == 0) {
 			amd_iommu_isolate = true;
-		if (strncmp(str, "share", 5) == 0)
+			amd_iommu_force_nopt = true;
+		}
+		if (strncmp(str, "share", 5) == 0) {
 			amd_iommu_isolate = false;
+			amd_iommu_force_nopt = true;
+		}
 		if (strncmp(str, "fullflush", 9) == 0)
 			amd_iommu_unmap_flush = true;
 	}