From: Chris Wright <chrisw@redhat.com>
Date: Sat, 12 Dec 2009 05:43:41 -0500
Subject: [pci] enable acs p2p upstream forwarding
Message-id: <20091212054341.GI12350@x200.localdomain>
Patchwork-id: 21910
O-Subject: [RHEL5.5 PATCH] PCI: enable ACS P2P Upstream Forwarding
Bugzilla: 518305
RH-Acked-by: Don Dutile <ddutile@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=518305

This is a backport of the following commits pending in the upstream
pci tree:

http://git.kernel.org/?p=linux/kernel/git/jbarnes/pci-2.6.git;a=commitdiff;h=5d990b627537e59a3a2f039ff588a4750e9c1a6a
http://git.kernel.org/?p=linux/kernel/git/jbarnes/pci-2.6.git;a=commitdiff;h=df0e97c6f1f2fdca686036998fe816cefd8e27d7
http://git.kernel.org/?p=linux/kernel/git/jbarnes/pci-2.6.git;a=commitdiff;h=ae21ee65e8bc228416bbcc8a1da01c56a847a60c

It's mostly a straight forward backport.  Only noticeable divergence
from upstream is the Xen bit.  In RHEL 5 we request ACS to be enabled
from drivers/xen/core/pci.c when it hooks into the PCI layer for dom0.

This is only relevant for x86_64 bare metal and 32 and 64-bit x86 Xen
dom0, basically environments where we have an IOMMU and expect to do PCI
device assignment.  I have tested on x86_64 bare metal (AMD and Intel,
w/ and w/out IOMMU) and 32 and 64-bit Xen dom0 (needs kernel cmdline arg
of pci_pt_e820_access=on just like SR-IOV does for MMCONFIG PCIe access).

ACS is enabled on x86_64 w/ AMD or Intel IOMMU enabled or in 32 and
64-bit x86 Xen dom0, and shows up like this (lspci diff):

-               ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
+               ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-

Brew build:

https://brewweb.devel.redhat.com/taskinfo?taskID=2148141

Here are the relevant upstream commit logs:

    commit 5d990b627537e59a3a2f039ff588a4750e9c1a6a
    Author: Chris Wright <chrisw@sous-sol.org>
    Date:   Fri Dec 4 12:15:21 2009 -0800

    PCI: add pci_request_acs

    Commit ae21ee65e8bc228416bbcc8a1da01c56a847a60c "PCI: acs p2p upsteram
    forwarding enabling" doesn't actually enable ACS.

    Add a function to pci core to allow an IOMMU to request that ACS
    be enabled.  The existing mechanism of using iommu_found() in the pci
    core to know when ACS should be enabled doesn't actually work due to
    initialization order;  iommu has only been detected not initialized.

    Have Intel and AMD IOMMUs request ACS, and Xen does as well during early
    init of dom0.

    Cc: Allen Kay <allen.m.kay@intel.com>
    Cc: David Woodhouse <dwmw2@infradead.org>
    Cc: Jeremy Fitzhardinge <jeremy@goop.org>
    Cc: Joerg Roedel <joerg.roedel@amd.com>
    Signed-off-by: Chris Wright <chrisw@sous-sol.org>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

    commit df0e97c6f1f2fdca686036998fe816cefd8e27d7
    Author: Allen Kay <allen.m.kay@intel.com>
    Date:   Wed Oct 7 10:27:51 2009 -0700

    PCI: add xen dom0 checking before ACS initialization

    This patch is predicated on Jeremy's patch in include/xen/xen.h.  It'll
    prevent ACS init unless the platform has both an IOMMU and we're running
    as dom0.

    Signed-off-by: Allen Kay <allen.m.kay@intel.com>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

    commit ae21ee65e8bc228416bbcc8a1da01c56a847a60c
    Author: Allen Kay <allen.m.kay@intel.com>
    Date:   Wed Oct 7 10:27:17 2009 -0700

    PCI: acs p2p upsteram forwarding enabling

    Note: dom0 checking in v4 has been separated out into 2/2.

    This patch enables P2P upstream forwarding in ACS capable PCIe switches.
    It solves two potential problems in virtualization environment where a PCIe
    device is assigned to a guest domain using a HW iommu such as VT-d:

    1) Unintentional failure caused by guest physical address programmed
       into the device's DMA that happens to match the memory address range
       of other downstream ports in the same PCIe switch.  This causes the PCI
       transaction to go to the matching downstream port instead of go to the
       root complex to get translated by VT-d as it should be.

    2) Malicious guest software intentionally attacks another downstream
       PCIe device by programming the DMA address into the assigned device
       that matches memory address range of the downstream PCIe port.

    We are in process of implementing device filtering software in KVM/XEN
    management software to allow device assignment of PCIe devices behind a PCIe
    switch only if it has ACS capability and with the P2P upstream forwarding bits
    enabled.  This patch is intended to work for both KVM and Xen environments.

    Signed-off-by: Allen Kay <allen.m.kay@intel.com>
    Reviewed-by: Mathew Wilcox <willy@linux.intel.com>
    Reviewed-by: Chris Wright <chris@sous-sol.org>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

Signed-off-by: Chris Wright <chrisw@redhat.com>

diff --git a/arch/x86_64/kernel/amd_iommu_init.c b/arch/x86_64/kernel/amd_iommu_init.c
index 043efd1..595ac5c 100644
--- a/arch/x86_64/kernel/amd_iommu_init.c
+++ b/arch/x86_64/kernel/amd_iommu_init.c
@@ -1181,6 +1181,8 @@ void __init amd_iommu_detect(void)
 		iommu_aperture_disabled = 1;
 		iommu_aperture = 0;
 #endif
+		/* Make sure ACS will be enabled */
+		pci_request_acs();
 	}
 }
 
diff --git a/drivers/pci/dmar.c b/drivers/pci/dmar.c
index 77073be..c1bf2db 100644
--- a/drivers/pci/dmar.c
+++ b/drivers/pci/dmar.c
@@ -661,8 +661,11 @@ void __init detect_intel_iommu(void)
 #endif
 #ifdef CONFIG_DMAR
 		if (ret && !no_iommu && !iommu_detected && !swiotlb &&
-		    !dmar_disabled)
+		    !dmar_disabled) {
 			iommu_detected = 1;
+			/* Make sure ACS will be enabled */
+			pci_request_acs();
+		}
 #endif
 	}
 	dmar_tbl = NULL;
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 3ced3f8..6ec14fc 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1015,6 +1015,54 @@ void pci_enable_ari(struct pci_dev *dev)
 	bridge->ari_enabled = 1;
 }
 
+static int pci_acs_enable;
+
+/**
+ * pci_request_acs - ask for ACS to be enabled if supported
+ */
+void pci_request_acs(void)
+{
+	pci_acs_enable = 1;
+}
+
+/**
+ * pci_enable_acs - enable ACS if hardware support it
+ * @dev: the PCI device
+ */
+void pci_enable_acs(struct pci_dev *dev)
+{
+	int pos;
+	u16 cap;
+	u16 ctrl;
+
+	if (!pci_acs_enable)
+		return;
+
+	if (!dev->is_pcie)
+		return;
+
+	pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS);
+	if (!pos)
+		return;
+
+	pci_read_config_word(dev, pos + PCI_ACS_CAP, &cap);
+	pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl);
+
+	/* Source Validation */
+	ctrl |= (cap & PCI_ACS_SV);
+
+	/* P2P Request Redirect */
+	ctrl |= (cap & PCI_ACS_RR);
+
+	/* P2P Completion Redirect */
+	ctrl |= (cap & PCI_ACS_CR);
+
+	/* Upstream Forwarding */
+	ctrl |= (cap & PCI_ACS_UF);
+
+	pci_write_config_word(dev, pos + PCI_ACS_CTRL, ctrl);
+}
+
 int
 pci_get_interrupt_pin(struct pci_dev *dev, struct pci_dev **bridge)
 {
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index b594290..0bf784e 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -209,3 +209,5 @@ static inline int pci_ats_enabled(struct pci_dev *dev)
 }
 #endif /* CONFIG_PCI_IOV */
 struct pci_dev *pci_find_upstream_pcie_bridge(struct pci_dev *pdev);
+
+extern void pci_enable_acs(struct pci_dev *dev);
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 5588fb2..c61bbff 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -933,6 +933,9 @@ void __devinit pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 	/* Single Root I/O Virtualization */
 	pci_iov_init(dev);
 
+	/* Enable ACS P2P upstream forwarding */
+	pci_enable_acs(dev);
+
 	/*
 	 * Add the device to our list of discovered devices
 	 * and the bus list for fixup functions, etc.
diff --git a/drivers/xen/core/pci.c b/drivers/xen/core/pci.c
index 0c3b241..9cea0a5 100644
--- a/drivers/xen/core/pci.c
+++ b/drivers/xen/core/pci.c
@@ -55,6 +55,11 @@ static int __init hook_pci_bus(void)
 	pci_bus_remove = pci_bus_type.remove;
 	pci_bus_type.remove = pci_bus_remove_wrapper;
 
+#ifndef __ia64__
+	/* Make sure ACS will be enabled */
+	pci_request_acs();
+#endif
+
 	return 0;
 }
 
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 6ee67f9..e72580f 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -964,5 +964,7 @@ static inline void pci_disable_sriov(struct pci_dev *dev)
 }
 #endif
 
+void pci_request_acs(void);
+
 #endif /* __KERNEL__ */
 #endif /* LINUX_PCI_H */
diff --git a/include/linux/pci_regs.h b/include/linux/pci_regs.h
index 3742fcb..2092b3e 100644
--- a/include/linux/pci_regs.h
+++ b/include/linux/pci_regs.h
@@ -430,6 +430,7 @@
 #define PCI_EXT_CAP_ID_VC	2
 #define PCI_EXT_CAP_ID_DSN	3
 #define PCI_EXT_CAP_ID_PWR	4
+#define PCI_EXT_CAP_ID_ACS	13
 #define PCI_EXT_CAP_ID_ARI	14
 #define PCI_EXT_CAP_ID_ATS	15
 #define PCI_EXT_CAP_ID_SRIOV	16
@@ -590,4 +591,16 @@
 #define  PCI_SRIOV_VFM_MO	0x2	/* Active.MigrateOut */
 #define  PCI_SRIOV_VFM_AV	0x3	/* Active.Available */
 
+/* Access Control Service */
+#define PCI_ACS_CAP		0x04	/* ACS Capability Register */
+#define  PCI_ACS_SV		0x01	/* Source Validation */
+#define  PCI_ACS_TB		0x02	/* Translation Blocking */
+#define  PCI_ACS_RR		0x04	/* P2P Request Redirect */
+#define  PCI_ACS_CR		0x08	/* P2P Completion Redirect */
+#define  PCI_ACS_UF		0x10	/* Upstream Forwarding */
+#define  PCI_ACS_EC		0x20	/* P2P Egress Control */
+#define  PCI_ACS_DT		0x40	/* Direct Translated P2P */
+#define PCI_ACS_CTRL		0x06	/* ACS Control Register */
+#define PCI_ACS_EGRESS_CTL_V	0x08	/* ACS Egress Control Vector */
+
 #endif /* LINUX_PCI_REGS_H */