Skip to content

Commit 28b4780

Browse files
torvaldstorvalds
committed
Merge tag 'iommu-updates-v4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu
Pull IOMMU updates from Joerg Roedel: - code optimizations for the Intel VT-d driver - ability to switch off a previously enabled Intel IOMMU - support for 'struct iommu_device' for OMAP, Rockchip and Mediatek IOMMUs - header optimizations for IOMMU core code headers and a few fixes that became necessary in other parts of the kernel because of that - ACPI/IORT updates and fixes - Exynos IOMMU optimizations - updates for the IOMMU dma-api code to bring it closer to use per-cpu iova caches - new command-line option to set default domain type allocated by the iommu core code - another command line option to allow the Intel IOMMU switched off in a tboot environment - ARM/SMMU: TLB sync optimisations for SMMUv2, Support for using an IDENTITY domain in conjunction with DMA ops, Support for SMR masking, Support for 16-bit ASIDs (was previously broken) - various other small fixes and improvements * tag 'iommu-updates-v4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (63 commits) soc/qbman: Move dma-mapping.h include to qman_priv.h soc/qbman: Fix implicit header dependency now causing build fails iommu: Remove trace-events include from iommu.h iommu: Remove pci.h include from trace/events/iommu.h arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops() ACPI/IORT: Fix CONFIG_IOMMU_API dependency iommu/vt-d: Don't print the failure message when booting non-kdump kernel iommu: Move report_iommu_fault() to iommu.c iommu: Include device.h in iommu.h x86, iommu/vt-d: Add an option to disable Intel IOMMU force on iommu/arm-smmu: Return IOVA in iova_to_phys when SMMU is bypassed iommu/arm-smmu: Correct sid to mask iommu/amd: Fix incorrect error handling in amd_iommu_bind_pasid() iommu: Make iommu_bus_notifier return NOTIFY_DONE rather than error code omap3isp: Remove iommu_group related code iommu/omap: Add iommu-group support iommu/omap: Make use of 'struct iommu_device' iommu/omap: Store iommu_dev pointer in arch_data iommu/omap: Move data structures to omap-iommu.h iommu/omap: Drop legacy-style device support ...
2 parents 4a1e31c + 2c0248d commit 28b4780

48 files changed

Lines changed: 1188 additions & 765 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Documentation/admin-guide/kernel-parameters.txt‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1578,6 +1578,15 @@
15781578
extended tables themselves, and also PASID support. With
15791579
this option set, extended tables will not be used even
15801580
on hardware which claims to support them.
1581+
tboot_noforce [Default Off]
1582+
Do not force the Intel IOMMU enabled under tboot.
1583+
By default, tboot will force Intel IOMMU on, which
1584+
could harm performance of some high-throughput
1585+
devices like 40GBit network cards, even if identity
1586+
mapping is enabled.
1587+
Note that using this option lowers the security
1588+
provided by tboot because it makes the system
1589+
vulnerable to DMA attacks.
15811590

15821591
intel_idle.max_cstate= [KNL,HW,ACPI,X86]
15831592
0 disables intel_idle and fall back on acpi_idle.
@@ -1644,6 +1653,12 @@
16441653
nobypass [PPC/POWERNV]
16451654
Disable IOMMU bypass, using IOMMU for PCI devices.
16461655

1656+
iommu.passthrough=
1657+
[ARM64] Configure DMA to bypass the IOMMU by default.
1658+
Format: { "0" | "1" }
1659+
0 - Use IOMMU translation for DMA.
1660+
1 - Bypass the IOMMU for DMA.
1661+
unset - Use IOMMU translation for DMA.
16471662

16481663
io7= [HW] IO7 for Marvel based alpha systems
16491664
See comment before marvel_specify_io7 in

‎Documentation/devicetree/bindings/iommu/arm,smmu.txt‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,17 @@ conditions.
6060
aliases of secure registers have to be used during
6161
SMMU configuration.
6262

63+
- stream-match-mask : For SMMUs supporting stream matching and using
64+
#iommu-cells = <1>, specifies a mask of bits to ignore
65+
when matching stream IDs (e.g. this may be programmed
66+
into the SMRn.MASK field of every stream match register
67+
used). For cases where it is desirable to ignore some
68+
portion of every Stream ID (e.g. for certain MMU-500
69+
configurations given globally unique input IDs). This
70+
property is not valid for SMMUs using stream indexing,
71+
or using stream matching with #iommu-cells = <2>, and
72+
may be ignored if present in such cases.
73+
6374
** Deprecated properties:
6475

6576
- mmu-masters (deprecated in favour of the generic "iommus" binding) :
@@ -109,3 +120,20 @@ conditions.
109120
master3 {
110121
iommus = <&smmu2 1 0x30>;
111122
};
123+
124+
125+
/* ARM MMU-500 with 10-bit stream ID input configuration */
126+
smmu3: iommu {
127+
compatible = "arm,mmu-500", "arm,smmu-v2";
128+
...
129+
#iommu-cells = <1>;
130+
/* always ignore appended 5-bit TBU number */
131+
stream-match-mask = 0x7c00;
132+
};
133+
134+
bus {
135+
/* bus whose child devices emit one unique 10-bit stream
136+
ID each, but may master through multiple SMMU TBUs */
137+
iommu-map = <0 &smmu3 0 0x400>;
138+
...
139+
};

‎arch/arm/mm/dma-mapping.c‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2408,6 +2408,15 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
24082408
const struct dma_map_ops *dma_ops;
24092409

24102410
dev->archdata.dma_coherent = coherent;
2411+
2412+
/*
2413+
* Don't override the dma_ops if they have already been set. Ideally
2414+
* this should be the only location where dma_ops are set, remove this
2415+
* check when all other callers of set_dma_ops will have disappeared.
2416+
*/
2417+
if (dev->dma_ops)
2418+
return;
2419+
24112420
if (arm_setup_iommu_dma_ops(dev, dma_base, size, iommu))
24122421
dma_ops = arm_get_iommu_dma_map_ops(coherent);
24132422
else

‎arch/arm64/mm/dma-mapping.c‎

Lines changed: 19 additions & 124 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
#include <linux/dma-contiguous.h>
2929
#include <linux/vmalloc.h>
3030
#include <linux/swiotlb.h>
31+
#include <linux/pci.h>
3132

3233
#include <asm/cacheflush.h>
3334

@@ -879,34 +880,26 @@ static const struct dma_map_ops iommu_dma_ops = {
879880
.mapping_error = iommu_dma_mapping_error,
880881
};
881882

882-
/*
883-
* TODO: Right now __iommu_setup_dma_ops() gets called too early to do
884-
* everything it needs to - the device is only partially created and the
885-
* IOMMU driver hasn't seen it yet, so it can't have a group. Thus we
886-
* need this delayed attachment dance. Once IOMMU probe ordering is sorted
887-
* to move the arch_setup_dma_ops() call later, all the notifier bits below
888-
* become unnecessary, and will go away.
889-
*/
890-
struct iommu_dma_notifier_data {
891-
struct list_head list;
892-
struct device *dev;
893-
const struct iommu_ops *ops;
894-
u64 dma_base;
895-
u64 size;
896-
};
897-
static LIST_HEAD(iommu_dma_masters);
898-
static DEFINE_MUTEX(iommu_dma_notifier_lock);
883+
static int __init __iommu_dma_init(void)
884+
{
885+
return iommu_dma_init();
886+
}
887+
arch_initcall(__iommu_dma_init);
899888

900-
static bool do_iommu_attach(struct device *dev, const struct iommu_ops *ops,
901-
u64 dma_base, u64 size)
889+
static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
890+
const struct iommu_ops *ops)
902891
{
903-
struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
892+
struct iommu_domain *domain;
893+
894+
if (!ops)
895+
return;
904896

905897
/*
906-
* If the IOMMU driver has the DMA domain support that we require,
907-
* then the IOMMU core will have already configured a group for this
908-
* device, and allocated the default domain for that group.
898+
* The IOMMU core code allocates the default DMA domain, which the
899+
* underlying IOMMU driver needs to support via the dma-iommu layer.
909900
*/
901+
domain = iommu_get_domain_for_dev(dev);
902+
910903
if (!domain)
911904
goto out_err;
912905

@@ -917,109 +910,11 @@ static bool do_iommu_attach(struct device *dev, const struct iommu_ops *ops,
917910
dev->dma_ops = &iommu_dma_ops;
918911
}
919912

920-
return true;
913+
return;
914+
921915
out_err:
922-
pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
916+
pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
923917
dev_name(dev));
924-
return false;
925-
}
926-
927-
static void queue_iommu_attach(struct device *dev, const struct iommu_ops *ops,
928-
u64 dma_base, u64 size)
929-
{
930-
struct iommu_dma_notifier_data *iommudata;
931-
932-
iommudata = kzalloc(sizeof(*iommudata), GFP_KERNEL);
933-
if (!iommudata)
934-
return;
935-
936-
iommudata->dev = dev;
937-
iommudata->ops = ops;
938-
iommudata->dma_base = dma_base;
939-
iommudata->size = size;
940-
941-
mutex_lock(&iommu_dma_notifier_lock);
942-
list_add(&iommudata->list, &iommu_dma_masters);
943-
mutex_unlock(&iommu_dma_notifier_lock);
944-
}
945-
946-
static int __iommu_attach_notifier(struct notifier_block *nb,
947-
unsigned long action, void *data)
948-
{
949-
struct iommu_dma_notifier_data *master, *tmp;
950-
951-
if (action != BUS_NOTIFY_BIND_DRIVER)
952-
return 0;
953-
954-
mutex_lock(&iommu_dma_notifier_lock);
955-
list_for_each_entry_safe(master, tmp, &iommu_dma_masters, list) {
956-
if (data == master->dev && do_iommu_attach(master->dev,
957-
master->ops, master->dma_base, master->size)) {
958-
list_del(&master->list);
959-
kfree(master);
960-
break;
961-
}
962-
}
963-
mutex_unlock(&iommu_dma_notifier_lock);
964-
return 0;
965-
}
966-
967-
static int __init register_iommu_dma_ops_notifier(struct bus_type *bus)
968-
{
969-
struct notifier_block *nb = kzalloc(sizeof(*nb), GFP_KERNEL);
970-
int ret;
971-
972-
if (!nb)
973-
return -ENOMEM;
974-
975-
nb->notifier_call = __iommu_attach_notifier;
976-
977-
ret = bus_register_notifier(bus, nb);
978-
if (ret) {
979-
pr_warn("Failed to register DMA domain notifier; IOMMU DMA ops unavailable on bus '%s'\n",
980-
bus->name);
981-
kfree(nb);
982-
}
983-
return ret;
984-
}
985-
986-
static int __init __iommu_dma_init(void)
987-
{
988-
int ret;
989-
990-
ret = iommu_dma_init();
991-
if (!ret)
992-
ret = register_iommu_dma_ops_notifier(&platform_bus_type);
993-
if (!ret)
994-
ret = register_iommu_dma_ops_notifier(&amba_bustype);
995-
#ifdef CONFIG_PCI
996-
if (!ret)
997-
ret = register_iommu_dma_ops_notifier(&pci_bus_type);
998-
#endif
999-
return ret;
1000-
}
1001-
arch_initcall(__iommu_dma_init);
1002-
1003-
static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
1004-
const struct iommu_ops *ops)
1005-
{
1006-
struct iommu_group *group;
1007-
1008-
if (!ops)
1009-
return;
1010-
/*
1011-
* TODO: As a concession to the future, we're ready to handle being
1012-
* called both early and late (i.e. after bus_add_device). Once all
1013-
* the platform bus code is reworked to call us late and the notifier
1014-
* junk above goes away, move the body of do_iommu_attach here.
1015-
*/
1016-
group = iommu_group_get(dev);
1017-
if (group) {
1018-
do_iommu_attach(dev, ops, dma_base, size);
1019-
iommu_group_put(group);
1020-
} else {
1021-
queue_iommu_attach(dev, ops, dma_base, size);
1022-
}
1023918
}
1024919

1025920
void arch_teardown_dma_ops(struct device *dev)

‎arch/x86/kernel/tboot.c‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -514,6 +514,9 @@ int tboot_force_iommu(void)
514514
if (!tboot_enabled())
515515
return 0;
516516

517+
if (!intel_iommu_tboot_noforce)
518+
return 1;
519+
517520
if (no_iommu || swiotlb || dmar_disabled)
518521
pr_warning("Forcing Intel-IOMMU to enabled\n");
519522

‎drivers/acpi/arm64/iort.c‎

Lines changed: 69 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -618,6 +618,46 @@ static int arm_smmu_iort_xlate(struct device *dev, u32 streamid,
618618
return ret;
619619
}
620620

621+
static inline bool iort_iommu_driver_enabled(u8 type)
622+
{
623+
switch (type) {
624+
case ACPI_IORT_NODE_SMMU_V3:
625+
return IS_BUILTIN(CONFIG_ARM_SMMU_V3);
626+
case ACPI_IORT_NODE_SMMU:
627+
return IS_BUILTIN(CONFIG_ARM_SMMU);
628+
default:
629+
pr_warn("IORT node type %u does not describe an SMMU\n", type);
630+
return false;
631+
}
632+
}
633+
634+
#ifdef CONFIG_IOMMU_API
635+
static inline
636+
const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
637+
{
638+
return (fwspec && fwspec->ops) ? fwspec->ops : NULL;
639+
}
640+
641+
static inline
642+
int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
643+
{
644+
int err = 0;
645+
646+
if (!IS_ERR_OR_NULL(ops) && ops->add_device && dev->bus &&
647+
!dev->iommu_group)
648+
err = ops->add_device(dev);
649+
650+
return err;
651+
}
652+
#else
653+
static inline
654+
const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
655+
{ return NULL; }
656+
static inline
657+
int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
658+
{ return 0; }
659+
#endif
660+
621661
static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
622662
struct acpi_iort_node *node,
623663
u32 streamid)
@@ -626,14 +666,31 @@ static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
626666
int ret = -ENODEV;
627667
struct fwnode_handle *iort_fwnode;
628668

669+
/*
670+
* If we already translated the fwspec there
671+
* is nothing left to do, return the iommu_ops.
672+
*/
673+
ops = iort_fwspec_iommu_ops(dev->iommu_fwspec);
674+
if (ops)
675+
return ops;
676+
629677
if (node) {
630678
iort_fwnode = iort_get_fwnode(node);
631679
if (!iort_fwnode)
632680
return NULL;
633681

634682
ops = iommu_ops_from_fwnode(iort_fwnode);
683+
/*
684+
* If the ops look-up fails, this means that either
685+
* the SMMU drivers have not been probed yet or that
686+
* the SMMU drivers are not built in the kernel;
687+
* Depending on whether the SMMU drivers are built-in
688+
* in the kernel or not, defer the IOMMU configuration
689+
* or just abort it.
690+
*/
635691
if (!ops)
636-
return NULL;
692+
return iort_iommu_driver_enabled(node->type) ?
693+
ERR_PTR(-EPROBE_DEFER) : NULL;
637694

638695
ret = arm_smmu_iort_xlate(dev, streamid, iort_fwnode, ops);
639696
}
@@ -676,6 +733,7 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
676733
struct acpi_iort_node *node, *parent;
677734
const struct iommu_ops *ops = NULL;
678735
u32 streamid = 0;
736+
int err;
679737

680738
if (dev_is_pci(dev)) {
681739
struct pci_bus *bus = to_pci_dev(dev)->bus;
@@ -707,13 +765,23 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
707765

708766
while (parent) {
709767
ops = iort_iommu_xlate(dev, parent, streamid);
768+
if (IS_ERR_OR_NULL(ops))
769+
return ops;
710770

711771
parent = iort_node_map_platform_id(node, &streamid,
712772
IORT_IOMMU_TYPE,
713773
i++);
714774
}
715775
}
716776

777+
/*
778+
* If we have reason to believe the IOMMU driver missed the initial
779+
* add_device callback for dev, replay it to get things in order.
780+
*/
781+
err = iort_add_device_replay(ops, dev);
782+
if (err)
783+
ops = ERR_PTR(err);
784+
717785
return ops;
718786
}
719787

@@ -1052,6 +1120,4 @@ void __init acpi_iort_init(void)
10521120
}
10531121

10541122
iort_init_platform_devices();
1055-
1056-
acpi_probe_device_table(iort);
10571123
}

0 commit comments

Comments
 (0)