#asahi · 2021-06-07 15:11
arndbloom: it's extremely workload specific, and a number of other things also go wrong: anything that small bits of data but requires page alignment can explode. Other things I'm sure cause problems are swapspace handling, and network buffers#asahi · 2021-06-07 15:10
arndsven: ok, good#asahi · 2021-06-07 15:09
arndbloom: mostly a waste of memory, e.g. when dealing with small files your page cache often blows up to 5x the memory usage#asahi · 2021-06-07 15:08
arndmarcan: I wonder if it makes sense to address this as a general security issue then, and start by making the dma-iommu code use bounce buffers as an option, and make the sub-page protection as a way to make it faster#asahi · 2021-06-07 15:05
arndsven: have you found out if dart can do both cache-coherent dma mappings and non-coherent ones? Is bypass mode still coherent?#asahi · 2021-06-07 15:00
arnddo if we find a way change the iommu ->map() callback to pass the location within the page and do sub-page protection as well as fix the dma-iommu code to no longer assume iopagesize<=page_suze, I think that should allow safely using wireless and thunderbolt with 4KB pages, it would only prevent you from using device assignment into VM guests and user space like DPDK#asahi · 2021-06-07 14:54
arndok, perfect. So the dart would actually end up being more secure than most others for the streaming API ;-)#asahi · 2021-06-07 14:53
arnd(that would be the minimum the kernel might pass)#asahi · 2021-06-07 14:53
arndsven: what are the parameters for sub-page protections? Can you e.g. map an arbitrary naturally aligned 64 byte range in a page?#asahi · 2021-06-07 14:52
arnddrivers using only the coherent interfaces (dma_alloc_coherent() etc), currently get full pages, so those are safe as long as the page size matches, or we change the common code to use io page size allocations#asahi · 2021-06-07 14:50
arndespecially on USB, the kernel often just passes buffers that are only a few bytes long#asahi · 2021-06-07 14:49
arndfor the streaming mappings, I think there is already a small security hole regardless of the iommu type and the page size, as dma_map_single() often gets passed small buffers that live in the same page as other (possibly sensitive) data#asahi · 2021-06-07 14:46
arndof the iommu driver#asahi · 2021-06-07 14:46
arndsven: right, that fixing the common iommu code to work with this should be possible by the time that distros actually want to ship installers to work on mac. That is still a going to be a while, and your solution is enough for an initial merge#asahi · 2021-06-07 14:45
arndyes, makes sense (on wifi and thunderbolt)#asahi · 2021-06-07 14:43
arndthough a more likely outcome is that a lot of users run microbenchmarks that show 64K pages being faster because of lower TLB pressure#asahi · 2021-06-07 14:42
arndI still hope that opening up the discussion again will at least make some users understand just how bad 64K pages are for general-purpose workloads#asahi · 2021-06-07 14:41
arndI honestly don't know why Ubuntu did it#asahi · 2021-06-07 14:40
arndI expect distros to fight that very hard, as it adds a lot of extra validation work to have more than one#asahi · 2021-06-07 14:39
arndMost developer boards still have only Cortex-A53 or Cortex-A72, which don't support 16KB pages at all#asahi · 2021-06-07 14:38
arndzero#asahi · 2021-06-07 14:36
arndthough both of those also run firmware that might be attacked in theory#asahi · 2021-06-07 14:35
arndI was only thinking of ethernet and pci-xhci#asahi · 2021-06-07 14:35
arndah, indeed#asahi · 2021-06-07 14:34
arndfor the on-board devices, I don't see much of a problem#asahi · 2021-06-07 14:33
arndok. I suppose bypass mode on the thunderbolt wouldn't make much of a difference in practice, since that is too dangerous for practical use#asahi · 2021-06-07 14:31
arndI see. Have you checked all the other DART instances to see what those support?#asahi · 2021-06-07 14:22
arndok. for all of the address space or only the first 4GB?#asahi · 2021-06-07 14:21
arndsven: did you manage to enable bypass mode on all iommu instances in the end, or do you program a linear map into them?#asahi · 2021-06-07 14:20
arndnice!#asahi · 2021-05-31 21:31
arndagraf: there was some cryptic error message I got when I tried to install m1n1 onto the thunderbolt drive, which I think was related.#asahi · 2021-05-31 21:29
arndi.e. more than one "internal" drive#asahi · 2021-05-31 21:29
arndUSB drives can only be booted using recovery mode by long-pressing the power button, but I did not try that at all. The method I used was with a thunderbolt drive, because some part of the boot loader apparently treats that as an internal drive. My best guess is that it breaks down after that, since this is not really a supported configuration#asahi · 2021-05-31 21:27
arndI think there is also a difference between booting through recovery mode and booting through the normal "pick default OS" (whichever that is called) setting#asahi · 2021-05-31 21:23
arndagraf: yes. installing a second (or third) macos using the credentials from the internal drive works, but makes all other external partitions unbootable#asahi · 2021-05-31 21:22
arndwe could install a regular macos to a thunderbolt drive, but installing two separate macos installations to different partitions always failed, and the same happened when trying to replace the macos kernel on the external drive with m1n1#asahi · 2021-05-31 21:21
arndj_ey: I tried with my coworker's help and they couldn't do it with their credentials either#asahi · 2021-05-31 21:20
arndI couldn't actually get m1n1 to start from an external drive at all#asahi · 2021-05-28 09:49
arndpinskia: out of the Cortex cores, it seems that all cores starting with A73/A35 support all three granules (4K, 16KB, 64KB), just not the very earliest ones (A57, A53, A72) that are limited to 4KB and 64KB#asahi · 2021-05-28 08:34
arndI can also imagine running a 4KB page kernel on M1, as long as that rules out connecting arbitrary devices on thunderbolt. For the internal devices it's no worse than any of the other chips that have no usable IOMMU at all#asahi · 2021-05-28 08:32
arndUbuntu started shipping 64KB page kernels in addition to 4KB, so maybe they would add a third one as well#asahi · 2021-05-10 20:38
arndIt's probably best to wait another month now, in case there is going to be an M2 Mac next month#asahi · 2021-05-10 20:36
arndI've tried a few more variations of the same things on the external drive. I can consistently install a macos partition, but also consistently get the "Invalid argument: bless2_install_custom_obj() failed: 2" error when I actually try to install m1n1 with kmutil. I'll give up trying the external ssd method now, until I have my own hardware I can work on.#asahi · 2021-05-10 10:12
arndyes, it seems like a sensible limitation, but it probably means I won't be able to use this machine#asahi · 2021-05-10 10:10
arndagraf: I tried very hard to not touch the installation on the internal drive, and I don't have direct access to the OIK account (though I can ask the person for help occasionally)#asahi · 2021-05-10 10:09
arndagraf: my theory of what happened is that after a normal install to a empty partition on the external disk using OIK credentials, I can transfer the bootable state of that partition to another empty partition on that disk, which means I still only have one bootable external partitition#asahi · 2021-05-10 09:56
arndfrom the descriptgion on page 28 of that document it sounds like kmutil setting "permissive security" should have worked#asahi · 2021-05-10 09:53
arndagraf: the part that I don't understand is that I was able to install a bootable MacOS using the non-OIK credentials from a prior install (which was installed using the OIK owner credentials)#asahi · 2021-05-10 09:19
arndI also managed to create another Admin user on the internal rootfs by removing /Volumes/Macintosh\ HD/var/db/.AppleSetupDone. Apparently this user account can do everything except create a new bootable volume#asahi · 2021-05-10 09:14
arndI suspect this is still the same problem with permissions that prevents me from having multiple bootable partitions on the external drive