#
…

2021-06-07

00:20jevinskie[m] has joined #asahi
00:22al3xtjames has quit [Read error: Connection reset by peer]
00:23al3xtjames has joined #asahi
01:37bgb_ has joined #asahi
01:40bps has quit [charon.oftc.net liquid.oftc.net]
01:40apetresc has quit [charon.oftc.net liquid.oftc.net]
01:40ave has quit [charon.oftc.net liquid.oftc.net]
01:40skipwich has quit [charon.oftc.net liquid.oftc.net]
01:42ave has joined #asahi
01:42skipwich has joined #asahi
01:48bps has joined #asahi
02:01inglor has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
02:14phiologe has joined #asahi
02:14inglor has joined #asahi
02:17PhilippvK has quit [Ping timeout: 480 seconds]
02:46klltkr has joined #asahi
03:12skipwich has quit [Ping timeout: 480 seconds]
04:01marvin24_ has joined #asahi
04:04marvin24 has quit [Ping timeout: 480 seconds]
04:13zopieux has quit [Ping timeout: 480 seconds]
04:37klltkr has quit [Ping timeout: 480 seconds]
05:14VinDuv has joined #asahi
06:25VinDuv has quit [Quit: Leaving.]
06:58bisko has joined #asahi
07:02bisko has quit []
07:06bisko has joined #asahi
07:06bisko has quit []
07:37Glanzmann has joined #asahi
07:37Glanzmannkettenis: How is the status of OpenBSD on M1? Have you ironed out the SMP bug?
07:40kettenisno, SMP is still subtly broken, and there are some issues with NVMe as well still
07:43pitust[m] has joined #asahi
07:43kettenissome more OpenBSD developers are getting involved, so I have been working on the installer for folks to be able to install on NVMe without bricking their machines ;)
07:45Glanzmannkettenis: I see, what about the other drivers? Is usb working, what about ethernet?
07:50bisko has joined #asahi
07:56kettenisall works although the ethernet gets the default address programmed in the controller which I suspect is the same for all M1 machines
07:57kettenissome incompatible device tree binding changes coming up though, which is why I'm not advertising M1 support in OpenBSD much yet
08:05bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
08:10apetresc has joined #asahi
08:14bisko has joined #asahi
08:16GreatGodvin has joined #asahi
08:22kettenisI am curious though if the SMP issues will also show up on Linux when booted via m1n1
08:23kettenisdon't think anyone has used Linux in anger yet that way
08:23pipcet[m]doing what?
08:24kettenisdoing parallel builds of things natively
08:24_jannau_kettenis: I haven't seen any during kernel compiles. booting linux directly from m1n1 with sven's dart/usb-c patches
08:25pipcet[m]i've done that a lot with my kernel (based on corellium)
08:25pipcet[m]but I'm not sure what I should be looking for
08:26kettenisI'm seeing core dumps from (mostly) clang and ld.lld
08:27ketteniswas wondering if there are some chicken bits that m1n1 doesn't set
08:27_jannau_pipcet[m]: you're also not booting via m1n1
08:27pipcet[m]sometimes I am.
08:28kettenisbut if an upstream kernel with sven's patches works fine, this must be an OpenBSD problem
08:28_jannau_my the kernel builds should have used gcc/binutils
08:30EER has joined #asahi
08:31j_eykettenis: what's the symptoms?
08:33ketteniscore dumps typically early on when running C++ constructors before main()
08:34kettenisnot yet pinned down whether it is TLB, cache or memory ordering related
08:35kettenisreviewed whether the kernel adheres to the "break-before-make" rules
08:35kettenisand inserted loads of strong memory barriers in the kernel that didn't seem to make a significant difference
08:36choozy has joined #asahi
08:40GreatGodvin has quit [Quit: GreatGodvin]
09:14marcanWe do know that OSX configures some "interesting" nonstandard memory behavior, but I don"t *think* our chicken bits do. Also, that would probably break VMs unless they reset them in Hypervisor.framework
09:41pitust[m]i'm trying to shrink APFS partition according to the instructions at https://github.com/AsahiLinux/docs/wiki/Developer-Quickstart
09:41pitust[m]i ran `diskutil apfs resizeContainer disk0s2 300GB`
09:41pitust[m]and my mac just hangs
09:42pitust[m]after a while
09:42pitust[m]and i had to hard reset
09:42pitust[m]and after a reboot nothing changed
09:43kettenisworked for me when i tried it
09:53pitust[m]i tried again
09:53pitust[m]and it still hangs
10:04Glanzmannpitust[m]: If I remeber correctly, you sometimes have to wait for a long time. I have done it throught he GUI. Maybe give it an hour?
10:04pitust[m]Glanzmann: the whole computer hangs and becomes fully unresponsive
10:05pitust[m]and the progress thingy stops
10:05pitust[m]it hangs on fsck_apfs somehow
10:16GreatGodvin has joined #asahi
10:22GreatGodvin has quit [Quit: GreatGodvin]
10:32EER has quit [Ping timeout: 480 seconds]
10:41bgb has joined #asahi
10:41bgb_ has quit [Ping timeout: 480 seconds]
10:44choozy has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
10:47Glanzmannpitust[m]: I think that is normal.
10:47zopieux has joined #asahi
10:47GlanzmannJust give it some time.
10:52Glanzmannpitust[m]: HOw long did you wait? For me it took at least 45 minutes IIRC.
10:52pitust[m]ah ok thanks
10:52pitust[m]i'll try later today
11:01marcanpitust[m]: are you doing this from 1TR? it should work live too, but much more likely to hang
11:01marcanfrom 1TR it's safer
11:15GreatGodvin has joined #asahi
11:16GreatGodvin has quit []
11:24GreatGodvin has joined #asahi
11:29GreatGodvin has quit []
11:29GreatGodvin has joined #asahi
11:45malvo has quit [Ping timeout: 480 seconds]
12:02bisko has quit [Read error: Connection reset by peer]
12:17choozy has joined #asahi
12:30GreatGodvin has left #asahi [#asahi]
12:46choozy has quit [Ping timeout: 480 seconds]
12:54choozy has joined #asahi
13:19bgb has quit [Ping timeout: 480 seconds]
13:26Glanzmannjannau: Where can I get Sven's tree with the dart/usb-c patches from?
13:31_jannau_Glanzmann: AsahiLinux/Linux branch dart/dev and m1n1 pull request 65
13:34svennot sure if that contains the dwc3 nodes fwiw
13:35svenbut other than that it should have everything required
13:35Glanzmannjannau: Thanks.
13:36GlanzmannSven: Thanks.
13:45_jannau_sven: indeed. Did you remove the dts changes? if I'm not mistaken I was using the dtb from the kernel to boot last week (before friday).
13:45svenyes, i removed it becau
13:45svenyes, i removed it because i was preparing the upstream submission
13:58svenok, should be back
13:59svenand now it should boot just fine with a 4k pagesize kernel as well :)
14:07adamcstephens has quit [Quit: The Lounge - https://thelounge.chat]
14:08adamcstephens has joined #asahi
14:18bisko has joined #asahi
14:21arndsven: did you manage to enable bypass mode on all iommu instances in the end, or do you program a linear map into them?
14:21j_eythere's a linear map
14:22arndok. for all of the address space or only the first 4GB?
14:23adamcstephens has quit [Quit: The Lounge - https://thelounge.chat]
14:26j_ey4GB I think
14:26bisko has quit [Read error: Connection reset by peer]
14:26bisko has joined #asahi
14:30svenarnd: i found the bit in the DART mmio space that tells us if they support bypass mode
14:30sventhe pcie ones don't, the usb ones do
14:30svenso for the pcie ones i have a dma-ranges on the pcie bridge that limits DMA to the first 4G
14:31svenand then i just program a static linear map and tell the iommu api that this iommu really needs a bypass domain
14:31arndI see. Have you checked all the other DART instances to see what those support?
14:32svencan't access the thunderbolt ones but i suspect they won't allow bypass either
14:32svenbut all others seem to support bypass mode
14:33marcanI still think we outright shouldn't support putting those in bypass mode, or at most do it only with a nondefault, scary parameter
14:33arndok. I suppose bypass mode on the thunderbolt wouldn't make much of a difference in practice, since that is too dangerous for practical use
14:33marcanthe tbt ones in particular
14:34arndfor the on-board devices, I don't see much of a problem
14:34marcanyou are extending the system attack surface to include Broadcom firmware
14:34marcanBroadcom *wifi* firmware
14:34marcanthat's a *significant* downgrade
14:35marcanso if we do that at least I want a scary kmsg warning
14:35arndah, indeed
14:35arndI was only thinking of ethernet and pci-xhci
14:36marcanyeah, if we find a sane way to disable wifi in this scenario I'd be happier
14:36arndthough both of those also run firmware that might be attacked in theory
14:36marcanthough really nobody should be using these things with 4K pages, given all this, except for like installers
14:36marcanbut the wifi attack surface is a couple orders of magnitude worse (and it's had a bunch of CVEs already)
14:38svenhow many distro ship 16k pagesize kernels these days?
14:38marcanwe're going to change that ;)
14:38marcanenough people will want M1 not to suck
14:38marcanand besides, 16K has advantages on other platforms too
14:39marcando we even know if the GPU MMU does 4K pages? because if it doesn't, lol no 3D for 4K kernels
14:39arndMost developer boards still have only Cortex-A53 or Cortex-A72, which don't support 16KB pages at all
14:40marcanI expect distros to ship multiple kernels
14:40marcansome already do
14:40marcane.g. ubuntu has 64k and 4k options
14:40arndI expect distros to fight that very hard, as it adds a lot of extra validation work to have more than one
14:41svenall we know about the gpu mmu is that it's not a dart
14:41arndI honestly don't know why Ubuntu did it
14:41marcanI mean, look, if there is no better solution, that's that
14:41marcanif we can't make the kernel somehow deal with a 16K IOMMU when built for 4k, there is no other good option
14:41marcanand then if the distros don't ship 16k kernels, someone else will
14:41marcanpeople will use them
14:41sveni'd be happy with 4k without thunderbolt as a fallback. that way most people can at least use this
14:41marcanand then it's up to the distros whether to eventually cave or not
14:42svenand i'm sure there will be third party repos that will ship the distro kernels with 16k support then
14:42marcansven: I'd also kill wifi for "encouragement" ;)
14:42arndI still hope that opening up the discussion again will at least make some users understand just how bad 64K pages are for general-purpose workloads
14:43marcaninsmod dart.ko expose_me_to_broadcom_firmware_bugs=1 (or whatever module we put the gate in)
14:43marcan*expose_me_to_broadcom_firmware_vulnerabilities=1
14:43arndthough a more likely outcome is that a lot of users run microbenchmarks that show 64K pages being faster because of lower TLB pressure
14:43svenit might be possible to make the kernel deal with 16k iommu when built for 4k but the dart driver itself is already >1k loc or so
14:43marcansounds like it's more of a kernel issue
14:43svenat least it boots this way. we can always tackle the 16k support later
14:43marcanallocating DMA memory with higher than page size alignment
14:44marcanI just want the wifi thing to be opt-in
14:44marcanpeople should be *aware* that running in this mode is crap for security for several reasons, but especially that one
14:44marcanif distros want to flip the flag instead of ship 16k, that has to be an explicit decision
14:45arndyes, makes sense (on wifi and thunderbolt)
14:46arndsven: right, that fixing the common iommu code to work with this should be possible by the time that distros actually want to ship installers to work on mac. That is still a going to be a while, and your solution is enough for an initial merge
14:46arndof the iommu driver
14:46marcanas long as it's gated behind a nondefault argument I'm fine with that
14:47marcanwe can just have a dart.enable_bypass=1 or whatever for now
14:48marcanand say you need that for usb/eth on 4k kernels
14:48marcanand then we can be more specific about just breaking wifi by default if we want to in the future
14:49j_eysven: not too bad, arm-smmu-v3.c is over 3000 for example
14:49arndfor the streaming mappings, I think there is already a small security hole regardless of the iommu type and the page size, as dma_map_single() often gets passed small buffers that live in the same page as other (possibly sensitive) data
14:50svenj_ey: the point is that reviewing 1k lines of code is already quite a lot. adding the 16k/4k thing on top that makes it even worse
14:50arndespecially on USB, the kernel often just passes buffers that are only a few bytes long
14:50j_eysven: oh right, I was thinking as a later patchset
14:50svenj_ey: oh, sure.
14:52svendart also supports subpage protection fwiw, but patching *that* into the iommu code would be more annoying
14:52arnddrivers using only the coherent interfaces (dma_alloc_coherent() etc), currently get full pages, so those are safe as long as the page size matches, or we change the common code to use io page size allocations
14:53arndsven: what are the parameters for sub-page protections? Can you e.g. map an arbitrary naturally aligned 64 byte range in a page?
14:53arnd(that would be the minimum the kernel might pass)
14:54sveni think so. i haven't tested this in detail but from what i can tell you can limit access inside the page down to that level
14:54arndok, perfect. So the dart would actually end up being more secure than most others for the streaming API ;-)
15:00arnddo if we find a way change the iommu ->map() callback to pass the location within the page and do sub-page protection as well as fix the dma-iommu code to no longer assume iopagesize<=page_suze, I think that should allow safely using wireless and thunderbolt with 4KB pages, it would only prevent you from using device assignment into VM guests and user space like DPDK
15:01svenagreed
15:02svenso i don't think the iommu_pagesize > cpu_pagesize fix is going to require a lot of changes, but it'll certainly require reviewing the entire code to be comfortable it doesn't break anything by accident
15:03sven(it kinda works by accident already if you just remove the BUG_ON)
15:05arndsven: have you found out if dart can do both cache-coherent dma mappings and non-coherent ones? Is bypass mode still coherent?
15:05marcanarnd: by the way, "small buffers that live in the same page as other (possibly sensitive) data" isn't a "small" security hole
15:05marcanthis is how the ps4 got owned ;)
15:06marcanso yes, let's look into subpage protection
15:06marcandoesn't need to happen for the initial submission of course
15:06marcanbut this is very much worth doing
15:06brentr123[m]I wonder if the ps5 can get pwned the same way ;)
15:08arndmarcan: I wonder if it makes sense to address this as a general security issue then, and start by making the dma-iommu code use bounce buffers as an option, and make the sub-page protection as a way to make it faster
15:08bloomooi what's the issue with 64k pages?
15:08bloomfor general purpose workloads
15:08bloomarnd: ^
15:09arndbloom: mostly a waste of memory, e.g. when dealing with small files your page cache often blows up to 5x the memory usage
15:09svenarnd: from what i can tell it always maps everything as coherent.
15:09svenbypass mode hasn't broken anything yet while using usb but i'm not confident there yet
15:10arndsven: ok, good
15:11bloomarnd: Ah, got it.
15:11svenarnd: afaik the iommu code already has the option to use bounce buffers for untrusted pci devices
15:11marcanarnd: certainly, if we can make it use bounce buffers easily for sub-page maps, it makes sense to start off with that
15:11svenor maybe that was just a patchset i saw, let me check
15:11* marcanoff to sleep
15:11arndbloom: it's extremely workload specific, and a number of other things also go wrong: anything that small bits of data but requires page alignment can explode. Other things I'm sure cause problems are swapspace handling, and network buffers
15:13arndbloom: ok the other hand, anything that is purely CPU bound and fits within a small working set tends to benefit from the lower TLB usage, so microbenchmarks almost always show 64KB pages come out winning
15:16arndWhen I tested kernel builds on Amazon EC2 comparing page sizes, I found that with 4GB RAM per core, 64KB pages were a few percent faster, but once I reduced the amount of RAM I found that 4KB pages still worked fine even after it started swapping, but as soon as the 64KB kernel went into swap space, it would start thrashing and never complete the kernel build
15:16arndor complete it after a day of wearing out the disks, instead of a few minutes of building in RMA
15:17arndsven: ok, good to know, I have to read up on that again
15:17sveni think iommu_dma_map_page uses __iommu_dma_map_swiotlb which uses bounce buffers. and from what i can tell dma_map_page is used for streaming dma
15:19arndsven: ok, got it, thanks for the pointer
15:24arndit looks like that was added fairly recently (last November) based on code from the Intel iommu driver.
15:27arndsven: the dev_is_untrusted() logic in there looks like the exact thing to use for determining whether to set up a linear map or not
15:28arndfor the thunderbolt ports, it would be set using the "external-facing" property, for the wireless device I suppose the driver could set it
15:39svenoh, good point!
15:39sveni think if dev_is_untrested is set it defaults to DMA domains anyway
15:40sven*dev_is_untrusted
15:40svenyup, in iommu_get_def_domain_type there is a check before it calls to the iommu driver to get the default domain: if (dev_is_pci(dev) && to_pci_dev(dev)->untrusted) return IOMMU_DOMAIN_DMA;
15:41svenso if something is untrusted the dart driver will never see a IOMMU_DOMAIN_IDENTITY unless the user forces it
15:45arndThat sounds exactly like what we want, though I'm not sure it would work if the wireless driver sets the flag from the probe function, that is probably too late.
15:50malvo has joined #asahi
15:51svenhm, but i think we'll somehow need the wireless device in the dt anyway since it needs that special poweron sequence
15:52malvo has left #asahi [#asahi]
15:55arndsven: right, that would work, we could just put the "external-facing" property on the bridge node, but that feels wrong, as what we really want is to mark that whole class of broadcom wireless devices (or maybe any wireless network device?) as untrusted, regardless of what machine they are used in
16:07choozy has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
16:30VinDuv has joined #asahi
16:49skipwich has joined #asahi
16:54pugguu has joined #asahi
17:01marvin24 has joined #asahi
17:04marvin24_ has quit [Ping timeout: 480 seconds]
17:25bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
17:26EER has joined #asahi
18:16kettenissven: did you write down your findings about subpage protection somewhere?
18:22svenno, because i'm not 100% sure how it works
18:23sveniirc the region was encoded in the top 24 or so bits of the PTE and you had to clear bit 0 or 1 to enable it
18:26svenhrm, can't find my old code anymore
18:28Emantorsven: The pgsize calculation in the dart driver looks wrong, it's doing a 0x4 > PAGE_SIZE comparison for the USB darts. This way the force_bypass parameter will never work.
18:29EmantorYou probably wanted to shift 4096 instead of 1?
18:29svenshift?
18:30svenit's doing 0x4000 > PAGE_SIZE
18:30svenor am i missing something?
18:31Emantorfield get returns 0x2 under barebox, i guess the kernel is the same?
18:31svenhuh, that's weird
18:31svenit should be 0xe
18:31EmantorDart Params is:
18:31Emantor 16 0 24 12 0 24 20 16 12 8 4 0
18:31Emantor49283072 0x2f00000 0274000000 0b10111100000000000000000000 = 47.00 iM
18:32sveni don't understand that fomat
18:32svendart_params[0] = 0x2ed01010 for the display dart here
18:33Emantorhu, wierd.
18:33svenread32(0x382f00000) = 0x8ee01020
18:33sventhat should be the usb dart
18:36EmantorYeah, there is something broken on my end.
18:41EmantorArgh, wrong regs >_>
18:42EmantorGreat, now that the dart driver works by enabling bypass mode, gadget mode receives requests. Sorry for the noise.
18:43pipcet[m]gadget mode in barebox Just Works if you start it from linux :-)
18:43svensame if you start if from m1n1 with my pr *shrug*
18:43svenhost mode should just work as well there
18:44sventurns out you don't actually need a different phy init sequence for gadget/host mode (at least for usb2)
18:47pipcet[m]so switching between host and gadget modes works now?
18:47jannauno new hardware, haha
18:48sveni didn't try switching but the same phy init pokes i have work for either mode without an additional phy driver in linux
18:49svenoh, when you try to switch you'll run into the "using the same port twice isn't supported" issue :D
18:49svenbut once we figure that out mode switching should just work as well
18:49pipcet[m]sven: we can just reset the port once every few seconds except when it's connected :-)
18:49svenyeah, no
18:50svenyou also only need to reset once
18:50svenso if you want to hack it just spawn a thread that polls and checks if it's connected and if not resets it once
18:51pipcet[m]can't you fix it instead? pretty please?
18:51sveneventually, yeah
18:52sveni first want to get the DART driver and the clocks upstreamed though
18:52svenif i wanted to just get something somehow working we would have a kernel+distro hacked together a few months ago already ;)
18:55jannauI've started working on the gpio driver btw
18:57svenpeter[m]You’ve probably seen kettenis’ bindings haven’t you?
18:57svenpeter[m]I think they’ve already been reviewed by the maintainers
18:57kettenisyes, the bindings are going in via the pinctrl tree
19:05Andalu30 has joined #asahi
19:06Emantorpipcet[m]: chainloading the generic 2nd dt image for barebox fails for me, HV works obviously. But the gadget still spews lots of errors.
19:07* Emantorwishes for a USB PD board.
19:07jannauyes, using the v3 patches as base
19:07pipcet[m]emantor: silly question, but you're passing an appropriate device tree?
19:08Emantorpipcet[m]: I'm passing m1n1 with bb & dt concatenated. So DT should be fine.
19:08Emantorno, wait
19:09Emantorye, doesn't matter. Still get an apple logo screen chainloading m1n1 with a barebox payload.
19:10pipcet[m]the frame buffer is black on black unless you apply the patch or hack the DT to claim x8r8g8b8 layout.
19:12EmantorI know, this is with your patch applied on top.
19:12pipcet[m]are you running m1n1 without a logo? I'm confused by the apple logo
19:13EmantorYou are not alone. No m1n1 is running with a logo and correctly sets up the asahi logo.
19:14pipcet[m]...and then it goes back to the apple logo somehow?
19:15Emantoryep, white apple logo smack down in the middle.
19:18EmantorRunning under the hypervisor gives me nice simplefb output with debug apple logo. Where does that even come from?
19:18Emantordebug apple logo == color stripped apple logo.
19:19svenpeter[m]That’s an Easter egg inside mini 😅
19:19VinDuvm1n1 stores the original logo and restores it during fb_shutdown
19:19VinDuv(and changes it if fb_improve_logo is called)
19:21pipcet[m](...my excuse is I didn't get much sleep since I tried to get nvme to work in barebox)
19:26Erus_Iluvatar has quit [Quit: Erus_Iluvatar]
19:26Erus_Iluvatar has joined #asahi
19:33klltkr has joined #asahi
19:42erincandescent_ has quit [Remote host closed the connection]
19:43erincandescent has joined #asahi
19:47klltkr has quit [Quit: My MacBook Air has gone to sleep. ZZZzzz…]
19:48klltkr has joined #asahi
19:48klltkr has quit []
19:49Andalu30 has quit [Quit: Konversation terminated!]
19:49klltkr has joined #asahi
19:49klltkr has quit []
19:49klltkr has joined #asahi
19:49klltkr has quit []
19:50klltkr has joined #asahi
19:50klltkr has quit []
20:01pipcet[m]emantor: do you happen to know whether nvme support in barebox works on other machines? I got something to work but I had to touch the generic code...
20:03Emantorpipcet[m]: Yep, should work on boards with an nvme slot, which are the zii imx8mq boards AFAIK.
20:05VinDuv has quit [Quit: Leaving.]
20:06choozy has joined #asahi
20:06pugguu has quit [Ping timeout: 480 seconds]
20:12nico_32speaking of ubuntu and multiple kernel
20:12nico_32they ship a kernel variant just for microsoft azure
20:16pipcet[m]emantor: thanks. I'm probably doing something stupid...
20:17EmantorAren't we all? :-)
20:49thunfisch has quit [Remote host closed the connection]
20:50thunfisch has joined #asahi
21:19kettenis_ has joined #asahi
21:20kettenis has quit [Ping timeout: 480 seconds]
21:49EER has quit [Remote host closed the connection]
22:16choozy has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
…