#
…

2022-04-09

00:03kameks has joined #asahi-dev
00:12rqou_actually, even with a 16k kernel vfio would still fail with "no interrupt remapping support"
00:12rqou_how feasible would it be to get this actually working? since once we have thunderbolt, vfio passthrough to a vm isn't unthinkable
00:49rqou_... reading a bad address on pcie will instantly hang the whole system until it watchdog resets. that seems suboptimal
00:57nepeat has quit [Quit: ZNC - https://znc.in]
01:20Emantor has quit [Quit: ZNC - http://znc.in]
01:20Emantor has joined #asahi-dev
01:20nepeat has joined #asahi-dev
01:32yuyichao has quit [Ping timeout: 480 seconds]
01:43rqou_how difficult is it to bring up PCIe from within a m1n1 experiment?
01:44yuyichao has joined #asahi-dev
01:45rqou_apparently vfio-pci is just broken in a way i don't understand, because doing exactly what the macos driver is doing crashes the system
01:49rqou_unless i have to do something with the config space that i don't understand
01:51kenzie has quit [Quit: Ping timeout (120 seconds)]
01:51kenzie has joined #asahi-dev
01:59kenzie has quit [Ping timeout: 480 seconds]
02:29kenzie has joined #asahi-dev
02:54PhilippvK has joined #asahi-dev
02:57phiologe has quit [Ping timeout: 480 seconds]
03:25nicolas17 has quit [Quit: Konversation terminated!]
03:45kov has quit [Quit: Coyote finally caught me]
04:11gladiac is now known as Guest1528
04:11gladiac has joined #asahi-dev
04:17Guest1528 has quit [Ping timeout: 480 seconds]
05:01kevans91kettenis: I suspect this isn't quite right: https://github.com/openbsd/src/blob/master/sys/arch/arm64/dev/aplintc.c#L37 (noting that APL_IPI_LOCAL_RR_EL1 and APL_IPI_GLOBAL_RR_EL1 probably aren't the same)
05:15nafod has quit [Quit: Ping timeout (120 seconds)]
05:17nafod has joined #asahi-dev
05:35robinp_ is now known as robinp
05:47kameks has quit [Ping timeout: 480 seconds]
06:01svenrqou_: 4k kernels will never work with vfio (or anything else that uses the iommu api directly instead of going through dma-iommu)
06:22kameks has joined #asahi-dev
06:22the_lanetly_052___ has joined #asahi-dev
06:23rqou_why not though? the vfio api seems to be able to expose iommu page sizes that differ from cpu page sizes
06:28the_lanetly_052__ has quit [Ping timeout: 480 seconds]
07:01kode54kernel doesn't yet support mixed page sizes
07:02alexsv has quit [Remote host closed the connection]
07:02kode54otherwise this 16k/4k issue would be better dealt with by supporting that and having separate 16k/4k processes
07:02alexsv has joined #asahi-dev
07:03rqou_right, sven wrote something that can support mixed iommu/cpu pages sizes
07:03rqou_i don't understand why it can't work with vfio though
07:04svenyou’d have to audit all the code to make sure nothing makes the implicit iommu page size = cpu page size assumption
07:05sventhat’s easy enough for dma-iommu but gets annoying for everything that uses the raw iommu api
07:05rqou_seems like it should be possible for vfio?
07:05rqou_vfio exposes iommu page sizes in the userspace api
07:06svenI don’t know the api well enough
07:06rqou_neither do i lol, i just used it for the first time today
07:07rqou_but VFIO_IOMMU_GET_INFO returns a struct containing a bitmap of supported iova page sizes
07:07rqou_so this looks like it should be theoretically possible
07:08svenyou’d have to check if anyone even looks into that and handles the iommu > cpu case
07:08rqou_well, i tried, but it fails before it can even get there
07:08sventhat could also be meant for iommu < cpu or for large page support and all that
07:09rqou_i think it's probably for large page support, but it doesn't actually say so explicitly
07:09svenand then it’s a user space api so who knows what people do with it
07:10rqou_as far as i understand it, returning an iommu page size > cpu page size and then failing when attempting to map an improperly aligned page does not violate the api specification
07:10rqou_but who knows what userspace does
07:11rqou_and by userspace afaict i just mean "qemu"
07:11svenright, the same is true for the raw iommu api but I’m still sure lots of random things will break
07:11svenI’d at least hide that case between an explicit opt-in
07:11rqou_fair enough
07:11sven*behind
07:12rqou_won't be the first time vfio has a bunch of random scary opt-in config options
07:12svenOr just not support it at all :D
07:12rqou_like the option that outright ignores improperly configured iommu
07:12svendma-iommu makes sense to allow distro installers to boot and for stuff like box64/fex
07:12rqou_at least on x86, vfio has a bunch of enthusiast users doing totally wild things
07:13svenIf you want something special like vfio you’re an enthusiast and can just switch to 16k kernels 🤷🏻‍♂️
07:13rqou_yeah, i did that for testing, but i'm hitting a different issue unrelated to any of this
07:13rqou_btw 16k kernels break sublime text (because jemalloc)
07:13sven(Or apply a scary patch/config options)
07:14rqou_vfio even on a 16k kernel still has the problem with "unsafe interrupt remapping"
07:14svenyes, and hopefully by shipping 16k kernels people start fixing their code
07:14rqou_but i have no idea what that actually means or how interrupts actually work on pcie
07:14rqou_there's an option to tell vfio to ignore that, which works
07:14rqou_idk if that's something we can get properly supported
07:15svenhm, I’d have to actually understand vfio and see what’s up with that
07:15svenbut I really just know that vfio exists and roughly what it does :D
07:15rqou_i've used it for "enthusiast" gpu passthrough on x86 (mostly works)
07:16rqou_and like i said, i was testing it on m1 hoping i could prototype a driver in userspace
07:16sventhe scary patch to make vfio work is to just add that iommu large page whatever bit to the define for a non-dma-iommu domain
07:17svenwhere work probably means “will crash somewhere”
07:18rqou_btw there are also scary/cursed high performance networking use cases where people run userspace pci drivers
07:18rqou_idk if anybody would be nuts enough to try that on m1
07:18rqou_once we have tbt it's possible!
07:22rqou_anyways, i'm not familiar with the linux dma-iommu api, but vfio's interface is quite high-level and "should" be possible to get working with mismatched page sizes with "just" some coding
07:22rqou_but not super important
07:41jluthra has quit [Remote host closed the connection]
07:41jluthra has joined #asahi-dev
08:33kameks has quit [Ping timeout: 480 seconds]
08:44ketteniskevans91: LOL, yes, that can't be right (it seems to work just fine though)
08:44kettenisthanks for spotting that
08:58xiaomingcc[m] is now known as xiaoming[m]
09:11kgarrington has joined #asahi-dev
09:18the_lanetly_052___ has quit [Ping timeout: 480 seconds]
09:21kgarrington has quit [Remote host closed the connection]
09:23SSJ_GZ has joined #asahi-dev
09:28the_lanetly_052___ has joined #asahi-dev
09:33kameks has joined #asahi-dev
09:41the_lanetly_052___ has quit [Ping timeout: 480 seconds]
10:01robinp_ has joined #asahi-dev
10:09robinp has quit [Ping timeout: 480 seconds]
10:32ChaosPrincess has quit [Quit: WeeChat 3.4.1]
10:33ChaosPrincess has joined #asahi-dev
10:37svenrqou_: the problem really is that client/userspace drivers won’t expect that weird page size mismatch
10:37svenit’s the same for the iommu api: clients do get access to the iommu granule but they’ll likely make assumptions that are no longer true in our case
10:38svenso the solution is to make that weird case an explicit opt-in. That’s essentially what my patch does, and then dma-iommu uses that opt in
10:43herbas has joined #asahi-dev
10:47herbas has quit []
10:54kameks has quit [Ping timeout: 480 seconds]
11:21kameks has joined #asahi-dev
11:22pilonsi has joined #asahi-dev
11:39pilonsi has quit [Quit: Leaving]
11:43yuyichao has quit [Quit: Konversation terminated!]
11:43yuyichao has joined #asahi-dev
11:51yuyichao has quit [Quit: Konversation terminated!]
11:51yuyichao has joined #asahi-dev
12:15yuyichao has quit [Ping timeout: 480 seconds]
12:18jnn has joined #asahi-dev
12:19c10l7 has quit []
12:20jn has quit [Ping timeout: 480 seconds]
12:21yuyichao has joined #asahi-dev
12:28pilonsi has joined #asahi-dev
12:55pilonsi has quit [Quit: WeeChat 2.3]
13:04pilonsi has joined #asahi-dev
13:28florolf has joined #asahi-dev
13:33kameks has quit [Ping timeout: 480 seconds]
13:47the_lanetly_052___ has joined #asahi-dev
14:13jnn is now known as jn
14:30hectour[m] has left #asahi-dev [#asahi-dev]
14:39phire_ is now known as phire
14:50kevans91kettenis: no problem, thanks for the good reference drivers :-)
15:24opticron has quit [Ping timeout: 480 seconds]
15:34minecrell has quit [Quit: Ping timeout (120 seconds)]
15:34minecrell has joined #asahi-dev
16:16opticron has joined #asahi-dev
16:35kevans91kettenis: it looks like you can kill sc_cpuremap over in aplintc, it's write-only as of fast ipi support
16:44kov has joined #asahi-dev
16:59MajorBiscuit has joined #asahi-dev
17:07rqou_sven: oh ok, that makes sense. thanks for that explanation
17:19MajorBiscuit has quit [Ping timeout: 480 seconds]
17:59ketteniskevans91: it will be needed again when I add support for running interrupts on other cores
17:59kettenisso I've left it in place for now
18:12doggkruse has joined #asahi-dev
18:22doggkruse has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
18:32SSJ_GZ has quit []
19:13amarioguy has joined #asahi-dev
19:15yuyichao has quit [Ping timeout: 480 seconds]
19:17kevans91kettenis: ah, makes sense- I'll resist the urge to rip out our equivalent, then =-)
20:11nicolas17 has joined #asahi-dev
20:13amarioguy has quit [Ping timeout: 480 seconds]
20:30yuyichao has joined #asahi-dev
20:34yuyichao has quit []
20:35yuyichao has joined #asahi-dev
20:42yuyichao has quit [Quit: Konversation terminated!]
20:46yuyichao has joined #asahi-dev
21:07amarioguy has joined #asahi-dev
21:32amarioguy has quit [Ping timeout: 480 seconds]
22:18Retr0id has quit [Ping timeout: 480 seconds]
22:22amarioguy has joined #asahi-dev
22:30amarioguy has quit [Ping timeout: 480 seconds]
22:53Mase3206 has joined #asahi-dev
22:59Mase3206 has quit [Quit: Mase3206]
23:09Mase3206 has joined #asahi-dev
23:15Mase3206_ has joined #asahi-dev
23:21Mase3206 has quit [Ping timeout: 480 seconds]
23:21Mase3206_ is now known as Mase3206
23:24amarioguy has joined #asahi-dev
23:42amarioguy has quit [Ping timeout: 480 seconds]
23:44Mase3206 has quit [Quit: Mase3206]
23:53amarioguy has joined #asahi-dev
…