#
…

2023-05-04

00:00darkapex has joined #asahi-dev
00:03nsklaus has quit [Remote host closed the connection]
00:18bisko has joined #asahi-dev
00:19bisko has quit []
00:40brolin has quit [Ping timeout: 480 seconds]
01:28darkapex has quit [Remote host closed the connection]
01:43hightower3 has joined #asahi-dev
01:50hightower2 has quit [Ping timeout: 480 seconds]
01:50tobhe_ has joined #asahi-dev
01:57brolin has joined #asahi-dev
01:57tobhe has quit [Ping timeout: 480 seconds]
02:00gabuscus has quit []
02:17gabuscus has joined #asahi-dev
03:09brolin has quit [Ping timeout: 480 seconds]
03:25stipa is now known as Guest462
03:25stipa has joined #asahi-dev
03:26Guest462 has quit [Read error: Connection reset by peer]
03:54bisko has joined #asahi-dev
04:12seb91nihel has joined #asahi-dev
04:15bisko has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
04:19seb4nihel has quit [Ping timeout: 480 seconds]
05:28martinr1 has quit [Ping timeout: 480 seconds]
05:30martinr1 has joined #asahi-dev
05:36dhat has quit [Quit: KVIrc 5.0.0 Aria http://www.kvirc.net/]
05:38martinr1 has quit [Ping timeout: 480 seconds]
06:03greguu has quit [Ping timeout: 480 seconds]
06:04martinr1 has joined #asahi-dev
06:12jannauChaosPrincess: have you tried reverting the clock change?
06:12martinr1 has quit [Ping timeout: 480 seconds]
06:21martinr1 has joined #asahi-dev
06:29martinr1 has quit [Ping timeout: 480 seconds]
06:33chadmed_ has quit [Quit: Page closed]
06:39martinr1 has joined #asahi-dev
06:47StupidYui has joined #asahi-dev
06:50tanty has quit [Remote host closed the connection]
06:53tanty has joined #asahi-dev
06:53greguu has joined #asahi-dev
06:54StupidYui has quit [Remote host closed the connection]
06:54StupidYui has joined #asahi-dev
06:54martinr1 has quit [Ping timeout: 480 seconds]
06:56StupidYui has quit [Remote host closed the connection]
06:56StupidYui has joined #asahi-dev
06:56StupidYui has quit [Remote host closed the connection]
06:57StupidYui has joined #asahi-dev
06:57StupidYui has quit [Remote host closed the connection]
06:57StupidYui has joined #asahi-dev
07:08sarucchi has quit [Quit: Textual IRC Client: www.textualapp.com]
07:09tanty has quit [Quit: Ciao!]
07:10sarucchi has joined #asahi-dev
07:14hightower3 has quit [Ping timeout: 480 seconds]
07:25dh-- has joined #asahi-dev
07:26tanty has joined #asahi-dev
07:26martinr1 has joined #asahi-dev
07:28dh-- has quit [Remote host closed the connection]
07:28dh-- has joined #asahi-dev
07:31bps has joined #asahi-dev
07:31cylm has quit [Ping timeout: 480 seconds]
07:35bisko has joined #asahi-dev
07:43darkapex has joined #asahi-dev
07:53sarucchi has quit [Quit: Textual IRC Client: www.textualapp.com]
07:53sarucchi has joined #asahi-dev
08:02bps has quit [Ping timeout: 480 seconds]
08:05sarucchi has quit [Ping timeout: 480 seconds]
08:18nsklaus has joined #asahi-dev
08:31timokrgr has quit [Quit: User left the chat]
08:33bps has joined #asahi-dev
08:35timokrgr has joined #asahi-dev
09:06hightower2 has joined #asahi-dev
09:08kettenisCreating a UEFI boot option seems to do the trick
09:57Tomdownsouth has joined #asahi-dev
10:09bps has quit [Ping timeout: 480 seconds]
10:12bps has joined #asahi-dev
10:58cylm has joined #asahi-dev
11:22marcanmaz: repro'd the KVM pathological behavior. it's really obvious if you have a GUI.
11:22marcanthe symptoms are consistent with host AIC IRQ delivery being broken or severely delayed while running the guest
11:23marcanin particular, if the guest is running on p-cluster 0 and I put a host-side CPU burner on p-cluster 1, it gets better randomly (p-cluster IRQ delivery is random). if I put the burner on the e-cluster, everything goes back to normal since (in the default AIC config) IRQ delivery prioritizes the e-cluster (but prioritizes not waking up asleep CPUs first)
11:24mazmarcan: interesting. this would mean that a host interrupt doesn't kick us out of a guest? that'd be terminally broken...
11:24marcanI wonder if there's a magic bit we need to set somewhere to fix that...
11:25marcanI'll have to write a m1n1 test case first to prove this is the hardware and not KVM being terminally broken
11:25marcanbut if I can't repro in m1n1 we'll have to look at the latter option :p
11:27mazwell, we set IMO+FMO+AMO, so *everything* has to take us to EL2. If that's not obeyed, that's pretty bad.
11:27marcanyup, but we also know Apple have a magic IMPDEF we need to poke to make AIC IRQ delivery work *at all* so...
11:27marcanwe know there's weird magic involved
11:27mazanyway, if you find a KVM bug, you now where to shout!
11:29kettenisjannau: pushed the code to generate an EFI boot option to the asahi branch
11:29Tomdownsouth has quit [Ping timeout: 480 seconds]
11:30kettenisprobably best to remove the ubootefi.var from the ESP before trying this diff
11:31marcankettenis: should I be removing that from linux in our update script?
11:32kettenisthe u-boot that you've been shipping had the option that generates this file disabled
11:32kettenisso it shouldn't be needed
11:34kettenisbut I guess it doesn't hurt
11:34kettenisuntil users start messing around with EFI variables themselves
11:35kettenisI think Fedora does set EFI variables
11:35kettenis(that file is where u-boot stores the EFI variables)
11:36jannaukettenis: I'll try later
11:37kettenisI really should install Asahi Linux on at least one of the machines I have
11:37jannauI think some fedora variant shipped at some point u-boot built from the wrong branch and had efi variables enabled
11:39kettenisthat in itself shouldn't be a problem if they set the variables to point at the right partition and bootloader
11:41kettenisso what my new code does is create a Boot0000 boot option that points at the ESP with the UUID given by asahi,efi-system-partition and creates a BootOrder variable that points at that option
11:42kettenisif the BootOrder variable already exists, it doesn't do anything
11:42kettenisthis way, a newly installed system should boot from the right partition
11:43kettenisbut if a user (or OS installer) overrides BootOrder it will do what the user requested
12:34Tomdownsouth has joined #asahi-dev
12:53marcanmaz: IRQs do kick us out of the guest but I think the latency is total crap. I suspect an IMPDEF to control that...
13:01marcanactually no, the real latency looks small?
13:06kettenismarcan: regarding the oslog buffers, wouldn't it be better for u-boot to allocate those in RAM and make sure the memory gets reserved in the FDT and the EFI memory map?
13:06marcanno, because they go through a DART
13:06marcanand passing through DART mappings is a world of pain (see DCP)
13:07marcanthe SRAM bypasses that
13:07korreckj328 has joined #asahi-dev
13:08mazmarcan: what latency do you measure? from what point to what point? m1n1 or KVM?
13:08kettenisdid you push the updated masks for iova and size for the oslog messages somewhere?
13:10marcankettenis: not yet
13:10marcanmaz: m1n1, but this is weird.
13:19Z750 has quit [Quit: Ping timeout (120 seconds)]
13:20Z750 has joined #asahi-dev
13:21marcanmaz: aaand it's the impdef register
13:21marcanI don't know *why* it's broken, but the symptoms seem to be that the IRQ gets delivered but AIC somehow thinks it isn't, even after it gets acked, and ends up timing out and also delivering to some other CPU
13:22marcanit's as if the "delivered to CPU core" signal never gets back to AIC even though it has
13:22marcanthat's with magic_reg = 0
13:22marcanmagic_reg = 3 is the boot value we know doesn't work
13:22marcanturns out magic_reg = 2 makes everything work
13:22marcansooo... m1n1 fix.
13:23marcanmacOS sets magic_reg = 0 on normal boots, but who knows what it does with virtualization
13:23marcanbet it changes
13:24mazhmmm. I positively hate magic.
13:24mazI wonder if that has something to do with how long it takes to ack the interrupt.
13:25marcanthe ack has the same latency in both cases (I'm grabbing the counter right before triggering the IRQ and right after acking it in the EL2 IRQ handler that fired from EL1 context)
13:25mazbecause if you're interrupting while running a guest, it takes significantly longer to handle the interrupt than if you were running userspace.
13:25marcanbut it looks like what fails to ack here isn't the IRQ itself, it's the lower level mechanism that picks a CPU to interrupt
13:26marcanI'm not sure how this causes breakage, but for example I wouldn't be surprised if this machinery running is blocking
13:26marcanwhich means when the *next* IRQ comes in, until this thing times out, it doesn't get delivered
13:26marcanso the first IRQ arrives fine, the next one is bork
13:26marcanis my hypothesis
13:28marcananother possiblity is that this is an intentional mechanism so that the extra HV IRQ latency is considered in the "pick a CPU" heuristic, and the HV is supposed to manually ack the low-level IRQ in some way with that bit clear
13:32brolin has joined #asahi-dev
13:48Tomdownsouth has quit [Ping timeout: 480 seconds]
13:58sarucchi has joined #asahi-dev
14:07marcanmaz: I'm staring at the kernel and this looks like an errata on t600x...
14:08marcanthere's a chicken kernel arg and some special handling here, and it only applies to pcores
14:20marcanmaz: ok no, it only enables the workaround on dev machines by default as far as I can tell, and I see the same behavior on ecores anyway.
14:20marcanso there is *some* bug but it's not our problem
14:21mazOK, so it all edges on this magic sysreg?
14:24marcanyeah, what I'm trying to figure out is whether the hypervisor pokes it (which would answer the question) or whether we screwed something else up.
14:29nela has quit [Ping timeout: 480 seconds]
15:02gladiac has joined #asahi-dev
15:14cylm has quit [Quit: WeeChat 3.6]
15:39marcanmaz: I give up, but setting the magic reg to 2 fixes it so shrug
15:39marcan(tested on linux)
15:39marcanpushing a new m1n1 now with that
15:41marcanpushed to asahi-dev
15:43mazmarcan: fair enough, thanks for having investigated it. eventually, I need to rebuild my t6002 anyway, as it still runs on the stuff I built when I got it last year (it hasn't rebooted yet).
15:43mazI'll get to try it then.,
15:44mazuptime
15:44maz 16:43:21 up 260 days, 23:40, 5 users, load average: 0.02, 0.08, 0.12
15:44kettenisand you call yourself a kernel developer? ;)
15:45mazhey, I have 3 other M1/M2 to play with! :D
15:46maz(and no, I don't call myself a kernel developer anymore -- more like a part time tinkerer...)
15:57martinr1 has quit [Ping timeout: 480 seconds]
16:12brolin has quit [Ping timeout: 480 seconds]
16:21brolin has joined #asahi-dev
16:25korreckj328 has quit [Quit: Leaving]
16:29brolin has quit [Ping timeout: 480 seconds]
16:37brolin has joined #asahi-dev
16:46brolin has quit [Ping timeout: 480 seconds]
16:54brolin has joined #asahi-dev
17:01stipa has quit [Ping timeout: 480 seconds]
17:03brolin has quit [Ping timeout: 480 seconds]
17:11brolin has joined #asahi-dev
17:15drubrkletern has joined #asahi-dev
17:17bps has quit [Ping timeout: 480 seconds]
17:26brolin has quit [Ping timeout: 480 seconds]
17:42bisko has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
17:43bisko has joined #asahi-dev
17:57hightower2 has quit [Ping timeout: 480 seconds]
18:00bisko has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
18:04bisko has joined #asahi-dev
18:11bisko has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
18:12roxfan2 has joined #asahi-dev
18:12bisko has joined #asahi-dev
18:15bisko has quit []
18:16roxfan has quit [Ping timeout: 480 seconds]
18:32sarucchi has quit [Ping timeout: 480 seconds]
18:37nela has joined #asahi-dev
18:48salimterryli has joined #asahi-dev
18:49SalimTer- has quit [Ping timeout: 480 seconds]
18:54gladiac has quit [Quit: k thx bye]
19:17marcanat least today's distraction had me find more stuff :p
19:17marcansome of those chicken bit names are fun
19:26j`ey(SYS_IMP_APL_SIQ_CFG_EL1)
19:29marcanbeats me
19:29marcanthat's what it's called though
19:30j`eySmart Interrupt? :p
19:39jannaudcpext seems to be stable when this display is powered over usb-c
20:10svenI’ve started to work on a uh.. hack that bypasses most of typec/altmode for now
20:24jannausounds sensible, I think I hacked typec/displayport to point of breaking usb for better dp reliability
20:29nela has quit [Read error: Connection reset by peer]
20:30nela has joined #asahi-dev
20:54compassion4 has joined #asahi-dev
20:55compassion47 has joined #asahi-dev
20:55compassion4 has quit []
20:57compassion has quit [Ping timeout: 480 seconds]
20:57compassion47 is now known as compassion
21:08eiln has joined #asahi-dev
21:08nela has quit [Remote host closed the connection]
21:08nela5 has joined #asahi-dev
21:08nela5 is now known as nela
21:08nela has quit []
21:09nela has joined #asahi-dev
21:16eilnthe problem w/ t6000 seems to be:
21:16eilnTTY> pmgr: timeout while trying to set mode f for device at 0x28e0802c8: 10f
21:16eilnwhich seems to be the PMGR_WAS_PWRGATED flag
21:17eilnthe updated experiments/ane_t600x_power.py in the ane-power branch (https://github.com/eiln/m1n1.git) tries to manually powergate ane_sys_cpu before turning it on
21:17eilnplease lmk if anyone w/ a t600x ane0 can test
21:35jannaueiln: seems to work
21:36eilnjannau: do you mind attaching the mon.poll()? the other one wasn't fully on then
21:40eilnjannau: ah finally looks right!! thank you sm :)
21:56brolin has joined #asahi-dev
22:18eilntake 2, lmk if experiments/ane.py under ane-t6000 branch works (either ane0/ane2)
22:19eilnif it does i should fix src/pmgr.c to handle pwrgating
22:33brolin has quit [Ping timeout: 480 seconds]
22:41jannaueiln: math checks out, there was "pmgr: timeout while trying to set mode f for device at 0x28e0802c8: b0f" on the first run on both ane0 and ane2
22:41jannau0x228e0802c8 for ane2
22:43brolin has joined #asahi-dev
22:43eilnjannau: woohoo! the first run is expected since it's not pwrgated yet
22:47cylm has joined #asahi-dev
22:48hightower2 has joined #asahi-dev
23:03drubrkletern has quit [Remote host closed the connection]
23:26nsklaus has quit [Ping timeout: 480 seconds]
…