2023-04-25
00:59nsklaus has quit [Ping timeout: 480 seconds]
01:08nsklaus has joined #asahi-dev
01:11korreckj328 has quit []
01:17nsklaus has quit [Ping timeout: 480 seconds]
01:20nsklaus has joined #asahi-dev
01:29nsklaus has quit [Ping timeout: 480 seconds]
01:38nsklaus has joined #asahi-dev
01:46nsklaus has quit [Ping timeout: 480 seconds]
01:55nsklaus has joined #asahi-dev
01:57kesslerd1pont has quit [Ping timeout: 480 seconds]
01:58rhysmdnz has joined #asahi-dev
02:00gabuscus has quit []
02:03nsklaus has quit [Ping timeout: 480 seconds]
02:06rhysmdnz has quit [Quit: Bridge terminating on SIGTERM]
02:06rhysmdnz has joined #asahi-dev
02:12nsklaus has joined #asahi-dev
02:14gabuscus has joined #asahi-dev
02:15Jamie has joined #asahi-dev
02:16kesslerdupont has joined #asahi-dev
02:16Jamie is now known as Guest12081
02:17SalimTer- has joined #asahi-dev
02:18salimterryli has quit [Ping timeout: 480 seconds]
02:20nsklaus has quit [Ping timeout: 480 seconds]
02:24kesslerdupont has quit [Ping timeout: 480 seconds]
02:28nsklaus has joined #asahi-dev
02:41kesslerdupont has joined #asahi-dev
02:41nsklaus has quit [Ping timeout: 480 seconds]
02:49kesslerdupont has quit [Ping timeout: 480 seconds]
02:50nsklaus has joined #asahi-dev
02:59nsklaus has quit [Ping timeout: 480 seconds]
03:08nsklaus has joined #asahi-dev
03:17nsklaus has quit [Ping timeout: 480 seconds]
03:25nsklaus has joined #asahi-dev
03:35nsklaus has quit [Ping timeout: 480 seconds]
03:44nsklaus has joined #asahi-dev
03:50salimterryli has joined #asahi-dev
03:51SalimTer- has quit [Ping timeout: 480 seconds]
03:52nsklaus has quit [Ping timeout: 480 seconds]
04:01nsklaus has joined #asahi-dev
04:09nsklaus has quit [Ping timeout: 480 seconds]
04:17nsklaus has joined #asahi-dev
04:26nsklaus has quit [Ping timeout: 480 seconds]
04:34nsklaus has joined #asahi-dev
04:42nsklaus has quit [Ping timeout: 480 seconds]
04:52nsklaus has joined #asahi-dev
05:01nsklaus has quit [Ping timeout: 480 seconds]
05:08nsklaus has joined #asahi-dev
05:17nsklaus has quit [Ping timeout: 480 seconds]
05:22salimterryli has quit [Remote host closed the connection]
05:24salimterryli has joined #asahi-dev
05:26nsklaus has joined #asahi-dev
05:35nsklaus has quit [Ping timeout: 480 seconds]
05:42nsklaus has joined #asahi-dev
05:51nsklaus has quit [Ping timeout: 480 seconds]
05:55kesslerd1pont has joined #asahi-dev
06:01nsklaus has joined #asahi-dev
06:03kesslerd1pont has quit [Ping timeout: 480 seconds]
06:10nsklaus has quit [Ping timeout: 480 seconds]
06:18nsklaus has joined #asahi-dev
06:26nsklaus has quit [Ping timeout: 480 seconds]
06:33nsklaus has joined #asahi-dev
06:42nsklaus has quit [Ping timeout: 480 seconds]
06:43bps has quit [Ping timeout: 480 seconds]
06:48MajorBiscuit has joined #asahi-dev
06:50nsklaus has joined #asahi-dev
06:58nsklaus has quit [Ping timeout: 480 seconds]
06:59MajorBiscuit has quit [Quit: WeeChat 3.6]
07:08nsklaus has joined #asahi-dev
07:11jannaudoes pmgr-misc exist on die 1? I'm getting transalation faults on 0x228e20c000 and 0x228e20c800 on m1 ultra
07:11bps has joined #asahi-dev
07:15MajorBiscuit has joined #asahi-dev
07:16nsklaus has quit [Ping timeout: 480 seconds]
07:18jannauI see dart translation faults for 583008000.iommu on s2idle and the nvme didn't boot on the second resume
07:23jannauSErrors but the HV complains first https://gist.github.com/jannau/526d76696ab10f0ac2308151fc68d975
07:25nsklaus has joined #asahi-dev
07:25chadmed has quit [Remote host closed the connection]
07:33nsklaus has quit [Ping timeout: 480 seconds]
07:34kesslerdupont has joined #asahi-dev
07:38marcanI'm guessing that is a SoC-global block and those clocks/pstates also forward to die1 automatically then
07:39jannaudid the sleep/shutdown tracing made the system slower under the hypervisor? boot timing seems to be much slower than I remember and macos felt slow as well
07:42kesslerdupont has quit [Ping timeout: 480 seconds]
07:42nsklaus has joined #asahi-dev
07:43marcanIIRC the Ultra was always kind of slow under the HV though, but there may be some pathological issue there
07:51nsklaus has quit [Ping timeout: 480 seconds]
07:58Z750 has quit [Quit: Ping timeout (120 seconds)]
07:59Z750 has joined #asahi-dev
07:59nsklaus has joined #asahi-dev
08:11nsklaus has quit [Ping timeout: 480 seconds]
08:13marcanhm, it could be cpufreq is broken in the guest with clpc=0 and we don't bump the pcores, yeah
08:14marcanalso I haven't tested on ultra but at least on M2 Max macos seemed to just... not use the pcores?
08:21nsklaus has joined #asahi-dev
08:29nsklaus has quit [Ping timeout: 480 seconds]
08:30nsklaus has joined #asahi-dev
08:35cylm has joined #asahi-dev
10:05kesslerdupont has joined #asahi-dev
10:13kesslerdupont has quit [Ping timeout: 480 seconds]
10:13chadmed has joined #asahi-dev
10:17chadmed has quit [Remote host closed the connection]
10:22chadmed has joined #asahi-dev
10:25c10l has quit [Quit: Bye o/]
10:26c10l has joined #asahi-dev
10:43jannaucpuidle makes linux boot under HV slow. with cpuidle.off=1 in the commandline it's 2-3 times faster
10:47nsklaus has quit [Quit: WeeChat 3.8]
10:55kesslerdupont has joined #asahi-dev
11:09kesslerdupont has quit [Ping timeout: 480 seconds]
11:14kesslerdupont has joined #asahi-dev
11:14jannaumarcan: it does, might be even a little faster than cbd9b7b9e1724244 with cpuidle disabled. thanks
11:31kesslerdupont has quit [Read error: Connection timed out]
11:36nsklaus has joined #asahi-dev
11:36jannauI see the same on ultra, macos (12.3) doesn't use the p-cores with clpc=0. without it does
11:40ChaosPrincess has quit [Quit: WeeChat 3.8]
11:41ChaosPrincess has joined #asahi-dev
11:42kesslerdupont has joined #asahi-dev
11:59kesslerdupont has quit [Quit: leaving]
12:17i509vcb has quit [Quit: Connection closed for inactivity]
12:19c10l has quit [Quit: Ping timeout (120 seconds)]
12:20c10l has joined #asahi-dev
13:24_jannau_axboe: I couldn't reproduce it (using HV with console_no_suspend and sddm as X client) so quite a few things which could explain different behavior
13:32axboelooks like X is still running, but my session is just gone and I get a lightdm login screen
13:33axboeI did update m1n1 and u-boot at the same time, I can try my previous kernel and see if it happens there too. was just assuming it's a kernel side thing with dcp
13:46_jannau_could be still the X server of your session which was killed. DCP shouldn't have changed much for existing installs
13:47axboeyeah, what I mean is that X itself didn't restart, it's still there and same pid, but something made it exit the session
13:47_jannau_main change was supporting the 13.3 firmware for m2 devices but is mostly separate from the existing support
13:48axboemight be xfce4 triggering it. I don't recall seeing those "display went away" on suspend before, but it could just be that I never looked
13:48_jannau_there should be be 2 X sessions after you logged in. the lightdm one and you user session
13:53axboeusually if I suspend it's because I'm heading to the office, so probably 20-30 minutes min suspend
14:04tobheaxboe: I think I have seen something similar before, even on wayland. out of curiosity: does your wifi still work after wakeup?
14:10axboewifi has always been a bit hit or miss for me. I'd say that 9/10 wakeups wifi is fine, 1/10 it doesn't come back and it needs a reboot
14:13tobhethe few times this has happened to me it was usually both at the same time so I assumed it might be gnome related
14:16Knedlik has joined #asahi-dev
14:17cylm has quit [Ping timeout: 480 seconds]
14:20cylm has joined #asahi-dev
14:40KnedlikHey guys, I would like to use C++ for the touchbar Fxx buttons I was tasked with earlier. What should I use? I assume the DRM has some headers, but what should I use for touch input?
14:46enron has quit [Quit: Ping timeout (120 seconds)]
14:46nopeslide13 has quit []
14:46enron has joined #asahi-dev
14:46nafod0 has joined #asahi-dev
14:48nafod has quit [Read error: Connection reset by peer]
14:48nafod0 is now known as nafod
14:49nopeslide13 has joined #asahi-dev
14:55KnedlikOkay I'm getting a bit confused reading through all the DRM-related stuff on the internet... how do I use it to draw on the touchbar?
15:05Knedlik has quit [Quit: Konversation terminated!]
15:06kesslerdupont has joined #asahi-dev
15:07marcanaxboe: we've long had that thing with systemd-journald crashing on long suspend due to some watchdog thing, but that's exactly what cpuidle should have fixed...
15:08_jannau_it didn't, systemd-journal still crashed for me after waking from 15 minutes of suspend
15:11bps has quit [Ping timeout: 480 seconds]
15:13kesslerdupontchadmed: I'll take a look today at speakersafetyd and see what I can do
15:16marcanbecause it basically triggers forever spurious wakeups, which will cause the kernel to repeatedly re-enable and re-disable timekeeping
15:24marcanyou also probably want hv.p.hv_set_time_stealing(0, 1) to prove that it isn't just time stealing warping time
15:28marcanso. not an issue but not what axboe is seeing either (assuming he wasn't running under the HV)
15:29MajorBiscuit has quit [Ping timeout: 480 seconds]
15:32rosefromthedead has joined #asahi-dev
15:40marcanaxboe: I think we're going to need more info/logs then... I don't have any good ideas of what to look for right now :/
15:43axboelet me boot the latest, and will be happy to grab whatever you'd want in terms of logs or messages
15:44_jannau_I've seen a dcp call ordering issue on wakeup but everything still worked. so probably some kind of race where at least one call is not needed
15:45bps has joined #asahi-dev
15:49marcanI'm adding an explicit timewarp op to the HV so I can test this, but if it really has to do with time elapsed I can only guess it's a userland bug with the calendar time jumping somehow
15:57marcanjust tested under the HV on the M1 Studio, warped time 10 hours forwards and nothing bad happened (kde on x11)
15:59axboethis is with current asahi-wip, running mesa from a month ago or so (updated whenever the api changed)
15:59marcancould be the GPU stuff is also breaking due to its power domain shutting down. can you add apple,always-on; to the ps_gfx power domain and see if that helps?
16:02marcanthough that might actually kind of break for die1, but that should only cause some warnings at most
16:05marcanI wonder if it's something like gfx-asc goes down unexpectedly, it has a backdoor pipe to SMC, SMC is sending it notifications for some reason, the FIFO fills up and SMC panics and reboots the machine
16:10nyx_o has quit [Ping timeout: 480 seconds]
16:12axboeI'm still doing the whole osx recovery thing for dtb updates like probably a moron (assuming there's a better way now)
16:17marcanso IIRC the SMC reboot watchdog is like 30 seconds or something, and if this is along those lines it might be crashing directly on suspend
16:18bps has quit [Ping timeout: 480 seconds]
16:18axboe2023-04-25T10:14:57.380543-06:00 m1max kernel: [ 33.664422] nvme-apple 393cc0000.nvme: RTKit: syslog message: cmd.c:7861: NVMe shutdown start seg->lba: 0, seg->size: 0
16:18axboe2023-04-25T10:14:57.380548-06:00 m1max kernel: [ 33.801967] nvme-apple 393cc0000.nvme: RTKit: syslog message: cmd.c:7874: seg->lba 0 saveCtx 1 took 137 ms
16:18axboe2023-04-25T10:14:57.380550-06:00 m1max kernel: [ 33.834543] macsmc-rtkit 290400000.smc: RTKit: syslog message: apComms.cpp:373: SMC HID Event: 03 00 01
16:18axboe2023-04-25T10:14:57.380552-06:00 m1max kernel: [ 33.834560] macsmc-hid macsmc-hid: Lid wakeup
16:18axboe2023-04-25T10:14:57.380554-06:00 m1max kernel: [ 33.835102] apple-pmgr-pwrstate 28e580000.power-management:power-controller@288: PS mca0: Failed to reach power state 0xf (now: 0x24f)
16:18axboe2023-04-25T10:14:57.380557-06:00 m1max kernel: [ 33.835223] apple-pmgr-pwrstate 28e580000.power-management:power-controller@290: PS mca1: Failed to reach power state 0xf (now: 0x24f)
16:45Jamie has joined #asahi-dev
16:45rhysmdnz1 has joined #asahi-dev
16:45Jamie is now known as Guest12142
16:49rhysmdnz has quit [Ping timeout: 480 seconds]
16:49Guest12081 has quit [Ping timeout: 480 seconds]
16:51axboeand again looks like suspend was the culprit, because it was sitting instantly in lightdm login when I opened the lid
16:53marcanI'm running out of ideas... AFAIK the coprocessors that can nuke the system like that are SMC and the CIO stuff (cc sven), but the latter shouldn't be enabled yet...
16:54marcanyeah but I don't think I made any changes that could cause this other than the cpuidle stuff :(
16:56marcanif cpuidle fixes it, it's probably unrelated. if it doesn't, try to get me another log where you get some stuff from resume (so at least we have a complete log of the suspend process)
16:57marcanyeah the result should be the same either way, I don't care if it doesn't crash at that point
17:16kesslerdupont has quit [Ping timeout: 480 seconds]
17:17kesslerdupont has joined #asahi-dev
17:21marcanadd that and remove the cpuidle.off and see if it repros, if it does either catch enough of a log or disable cpuidle again and catch it that way
17:23marcanthere's also one interesting thing we can try. set macOS as the default boot OS, do a one-time boot into Linux, repro, and see if macOS gives you any panic logs. I think it can catch certain system engine/management errors.
17:26axboebuilding with larger dmesg and not disabling cpuidle and doing those first, then we can try that if it still craps out without being able to capture anything
17:34svenor, well, I have seen that but only when I messed up the init sequence which shouldn’t happen with the Linux driver
17:35kesslerdupont has quit [Ping timeout: 480 seconds]
17:37axboe55 seconds is probably roughly the time it takes to power cycle, and go through all of boot
17:40axboethere's a ton of stuff before the entering suspend, let me pastebin it in case there's anything interesting
17:40marcanas long as I can see the whole suspend cycle (i.e. there should be one line of wakeup, otherwise you probably didn't get anything) we should be good
17:41axboebut there is no full cycle, it looks like it's power cycling right after having entered suspend
17:46lebakassemmerl has joined #asahi-dev
17:46lebakassemmerl has quit [Remote host closed the connection]
17:49marcanok, so you have a fairly short list of power domains that shut down between `Suspending console(s)` and `Lid wakeup`
17:50marcanI really need to get some sleep, but if you want to continue debugging on your own: make all of those always-on, and see if that fixes things. if it does, manually bisect (though really gpio is the only one there that I'd even slightly suspect)
17:50marcanI also pushed a thing to the linux.git:pmgr-stuff branch that you might want to cherry pick and see if it helps
17:51marcanif making everything on that list always-on doesn't fix it either, then we have some weirder problem where cpuidle is broken, but it somehow only triggers during sleep for unknown reasons (e.g. because it sleeps more then, or because other stuff in the system is going lower power anyway)
17:52marcanbasically my theory right now is that the CPUs going into deep idle is triggering some machinery that we otherwise haven't triggered yet, and it hates something we're doing, and something critical crashes and takes down the machine
17:55marcanthis *may* be related to how I couldn't get the CPUs properly fully shut down (with loss of MMIO) but could when I let macOS do it
17:56marcanaxboe: so basically if making everything always-on doesn't help, do the macos thing and whatever the result of that is, I think that's as much as you can do at this point
18:05Cyrinux9 has quit []
18:08Cyrinux9 has joined #asahi-dev
18:16WindowPain has joined #asahi-dev
18:21WindowPa- has quit [Ping timeout: 480 seconds]
18:23MajorBiscuit has joined #asahi-dev
18:31cylm has quit [Ping timeout: 480 seconds]
19:41jannaucan you think of anything out of the ordinary? you're using xfce4 if I remember correctly and I gather you suspend/resume by closing/opening the lid
20:06deteg1337 has joined #asahi-dev
20:06deteg1337 has quit []
20:22pharonix71 has quit [Read error: No route to host]
20:27flying_sausages has quit []
20:38flying_sausages has joined #asahi-dev
20:41kesslerdupont has joined #asahi-dev
21:05bluetail has quit [Quit: The Lounge - https://thelounge.chat]
21:22Z750 has quit [Quit: Ping timeout (120 seconds)]
21:22Z750 has joined #asahi-dev
21:38zzywysm has joined #asahi-dev
22:03kesslerdupont has quit [Ping timeout: 480 seconds]
22:16WindowPain has quit [Quit: ZNC 1.8.2 - https://znc.in]
22:17WindowPain has joined #asahi-dev
22:33kesslerdupont has joined #asahi-dev
22:43psykose_ has joined #asahi-dev
22:46psykose has quit [Ping timeout: 480 seconds]
22:54kesslerdupont has quit [Ping timeout: 480 seconds]
23:23bluetail422 has joined #asahi-dev
23:24bluetail422 has quit []
23:24bluetail422 has joined #asahi-dev
23:25bluetail422 has quit []
23:25bluetail422 has joined #asahi-dev
23:26bluetail422 has quit []
23:26bluetail422 has joined #asahi-dev
23:26bluetail422 has quit []
23:27bluetail422 has joined #asahi-dev
23:27bluetail422 has quit []
23:28bluetail422 has joined #asahi-dev
23:28bluetail422 has quit []
23:29bluetail422 has joined #asahi-dev
23:29bluetail422 has quit []
23:30bluetail422 has joined #asahi-dev
23:31kesslerdupont has joined #asahi-dev
23:31abd has joined #asahi-dev
23:32bluetail422 has quit []
23:33bluetail422 has joined #asahi-dev
23:34bluetail422 has quit []
23:35bluetail422 has joined #asahi-dev
23:35bluetail422 has quit []
23:35bluetail422 has joined #asahi-dev
23:36bluetail422 has quit []
23:36bluetail has joined #asahi-dev
23:38bluetail has quit []
23:38bluetail has joined #asahi-dev
23:38bluetail has quit []
23:39bluetail has joined #asahi-dev
23:39bluetail has quit []
23:40bluetail has joined #asahi-dev
23:41bluetail has quit []
23:51abd has quit [Ping timeout: 480 seconds]
23:53bluetail has joined #asahi-dev
23:57bluetail has quit [Remote host closed the connection]
23:58bluetail has joined #asahi-dev