#asahi-gpu · 2023-03-06 11:09
linaOK, I think I finally fixed this one ^^#asahi-gpu · 2023-03-06 10:38
linaI think I need to modify the C side for this, this is just broken, I have no idea how to safely wrap this API without duplicating job tracking...#asahi-gpu · 2023-03-06 10:35
linaNo, but that only cleans up completed jobs and detaches the callbacks, it doesn't actually free pending jobs because it assumes you want to restart the queue later...#asahi-gpu · 2023-03-06 10:28
linaMaybe drm_sched_stop() before killing the scheduler will do what I want...?#asahi-gpu · 2023-03-06 10:14
linaOkay, I can't put a reference to the scheduler in the job, because if it is the last reference dropping the scheduler from the job cleanup callback deadlocks ;;#asahi-gpu · 2023-03-06 10:13
linaYeeeah...#asahi-gpu · 2023-03-06 10:11
linaOhh... am I deadlocking by any chance?#asahi-gpu · 2023-03-06 10:03
linaThis is weeeird... the job cleanup callback gets called but the job doesn't get dropped sometimes?#asahi-gpu · 2023-03-06 09:55
linaThat could just be the context issue I mentioned though... but first I want to find out why I'm leaking jobs...#asahi-gpu · 2023-03-06 09:54
linaAnd this time I crashed RTKit ^^;;#asahi-gpu · 2023-03-06 09:35
linaNope, this is getting signaled... so why is the scheduler not cleaning this up?#asahi-gpu · 2023-03-06 09:24
linaAh, this could be a bad firmware interaction... I do know I invalidate context before waiting for jobs to complete, which is probably a bad idea. Maybe that just kills things and leaves the jobs dangling, never to complete.#asahi-gpu · 2023-03-06 09:03
linaWait no I do#asahi-gpu · 2023-03-06 09:03
linaOhhh wait, I think I'm never calling the entity cleanup function. Okay, that one's on me then...#asahi-gpu · 2023-03-06 09:01
linaWhy do I get the feeling this is drm_sched again... something like killing the entity stops jobs from being run, but doesn't cancel/free pending jobs...#asahi-gpu · 2023-03-06 08:52
lina... and I can't reproduce it now? ;;#asahi-gpu · 2023-03-06 08:46
linaOkay, I fixed the splat but I have another issue... I'm leaking slots somewhere with this workload, it runs out#asahi-gpu · 2023-03-06 08:24
linaLet's see if my fix worked...#asahi-gpu · 2023-03-06 08:24
linaOkay, reproduced the sched thing with something deliberate (8Kx8K glmarks getting killed in a loop) ^^#asahi-gpu · 2023-03-06 08:17
linaAnd GPU crashes are better behaved now, at least it doesn't just hang your system#asahi-gpu · 2023-03-06 08:17
linaI'll look at the piglit stuff next though I'm less worried about that since I think we've always had corner case GPU crash bugs (as much as I've tried to eliminate them...)#asahi-gpu · 2023-03-06 08:15
linaThis ownership/lifetime stuff is so subtle and completely undocumented in C APIs, it's such a mess T_T#asahi-gpu · 2023-03-06 08:15
linaI've added a reference from the job to the scheduler and I think that will fix it... since the job should never get destroyed until the scheduler cleans it up from its main loop, which can only happen when the fence gets signaled or fails, so that should mean the scheduler always outlives the job outlives the fence...#asahi-gpu · 2023-03-06 08:14
linaTo make things even more confusing, the job completion isn't via a job reference, it's via a fence...#asahi-gpu · 2023-03-06 08:13
linawas gone.#asahi-gpu · 2023-03-06 08:13
linaalyssa: I can't reproduce your splat but I think I know what happened. I thought there would be something in drm_sched to wait for in-progress jobs when a scheduler/entity gets destroyed, but it doesn't look like it... so you probably had a GPU job submitted, then the userspace process aborted, then the kernel freed all the scheduler stuff and when the GPU job completed it crashed because the scheduler #asahi-gpu · 2023-03-04 14:55
linaI think console=tty0 will fix it, otherwise just do what I do and echo 0 > /proc/sys/kernel/printk before doing anything interesting (dmesg still works) ^^#asahi-gpu · 2023-03-04 14:51
linaIn particular if you use linux.py, I think the default stdout-path gets set to serial (u-boot normally changes that to fb...)#asahi-gpu · 2023-03-04 14:49
lina(Or on how you actually boot linux)#asahi-gpu · 2023-03-04 14:49
linaDepending on your cmdline options you might be sending dmesg to serial anyway ^^#asahi-gpu · 2023-03-04 09:58
linaI'll fix that by bumping the timeout way up for those invals#asahi-gpu · 2023-03-04 09:58
linaThat doesn't explain the drm_sched splat you got though...#asahi-gpu · 2023-03-04 09:58
linaOne thing is the timestamps in your log suggest you have a slow dmesg consumer, maybe serial? So just printing fault messages takes a while, and that blocks the GPU, and then GPU context invals are getting dropped which can definitely crash the firmware.#asahi-gpu · 2023-03-04 09:56
linaalyssa: Do you have a log from the piglit thing? I don't know if I'll get to it today/tomorrow...#asahi-gpu · 2023-03-03 19:32
linanini~!#asahi-gpu · 2023-03-03 19:32
linaAnd I should get some sleep#asahi-gpu · 2023-03-03 19:32
linaGot it ^^#asahi-gpu · 2023-03-03 19:07
linaI'm not sure how bad that would be but I can see it being anywhere from a perf issue to a deadlock issue.#asahi-gpu · 2023-03-03 19:07
linaalyssa: Make sure to keep the writer_syncobj clear in that path somehow, that's important (without it you could end up importing the wrong fence on export, we don't want to import if there is no current writer)#asahi-gpu · 2023-03-03 18:56
linaStory of explicit sync: cargo cult code from other drivers then realize they're all broken ^^ (first panfrost, then iris/xe)#asahi-gpu · 2023-03-03 18:55
linaOh yeah the fence stuff is definitely broken in panfrost (at least the missing null checks in fence_reference) FYI#asahi-gpu · 2023-03-03 18:52
linaYou get to figure this one out ^^#asahi-gpu · 2023-03-03 18:51
linaI had enough crazy memory safety issues today... (see MR, you have no idea how long I spent tracking down that state tracker/blitter interaction one and deciding how to fix it...)#asahi-gpu · 2023-03-03 18:12
linaAlso I think iris has the magenta rectangles bug, I don't see anything in that MR to address the case that fixed it ^^#asahi-gpu · 2023-03-03 18:09
linaMaking it work with the implicit sync world on the other side on the other hand...#asahi-gpu · 2023-03-03 18:09
linaExplicit sync was easy...#asahi-gpu · 2023-03-03 18:09
linaYes, also I hate implicit sync ^^;;;#asahi-gpu · 2023-03-03 18:07
linaDidn't repro on stk at all for me but it did on nautilus ^^#asahi-gpu · 2023-03-03 18:01
linaDidn't get around to reviewing MRs, sorry... (nor to addressing all the feedback on the uapi-prep one, though at least I pushed updates to it with the bugfixes)