#
…

2023-06-29

00:02alyssafedora is my new love
00:02* alyssagot the valgrind symbols
00:06jlco has joined #asahi-gpu
00:10darkapex has joined #asahi-gpu
00:28nsklaus_ has quit [Remote host closed the connection]
00:31alyssakicking off another run with the leak fixes
00:32alyssalina: jannau: we should coordinate who's doing which hardware
00:32alyssaI'll supply classic M1
00:32alyssajannau: said you can do M1 Max/Ultra/M2
00:33alyssalina: that leaves you with M1 Pro, if we even want a separate run for that?
00:47mini_ has quit [Quit: ZNC closing...]
00:51mini_ has joined #asahi-gpu
00:51alyssaOh come on. Failed in the exact same place.
00:51alyssaevidently did not fix the leaks hard enough
00:54mini_ has quit []
00:56alyssalina: This is interesting
00:57mini_ has joined #asahi-gpu
00:57alyssaIf I `./glcts -n dEQP-GLES31.functional.*`, I get "collect_garbage: failed to reserve space" in under 5 minutes
00:57alyssa(Well before it gets to the spicy Big Framebuffer test that causes the run to fail.)
00:57alyssaand this is despite low, ~constant memory usage according to top.
00:59mini_ has quit []
01:01mini_ has joined #asahi-gpu
01:11cylm has joined #asahi-gpu
02:03thevar1able_ has quit [Remote host closed the connection]
02:03thevar1able_ has joined #asahi-gpu
03:01c10l7 has joined #asahi-gpu
03:07c10l has quit [Ping timeout: 480 seconds]
03:08pyropeter3 has joined #asahi-gpu
03:10pyropeter2 has quit [Ping timeout: 480 seconds]
03:26linaI can do M2 and M1 Pro, which given our call earlier sounds like all we need with your M1? Or if jannau runs M2 and M1 Max, that covers it as well, or any other combination.
03:26lina(We agreed we only need one each of t8103, t600x, t8112, and eventually t602x)
03:28linaAlso conclusion about that OOM thing is that it was the 4K kernel... which is not very surprising since it has known bugs.
03:29linaThe actual driver codepath there is just a GFP_KERNEL alloc with lots of memory free/cached is failing, which is definitely not a problem with the driver itself, but I wouldn't be surprised if the 4K/16K shmem thing is breaking memory management...
04:38lina has quit [Quit: Lost terminal]
04:44possiblemeatball has joined #asahi-gpu
05:30possiblemeatball has quit [Quit: Quit]
05:51lina has joined #asahi-gpu
05:53jannauI was under the impression that we wanted to test t6000/t6001/t6002 separately under the assumption that we effectively test Apple's firmware as well and probably hit different execution paths
05:54jannauI'd be surprised if "there are twice as many clusters" is the only difference on the ultra
06:10jannaulina: piglit run deqp_gles3 seems to hang at test 11377, using VK-GL-CTS opengl-es-cts-3.2.9 DEQP_TARGET=wayland on m1 ultra
06:10xcpy0 has quit [Quit: Ping timeout (120 seconds)]
06:11jannaukernel includes yesterday's drm sched fix
06:11xcpy0 has joined #asahi-gpu
06:11jannauseems related to "asahi 406400000.gpu: QueueJob 651041: Job timed out on the DRM scheduler, things will probably break (ran: true)"
06:12jannauare "Unknown event message" interesting? There is one (reproducible) ~100 seconds prior
07:25jannau'deqp-gles3/functional/shaders/builtin_functions/precision/matrixcompmult/highp_fragment/mat3' seems to be the hanging test. I do not want to disturb the cts run so I will try to confirm that later
07:29cylm has quit [Ping timeout: 480 seconds]
07:48jannautest passes in the cts run with alyssa's es31 branch and VK-GL-CTS opengl-es-cts-3.2.9.3 with x11_egl
07:49jannauso it could be either a mesa driver bug, a wayland wsi bug or a VK-GL-CTS bug
08:04linajannau: "Unknown event" is something we can't handle, usually dynamic TVB allocation or similar, so it's no surprise it would hang after that.
08:05linaIs it reproducible if you increase asahi.initial_tvb_size=64 or something like that?
08:06linaI think the logic to compute the minimum initial TVB size is not complete
08:07linaDo you know if the hanging test uses MSAA? I wouldn't be surprised if I need a *sample_count factor in there or something
08:09linaRe CTS runs, I think it's largely the same execution paths for all of a series, plus it's not like we're submitting conformance separately for each firmware version either...
08:09linaI think Ultra really is mostly the same execution path when it comes to rendering, just initialization is probably different.
08:12linaAlso I know the initial TVB size is different with clustering, so it wouldn't surprise me if this breaks on Ultra only (or on the whole Pro/Max/Ultra series)
08:16i509vcb has quit [Quit: Connection closed for inactivity]
08:26mkurz has quit [Ping timeout: 480 seconds]
08:31nsklaus_ has joined #asahi-gpu
08:35lina has quit [Ping timeout: 480 seconds]
08:35jannauall precision/matrixcompmult/highp_fragment/mat3* tests fail but mat2* / mat4* don't, I doubt the tests use MSAA
08:42jannautests pass with asahi.initial_tvb_size=64
08:49mkurz has joined #asahi-gpu
08:54chadmed has quit [Remote host closed the connection]
09:28c10l7 has quit []
10:28c10l7 has joined #asahi-gpu
10:29c10l7 has quit []
10:30c10l7 has joined #asahi-gpu
10:41cylm has joined #asahi-gpu
10:57kode54 has left #asahi-gpu [#asahi-gpu]
10:58kujeger has quit [Quit: ZNC 1.8.2 - https://znc.in]
11:08chadmed has joined #asahi-gpu
11:32mkurz has quit [Ping timeout: 480 seconds]
11:46c10l7 has quit []
12:00c10l7 has joined #asahi-gpu
12:43lina has joined #asahi-gpu
12:56thelounge60655 has joined #asahi-gpu
13:00linajannau: Okay, so it's the TVB size thing... what machine is this?
13:02thelounge6065 has quit [Ping timeout: 480 seconds]
13:02thelounge60655 is now known as thelounge6065
13:02kujeger has joined #asahi-gpu
13:03_jannau__lina: mac studio m1 ultra, 48 gpu cores
13:04_jannau__cts results are worse on t6002 compared to t8103, testing now t6001
13:05_jannau__that is with asahi.initial_tvb_size=64 and without unknown event messages
13:41linaWhat is failing?
13:42mkurz has joined #asahi-gpu
13:55jannaulina: quite a few atomic_counter tests
13:56jlco has quit [Ping timeout: 480 seconds]
13:56jannauresults on m1 max looked very similar or identical
13:56jlco has joined #asahi-gpu
14:05tirr has joined #asahi-gpu
14:07jannauat least this ones: https://paste.debian.net/1284465/ - rerunning the cts atm due to mesa config misshap on my side
14:09alyssajannau: Those tests at the end are interesting
14:09alyssaThey're all compute shader tests, but tests with multiple invocations/groups
14:09alyssathere are also single invocation/group versions of all those, which I assume are passing
14:10alyssaAll of these tests use atomics.
14:14jannauyes, the other tests in the same group fail, also only ~60% of the atomic_counter tests with multiple threads fail
14:15jannauerr, the other compute shader tests in the same group pass
14:22alyssathis really could be either Mesa or kernel, tbh
14:22jannauis it expected that dEQP-GLES3.functional.flush_finish.flush_wait takes a long time (several seconds) and makes desktop use slow?
14:23alyssaflush_finish.* are the stupid tests that we skip in CI
14:26alyssathinking either the kernel is configuring the CDM badly on t600x, or Mesa is doing something silly, or..
14:26alyssaOr, geez, I wonder
14:27alyssaI wonder if compute kernels with atomics need to be pinned to a single cluster for correct results
14:27alyssaor need to have barriers on multi-cluster parts that aren't needed on single-cluster parts
14:27alyssabecause the atomics aren't coherent across clusters
14:27alyssaand so the compiler needs to report whether atomics are used, and if so either
14:27alyssa1. add more barriers?
14:28alyssa2. set a flag in the CDM dispatch structs forcing things to stay together?
14:28alyssa3. pass a flag to the kernel telling the kernel to pass a flag to the hardware to force things together?
14:28alyssathe way to go about checking this is to diff the agxdecode on t600x for a compute kernel without atomic + compute kernel with
14:28alyssaand see if the shaders diff (other than adding an atomic, obviously) and/or if the structures differ
14:36alyssa has quit [Quit: leaving]
14:43alyssa has joined #asahi-gpu
14:44alyssa_ has joined #asahi-gpu
14:46alyssa_ has left #asahi-gpu [#asahi-gpu]
14:46alyssa_ has joined #asahi-gpu
14:46alyssa_alyssa: bar
14:46alyssaok, that does what I want
14:49jannaualyssa: btw "static_assert(len_words == 4, "64-bit pointer");" doesn't compile on arch since gcc can't see that len_words compile time constant is
14:49alyssajannau: dammit
14:49jannauin the case we need commit accurate mesa version information for cts
14:49alyssaalright.
14:49alyssawill fix that
14:54jannaum2 seems to hang on 'dEQP-EGL.functional.resize.surface_size.shrink'
15:01alyssahang how?
15:03jannaucts-runner doesn't progress for at least 3 minutes, with the last printed test dEQP-EGL.functional.resize.surface_size.shrink
15:19alyssa has quit [Quit: alyssa]
15:19alyssa_ has quit [Remote host closed the connection]
15:22alyssa has joined #asahi-gpu
15:23alyssa_jannau__: curious
15:39jannaupiglit deqp_gles31 results on t8112: 8x fail, 7x crash https://paste.debian.net/1284478/
15:49dylanchapell has joined #asahi-gpu
15:59jannaupiglit deqp_egl results on t8112: 1x fail, 1x crash, 2x hangs (both surface_size "crashes"( https://paste.debian.net/1284481/
16:47alyssawhy piglit btw?
16:48alyssaI have `run-deqp31` aliased to
16:48alyssaMESA_SHADER_CACHE_DISABLE=1 EGL_PLATFORM=surfaceless deqp-runner run --deqp ~/GLCTS/glcts --output output --jobs 8 --renderer-check G13 --caselist ~/GLCTS/gl_cts/data/mustpass/gles/aosp_mustpass/main/gles31-master.txt --skips ~/mesa/.gitlab-ci/all-skips.txt -- --deqp-surface-type=pbuffer --deqp-gl-config-name=rgba8888d24s8ms0 --deqp-surface-width=256 --deqp-surface-height=256
16:48alyssa--deqp-log-images=disable --deqp-log-shader-sources=disable
16:49alyssa_ has joined #asahi-gpu
16:49alyssa_alyssa: bloop
16:49alyssa_ has quit []
16:50alyssajannau: as for the deqp-gles31 results--
16:50alyssathat is passing to me
16:50alyssaand run-deqp31 would indicate that as passing
16:50alyssathe image atomic comp swap return value tests are known broken and not included in the mustpass list, piglit is only running them because piglit is wrong
16:51alyssaand the stress tests are also not included in mustpass, only the functional ones contribute to conformance
16:51alyssait'd be great if we passed stress too but it's not part of the CTS currently
16:56jannaubecause I have a bad memory, it's noted somewhere in the developer section on the website and it's at least nicer than running it from vk-gl-cts
16:57alyssaanholt's cts runner is a lot newer than piglit tbf
16:57jannaui.e because I don't know better
16:59alyssatoday's lucky 10,000
16:59alyssaor like lucky 1 tbf
17:00dylanchapell_ has joined #asahi-gpu
17:35jannaudo the configs using "gles/khronos_mustpass*" have much additional test coverage over aosp {egl,gles2,gles3,gles31} or are the latter a good approximation of a passed cts run?
17:36jannauwith deqp-runner only dEQP-EGL.functional.resize.surface_size.{shrink,stretch_width} remain as timeout/hang
17:36jannauon t8112
17:38alyssajannau: IIRC the khronos_mustpass is for KHR-GLES* tests
17:38alyssathe aosp_mustpass is for dEQP tests
17:38alyssathe CTS run contains both
17:39alyssajannau: and woo
17:39alyssaI wonder what that's about
17:40alyssajannau: to confirm that's with x11-egl?
17:40alyssaeric_engestrom: ^^ any ideas?
17:40jannauyes, I'm just wondering if it makes sense to prepare deqp-runner cmdlines for the KHR-GLES* tests
17:41alyssaOh. Yeah, it is
17:41alyssathe glcts binary contains all of them
17:41alyssaso if you want you can `cat` together the different mustpass files for a big combined run
17:41* alyssashould probably do that
17:42alyssafwiw there are a lot more deqps than khr-gles's, so it's not adding much wall clock time to run the KHR tests too
17:42jannaudon't they all use different options
17:43jannauthey are repeated with different --deqp-gl-config-id options
17:49jannauthey seem to wait on (resize?) events which either are not send or arrive too early
18:00jannauafter looking at the code it might be a fractional scaling issue
18:01jannauI'll start a full CTS run on t8112
18:07alyssa17:43 jannau | they are repeated with different --deqp-gl-config-id options
18:07alyssaoh, yeah
18:07alyssaYeah, I don't think that's worth it to replicate with deqp-runner
18:07alyssaI just meant running the khr tests at all (with a standard config)
18:33dylanchapell has quit [Ping timeout: 480 seconds]
18:45c10l7 has quit []
18:50c10l7 has joined #asahi-gpu
18:55alyssaruh roh why am I seeing faults trying to open keyboard shortcuts in gnome
18:55alyssaone bug at a time. beh
18:55alyssaoh that's probably the thing Lina just fixed
19:03c10l7 has quit []
19:04c10l has joined #asahi-gpu
19:13bisko has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
19:50cylm has quit [Ping timeout: 480 seconds]
19:59adryzz has joined #asahi-gpu
20:40adryzz has quit [Quit: .]
21:41possiblemeatball has joined #asahi-gpu
21:49brolin has joined #asahi-gpu
21:51brolin has quit []
22:37alyssa has left #asahi-gpu [#asahi-gpu]
22:37alyssa has joined #asahi-gpu
22:47abd has joined #asahi-gpu
22:56jlco_ has joined #asahi-gpu
22:57i509vcb has joined #asahi-gpu
22:57jlco has quit [Ping timeout: 480 seconds]
23:26abd has quit [Ping timeout: 480 seconds]
23:50alyssajannau: I've just pushed 2c0d4fc71fb ("DONOTMERGE: agx: Don't emit wait_pix multiple times")
23:50alyssaWith that, I can't seem to repro the Fail, although I can still hit timeouts
23:50alyssa---Fuck, never mind, just got the fail
23:50alyssaignore me.
23:51alyssaalthough seemingly the fail might've been carried over
23:53alyssaI do find this flake concerningly difficult to repo.
…