#
#asahi-gpu · 2022-10-28 04:47
phirethe bottom one might be to do with inter-die work sharing
#asahi-gpu · 2022-10-28 04:46
phirethat might actually be three common areas. One at the top, one in the middle, one at the bottom. top and bottom seem to be diffrent
#asahi-gpu · 2022-10-28 04:46
linaAnd on the M1 Max, the little block on die 1 would be unused, everything else would be used, with the tiling/sorting stuff coordinating via the extra buffers
#asahi-gpu · 2022-10-28 04:45
linaSo I'm guessing that little block is the top-level buffer management / dispatch stuff, the per-cluster structures are the actual tiling/sorting/whatever, and then there's the actual cores
#asahi-gpu · 2022-10-28 04:45
linaIf you look at the die shot, there are 8 compute units per cluster and then a bunch of structures that are also per cluster. And then there's one common area that is global in the M1 Pro, and not even symmetric in the M1 Max
#asahi-gpu · 2022-10-28 04:45
phireor draw VBO if that's enabled. not sure which path is used by default
#asahi-gpu · 2022-10-28 04:44
phireyeah, the mesh is just a single DrawArrays call
#asahi-gpu · 2022-10-28 04:43
linaI imagine that all that logic comes from die 0, and is unused in die 1, only the actual compute units and tiling output logic are used in die 1
#asahi-gpu · 2022-10-28 04:43
linaAnd I mean it has to balance across cores anyway... balancing across clusters just means, I'm guessing, extra handling for tile maps and buffer management metadata
#asahi-gpu · 2022-10-28 04:42
linaI think so, since I think the glmark2 bunny is a single draw?
#asahi-gpu · 2022-10-28 04:41
linamacOS only ever uses either 1 cluster rolling, or all clusters, and it doesn't seem to be per client at least. Might be a heuristic.
#asahi-gpu · 2022-10-28 04:41
phireinteresting, so it's splitting vertex workloads at a pretty fine granularity. Within a single draw?
#asahi-gpu · 2022-10-28 04:41
linaBut I can't get it to actually run multiple jobs across clusters like this, so I think this is just some kind of power/load spreading for simple jobs, like the compositor
#asahi-gpu · 2022-10-28 04:41
linaAlso, for some queues/jobs, macOS does just that, rolling the cluster it picks
#asahi-gpu · 2022-10-28 04:40
linaSame as M1
#asahi-gpu · 2022-10-28 04:40
linaIf I restrict it to just one cluster, the buffers are not used at all (any cluster will do)
#asahi-gpu · 2022-10-28 04:40
linaFancy.
#asahi-gpu · 2022-10-28 04:40
linaI can restrict which clusters it uses for vertex via a mask, and it slows down if I do that
#asahi-gpu · 2022-10-28 04:40
linaIt can, it's what these buffers are for.
#asahi-gpu · 2022-10-28 04:39
phireif the guranuality is that small, I wonder if it can split vertex workloads across clusters. Or if vertex workloads are limited to a single cluster
#asahi-gpu · 2022-10-28 04:38
linaOhh I found the stride... and yup, I have it wrong, I remember a factor of 4 I wondering about here. This also makes sense, I remember messing with this one on M1 and it not mattering...
#asahi-gpu · 2022-10-28 04:37
phirescrolls down to correct die shot
#asahi-gpu · 2022-10-28 04:37
lina8 cores per cluster
#asahi-gpu · 2022-10-28 04:37
phireso it's 4 cores per cluster?
#asahi-gpu · 2022-10-28 04:36
linaMaybe it's a spill thing or maybe I have a stride setting wrong somewhere
#asahi-gpu · 2022-10-28 04:36
linaI don't know where the 4 comes from, it seems the last 3/4 of each block end up unused...
#asahi-gpu · 2022-10-28 04:36
linaAnd then of the new tile buffers, most are fixed size * cluster count it seems, but one seems to be aux tilemaps, and the size is the tilemap size * 4 * nclusters
#asahi-gpu · 2022-10-28 04:35
phireyeah, I was supicious about the actual "gpu count" based on some of the initdata
#asahi-gpu · 2022-10-28 04:35
lina(deflake = preempt, I really need to replace that wording)
#asahi-gpu · 2022-10-28 04:35
linaAs for the deflake buffers, for some inexplicable reason macOS allocates them as exactly *9x* the M1 sizes... but with 8x it works, so maybe that's an off-by-one on their side (or they were padding for debugging)
#asahi-gpu · 2022-10-28 04:35
linaIt's just M1 doesn't need it since it is one cluster
#asahi-gpu · 2022-10-28 04:34
linaSo they planned for this scalability from the get go, it seems
#asahi-gpu · 2022-10-28 04:34
linaAll that stuff is also used on M1 Pro and Max
#asahi-gpu · 2022-10-28 04:34
linaI've been looking at the new buffer stuff. The buffers are sized based on the core count, and are used even if I turn off half the cores. So it seems the "2 GPUs" is a red herring. It's 8 GPUs!
#asahi-gpu · 2022-10-28 04:34
linaboth ^^
#asahi-gpu · 2022-10-28 04:34
phireupstream linux, or upstream rust-on-linux?
#asahi-gpu · 2022-10-28 04:33
linaIf upstream doesn't like it I can always replace it with something else
#asahi-gpu · 2022-10-28 04:33
phirefair
#asahi-gpu · 2022-10-28 04:33
linaphire: Performance really doesn't matter, and I need input/output in f32 format... I think I'm just going to wrap the existing kernel softfloat stuff for now
#asahi-gpu · 2022-10-27 22:02
phireWould allow you pass it around (and even compare) as if it was a 64bit int, but then the compiler can quickly unpack it into it's components just by just treating it as two 32bit ints
#asahi-gpu · 2022-10-27 21:59
phireLina: consider using a soft float working format of 64bit, sign in bit63, 8/16 bits of exponent at bit 32+, and 32bits of signficand in bits 0-31
#asahi-gpu · 2022-10-27 21:54
phireThe preformance isn't even that bad (when you inline it and avoid unpacking/packing into the proper ieee754 format)
#asahi-gpu · 2022-10-27 21:45
phireThough, I do really like the "let's just do soft float" solution. People are too scared of soft float
#asahi-gpu · 2022-10-27 21:44
phirestd::sync:: atomic::compiler_fence() // for rust
#asahi-gpu · 2022-10-27 21:44
phireasm volitile("" ::: "memory") // for c
#asahi-gpu · 2022-10-27 21:44
phireSurely there is a way to (safely) prevent reordering; compiler Instruction reordering barrier?
#asahi-gpu · 2022-10-27 21:44
phireI said this on Twitter but I'll repeat it here:
#asahi-gpu · 2022-10-27 13:45
alyssatrusts your Rust code to get it right more than any of the kernel C code doing FP math
#asahi-gpu · 2022-10-27 13:45
alyssasounds fun ;-)
#asahi-gpu · 2022-10-27 13:45
alyssaooh boy

plus ›