#
✕ nick:lina
#asahi-gpu · 2022-10-28 05:36
linaIt's possible they have some sort of pass to count the total number of primitives first, then split based on tat?
#asahi-gpu · 2022-10-28 05:35
linaFor a single draw op it's trivial, just split the primitives into equal 8ths. More interesting is what happens with multiple draw ops.
#asahi-gpu · 2022-10-28 05:34
linaAlso, they might have primitive IDs in the TVB data anyway, I think they need that later during rasterization? So maybe it doesn't matter...
#asahi-gpu · 2022-10-28 05:34
linaAnd then if you can do that, chaining just works
#asahi-gpu · 2022-10-28 05:34
linaProbably in draw order...
#asahi-gpu · 2022-10-28 05:34
linaSo that tells me the first cluster processed all the clear ops and every tile, and then the other clusters progressively processed additional slices of geometry for busy tiles
#asahi-gpu · 2022-10-28 05:33
linaYeah, so the question is whether they do something at the dispatch level to make sure every cluster gets an ordered slice of the overall workload. I think they do, because the first tilemap is ~complete, and they get sparser as you step through the buffer.
#asahi-gpu · 2022-10-28 05:32
linaSo it's very likely they just trivially chain those
#asahi-gpu · 2022-10-28 05:31
linaAnd since that next level is allocated out of the heap, it has to have chain pointers
#asahi-gpu · 2022-10-28 05:31
linaThe top level tile array just has 5-byte memory pointers to the initial position of every tile, and then there's at least one intermediate meta/pointer array structure before the raw data (it's been a while since I dumped these buffers...)
#asahi-gpu · 2022-10-28 05:30
linaIt's just chaining together slices for each tile
#asahi-gpu · 2022-10-28 05:30
linaThe actual vertex data pages are definitely not merged, that'd be wasteful
#asahi-gpu · 2022-10-28 05:30
linaSo when I say "merging" it's just writing out some pointers
#asahi-gpu · 2022-10-28 05:29
linaAnd then the rest of the top level pointers are probably chained together at the next level, or something like that
#asahi-gpu · 2022-10-28 05:29
linaThe top level tile array is just 5 bytes per tile, a memcpy of that seems wasteful but is trivial
#asahi-gpu · 2022-10-28 05:29
linaThe structures are trees, so they can probably just chain pointers
#asahi-gpu · 2022-10-28 05:22
linaBut between clusters, don't think so
#asahi-gpu · 2022-10-28 05:21
linaI mean I'm sure they have that within a cluster
#asahi-gpu · 2022-10-28 05:21
linaAnd that wouldn't scale well...
#asahi-gpu · 2022-10-28 05:21
linaThey have separate root tilemaps for every cluster, so they are definitely processing everything in chunks and then merging at the end
#asahi-gpu · 2022-10-28 05:21
linaNot sure how that would work with multiple draws though
#asahi-gpu · 2022-10-28 05:20
linaIf they split the input draw arrays by cluster in API order, then it's trivial
#asahi-gpu · 2022-10-28 05:19
linaI'd have to look into the pointee data and poll it and see what happens to it, I'm only looking at the top level now
#asahi-gpu · 2022-10-28 05:18
linaSo it looks like that just gets copied over (sounds inefficient?) and they probably either chain the rest of the pointers behind, or do some merging of top-level structures (but evidently not the actual vertex buffer data, this is all variable size anyway so they need to have some kind of linked list or growable array mechanism anyway)
#asahi-gpu · 2022-10-28 05:18
linaOK, so the final tile pointer array is == the first cluster-block of the temporary tile pointer array after a render
#asahi-gpu · 2022-10-28 05:15
linaI think it just can't preempt individual tile launches
#asahi-gpu · 2022-10-28 05:15
linaI don't see any depth meta buffer data with preemption but no TVB overflows, so I don't think that's it
#asahi-gpu · 2022-10-28 05:08
linaI wonder, they might have something special for compute
#asahi-gpu · 2022-10-28 05:07
linaI guess I should check whether preemption writes out partial data... should be easy...
#asahi-gpu · 2022-10-28 05:07
linaI mean, it has to be used for partial renders anyway
#asahi-gpu · 2022-10-28 05:07
linaWell yes
#asahi-gpu · 2022-10-28 05:06
linaFor tiles?
#asahi-gpu · 2022-10-28 05:06
lina(Either way it definitely can't snapshot tile memory wholesale)
#asahi-gpu · 2022-10-28 05:06
lina(Or maybe it can trigger tiles to early-exit and just flush to the FB, like a partial render?)
#asahi-gpu · 2022-10-28 05:05
linaSo it can't preempt individual core launches (at whatever granularity they use for that), which we know because there's no tile memory buffers.
#asahi-gpu · 2022-10-28 05:05
linaOh yeah, and this also explains why the preemption buffers are so small, and replicated like this. They hold the tiling accelerator/command processor state, not compute unit state.
#asahi-gpu · 2022-10-28 04:57
linaAnd the count kind of adds up (7 extra for M1 Max: 1 for AVE/scaler stuff, 1 for the mystery unused neural engine, 2 extra DCPs, 2 for the GPU clusters, 1 for prores or just random)
#asahi-gpu · 2022-10-28 04:56
linaAnd you see them all over the place (there's 4 on the top side of the GPU, but also a row of them near the top of the die, and one in each DDR IO cell)
#asahi-gpu · 2022-10-28 04:55
linaThe little square blocks like that always scream PLL, they're clearly analog blocks and very different from any logic stuff, but not attached to any IO edges
#asahi-gpu · 2022-10-28 04:54
linaIt has 7 PLLs and there are no other PLLs lying around in the M1 Max exclusive half of the die, so that has to be it
#asahi-gpu · 2022-10-28 04:52
linaI think top is shared/ASC/bufmgr, bottom is actually partially unrelated to the GPU (it has all the extra PLLs for the M1 Max, those are needed by the other blocks), and there's nothing specific for inter-die since the architecture already scales in that direction with just Apple Fabric
#asahi-gpu · 2022-10-28 04:50
linaAre you sure middle is not symmetric? It looks symmetric to me...
#asahi-gpu · 2022-10-28 04:50
linaI think top might include the ASC
#asahi-gpu · 2022-10-28 04:49
linaI've also noticed it uses more tile buffer pages than the M1, and that makes sense since allocation is at the 32K page level and with more slices, that's more overhead
#asahi-gpu · 2022-10-28 04:48
linaThat makes perfect sense, come to think of it
#asahi-gpu · 2022-10-28 04:48
linaIt's possible the way it works is there is an extra merge pass. Like, all the cores tile a random slice of geometry into separate buffers (and only have to coordinate to allocate buffer pages from the shared buffer manager), then once that's done the pointers are merged/chained (could be a linked list already, not sure exactly how it is organized) and then vertex gets to work on the result
#asahi-gpu · 2022-10-28 04:46
linaAnd on the M1 Max, the little block on die 1 would be unused, everything else would be used, with the tiling/sorting stuff coordinating via the extra buffers
#asahi-gpu · 2022-10-28 04:45
linaSo I'm guessing that little block is the top-level buffer management / dispatch stuff, the per-cluster structures are the actual tiling/sorting/whatever, and then there's the actual cores
#asahi-gpu · 2022-10-28 04:45
linaIf you look at the die shot, there are 8 compute units per cluster and then a bunch of structures that are also per cluster. And then there's one common area that is global in the M1 Pro, and not even symmetric in the M1 Max
#asahi-gpu · 2022-10-28 04:43
linaI imagine that all that logic comes from die 0, and is unused in die 1, only the actual compute units and tiling output logic are used in die 1

plus ›