#
…

2022-10-28

00:37Etrien__ has quit [Read error: Connection reset by peer]
01:11Dcow has quit [Remote host closed the connection]
01:13zzywysm_ has joined #asahi-gpu
01:14Dcow has joined #asahi-gpu
01:18zzywysm has quit [Ping timeout: 480 seconds]
01:25Dcow has quit [Ping timeout: 480 seconds]
01:32zzywysm_ has quit [Quit: Textual IRC Client: www.textualapp.com]
01:33Etrien has joined #asahi-gpu
01:38zzywysm has joined #asahi-gpu
01:39chadmed has joined #asahi-gpu
01:50Dcow has joined #asahi-gpu
01:59Dcow has quit [Ping timeout: 480 seconds]
02:01Etrien__ has joined #asahi-gpu
02:05Etrien___ has joined #asahi-gpu
02:06Etrien has quit [Ping timeout: 480 seconds]
02:09Etrien__ has quit [Ping timeout: 480 seconds]
02:12Etrien has joined #asahi-gpu
02:17Etrien___ has quit [Ping timeout: 480 seconds]
02:25Dcow has joined #asahi-gpu
02:33Dcow has quit [Ping timeout: 480 seconds]
02:44Etrien has quit [Remote host closed the connection]
02:51Mary has quit [Quit: The Lounge - https://thelounge.chat]
02:51Mary has joined #asahi-gpu
02:59pbsds has quit []
02:59Dcow has joined #asahi-gpu
03:00pbsds has joined #asahi-gpu
03:08Dcow has quit [Ping timeout: 480 seconds]
03:08chadmed has quit [Quit: Konversation terminated!]
03:16Etrien has joined #asahi-gpu
03:34Dcow has joined #asahi-gpu
03:42Dcow has quit [Ping timeout: 480 seconds]
03:46chadmed has joined #asahi-gpu
03:58ella-0_ has joined #asahi-gpu
04:02ella-0 has quit [Read error: Connection reset by peer]
04:08Dcow has joined #asahi-gpu
04:16Dcow has quit [Ping timeout: 480 seconds]
04:33linaphire: Performance really doesn't matter, and I need input/output in f32 format... I think I'm just going to wrap the existing kernel softfloat stuff for now
04:33linaIf upstream doesn't like it I can always replace it with something else
04:34phireupstream linux, or upstream rust-on-linux?
04:34linaboth ^^
04:34linaI've been looking at the new buffer stuff. The buffers are sized based on the core count, and are used even if I turn off half the cores. So it seems the "2 GPUs" is a red herring. It's 8 GPUs!
04:34linaAll that stuff is also used on M1 Pro and Max
04:34linaSo they planned for this scalability from the get go, it seems
04:35linaIt's just M1 doesn't need it since it is one cluster
04:35Etrien has quit [Remote host closed the connection]
04:35linaAs for the deflake buffers, for some inexplicable reason macOS allocates them as exactly *9x* the M1 sizes... but with 8x it works, so maybe that's an off-by-one on their side (or they were padding for debugging)
04:35Etrien has joined #asahi-gpu
04:35lina(deflake = preempt, I really need to replace that wording)
04:35phireyeah, I was supicious about the actual "gpu count" based on some of the initdata
04:36linaAnd then of the new tile buffers, most are fixed size * cluster count it seems, but one seems to be aux tilemaps, and the size is the tilemap size * 4 * nclusters
04:36linaI don't know where the 4 comes from, it seems the last 3/4 of each block end up unused...
04:36linaMaybe it's a spill thing or maybe I have a stride setting wrong somewhere
04:37phireso it's 4 cores per cluster?
04:37lina8 cores per cluster
04:37* phirescrolls down to correct die shot
04:38linaOhh I found the stride... and yup, I have it wrong, I remember a factor of 4 I wondering about here. This also makes sense, I remember messing with this one on M1 and it not mattering...
04:39phireif the guranuality is that small, I wonder if it can split vertex workloads across clusters. Or if vertex workloads are limited to a single cluster
04:40linaIt can, it's what these buffers are for.
04:40linaI can restrict which clusters it uses for vertex via a mask, and it slows down if I do that
04:40linaFancy.
04:40linaIf I restrict it to just one cluster, the buffers are not used at all (any cluster will do)
04:40linaSame as M1
04:41linaAlso, for some queues/jobs, macOS does just that, rolling the cluster it picks
04:41linaBut I can't get it to actually run multiple jobs across clusters like this, so I think this is just some kind of power/load spreading for simple jobs, like the compositor
04:41phireinteresting, so it's splitting vertex workloads at a pretty fine granularity. Within a single draw?
04:41linamacOS only ever uses either 1 cluster rolling, or all clusters, and it doesn't seem to be per client at least. Might be a heuristic.
04:42linaI think so, since I think the glmark2 bunny is a single draw?
04:42chadmed has quit [Quit: Konversation terminated!]
04:43linaAnd I mean it has to balance across cores anyway... balancing across clusters just means, I'm guessing, extra handling for tile maps and buffer management metadata
04:43linaI imagine that all that logic comes from die 0, and is unused in die 1, only the actual compute units and tiling output logic are used in die 1
04:44Dcow has joined #asahi-gpu
04:44phireyeah, the mesh is just a single DrawArrays call
04:45phireor draw VBO if that's enabled. not sure which path is used by default
04:45linaIf you look at the die shot, there are 8 compute units per cluster and then a bunch of structures that are also per cluster. And then there's one common area that is global in the M1 Pro, and not even symmetric in the M1 Max
04:45linaSo I'm guessing that little block is the top-level buffer management / dispatch stuff, the per-cluster structures are the actual tiling/sorting/whatever, and then there's the actual cores
04:46linaAnd on the M1 Max, the little block on die 1 would be unused, everything else would be used, with the tiling/sorting stuff coordinating via the extra buffers
04:46phirethat might actually be three common areas. One at the top, one in the middle, one at the bottom. top and bottom seem to be diffrent
04:47phirethe bottom one might be to do with inter-die work sharing
04:48linaIt's possible the way it works is there is an extra merge pass. Like, all the cores tile a random slice of geometry into separate buffers (and only have to coordinate to allocate buffer pages from the shared buffer manager), then once that's done the pointers are merged/chained (could be a linked list already, not sure exactly how it is organized) and then vertex gets to work on the result
04:48linaThat makes perfect sense, come to think of it
04:49linaI've also noticed it uses more tile buffer pages than the M1, and that makes sense since allocation is at the 32K page level and with more slices, that's more overhead
04:49phirethoery: top is inter-cluster dispatch; middle one is inter-cluster coherency fabric; bottom is for inter-die coherrency/comunication
04:50linaI think top might include the ASC
04:50linaAre you sure middle is not symmetric? It looks symmetric to me...
04:52phireI mean, it's made of 2-4 symetric slices. but those slices seem to be tied together into a single block
04:52Dcow has quit [Ping timeout: 480 seconds]
04:52phirein a way that reminds me of zen 2's l3 cache. but it doesn't look like a cache
04:52linaI think top is shared/ASC/bufmgr, bottom is actually partially unrelated to the GPU (it has all the extra PLLs for the M1 Max, those are needed by the other blocks), and there's nothing specific for inter-die since the architecture already scales in that direction with just Apple Fabric
04:53phireso I'm thinking tags?
04:53phireyeah, you might be right that it's just something sitting in otherwise unused space
04:54phire(at the bottom)
04:54linaIt has 7 PLLs and there are no other PLLs lying around in the M1 Max exclusive half of the die, so that has to be it
04:54phirehow can you tell they are PLLs?
04:55phire(that's not something I've learned to spot on a die shot)
04:55linaThe little square blocks like that always scream PLL, they're clearly analog blocks and very different from any logic stuff, but not attached to any IO edges
04:56linaAnd you see them all over the place (there's 4 on the top side of the GPU, but also a row of them near the top of the die, and one in each DDR IO cell)
04:57linaAnd the count kind of adds up (7 extra for M1 Max: 1 for AVE/scaler stuff, 1 for the mystery unused neural engine, 2 extra DCPs, 2 for the GPU clusters, 1 for prores or just random)
04:59phiremakes sense
05:03chadmed has joined #asahi-gpu
05:05linaOh yeah, and this also explains why the preemption buffers are so small, and replicated like this. They hold the tiling accelerator/command processor state, not compute unit state.
05:05linaSo it can't preempt individual core launches (at whatever granularity they use for that), which we know because there's no tile memory buffers.
05:06lina(Or maybe it can trigger tiles to early-exit and just flush to the FB, like a partial render?)
05:06lina(Either way it definitely can't snapshot tile memory wholesale)
05:06phireyes, was just going to say
05:06phirethere is already memory allocated for tiles, though we don't know if it's used
05:06linaFor tiles?
05:07phirethe framebuffer
05:07linaWell yes
05:07linaI mean, it has to be used for partial renders anyway
05:07linaI guess I should check whether preemption writes out partial data... should be easy...
05:08phiresome of those compute jobs are long-running. I wonder if compute launches require buffers for compute unit state
05:08linaI wonder, they might have something special for compute
05:08phirebut vertex/fragment jobs are useally short enough that you can just wait for them to end
05:10phirebut I would think indivudal tile infocations could be long enough that it's worth preempting them
05:12phireI'm not sure what their expected preemption granularity is. it would be weird for a single tile to exeute for longer than ~10us
05:15chadmed has quit [Quit: Konversation terminated!]
05:15linaI don't see any depth meta buffer data with preemption but no TVB overflows, so I don't think that's it
05:15linaI think it just can't preempt individual tile launches
05:16phireI wonder if it just aborts any tile that doesn't finish quick enough
05:18linaOK, so the final tile pointer array is == the first cluster-block of the temporary tile pointer array after a render
05:18linaSo it looks like that just gets copied over (sounds inefficient?) and they probably either chain the rest of the pointers behind, or do some merging of top-level structures (but evidently not the actual vertex buffer data, this is all variable size anyway so they need to have some kind of linked list or growable array mechanism anyway)
05:19phireso you can actually see it merging?
05:19phireI guess it does need to be sorted into API order, which would be hard without a pass over it
05:19linaI'd have to look into the pointee data and poll it and see what happens to it, I'm only looking at the top level now
05:20linaIf they split the input draw arrays by cluster in API order, then it's trivial
05:21linaNot sure how that would work with multiple draws though
05:21phirethe other option is a centralized vertex-writeback block, basically equlivant to the ROPs for pixels on desktop gpus
05:21phireeach compute core sets final vertex data to a centralized block, which sorts it and writes it to the TVB
05:21linaThey have separate root tilemaps for every cluster, so they are definitely processing everything in chunks and then merging at the end
05:21linaAnd that wouldn't scale well...
05:21phireor N centralied blocks, addressed by screenspace location
05:21linaI mean I'm sure they have that within a cluster
05:22linaBut between clusters, don't think so
05:22phireyes, there would be scaling issues
05:22phirebut merging just seems wasteful
05:22phirebut merging also has better scaling?
05:25phireHang on, is there any reason not to just implement reading from N TVBs (one per cluster) duing the 3d invocation?
05:26phireand reading and merging them in a single pass
05:29linaThe structures are trees, so they can probably just chain pointers
05:29linaThe top level tile array is just 5 bytes per tile, a memcpy of that seems wasteful but is trivial
05:29linaAnd then the rest of the top level pointers are probably chained together at the next level, or something like that
05:30linaSo when I say "merging" it's just writing out some pointers
05:30linaThe actual vertex data pages are definitely not merged, that'd be wasteful
05:30linaIt's just chaining together slices for each tile
05:30phireoh, point. we are just merging the indices
05:31phireit's a lot less waseful when you put it that way
05:31linaThe top level tile array just has 5-byte memory pointers to the initial position of every tile, and then there's at least one intermediate meta/pointer array structure before the raw data (it's been a while since I dumped these buffers...)
05:31linaAnd since that next level is allocated out of the heap, it has to have chain pointers
05:32linaSo it's very likely they just trivially chain those
05:32phireexcept you have to respect ordering, because stupid reasons
05:33phirethere are very slight graphical diffrences (were depths are equal) if you render triangles out of API order
05:33linaYeah, so the question is whether they do something at the dispatch level to make sure every cluster gets an ordered slice of the overall workload. I think they do, because the first tilemap is ~complete, and they get sparser as you step through the buffer.
05:34linaSo that tells me the first cluster processed all the clear ops and every tile, and then the other clusters progressively processed additional slices of geometry for busy tiles
05:34linaProbably in draw order...
05:34phireand much larger diffrences if blending or image load/store are enabled
05:34linaAnd then if you can do that, chaining just works
05:34linaAlso, they might have primitive IDs in the TVB data anyway, I think they need that later during rasterization? So maybe it doesn't matter...
05:35linaFor a single draw op it's trivial, just split the primitives into equal 8ths. More interesting is what happens with multiple draw ops.
05:36linaIt's possible they have some sort of pass to count the total number of primitives first, then split based on tat?
05:36phireactually, maybe they cheat and their depth sorting implementation also sorts by primitive ID
05:37phirewhich would have a side-effect that it has to fall back to single-cluster vertex processing when doing alpha blending
05:37linaThat seems really wasteful...
05:37phireyeah, and apple's docs go on and on about how alpha blending is expensive
05:38phirewhy not make it a bit more expensive?
05:38phirelike more than 95% of your geometery in a typical game will be opaque
05:38phirebecause alpha blending is somewhat expensive everywhere
05:39linaYeah, but if they have to do some kind of flush around alpha draws...
05:39phirethey already do
05:40linaAnyway, I should get ready for the stream (and have a sandwich or something) ^^
05:40phirealpha blending already puts the TBDR renderer into a very sub-optimal mode
05:40linaAt the fragment level though, right? Not the vertex level...
05:41phiredepending on the exact impelementation, you might have to allocate some of the tile buffer for a copy of the depth buffer
05:41phirehmm, does it really need a full flush to switch vertex modes?
05:43phirethe mask is surely per job, so you can just put the alpha blended verts in a new job
05:46phirethough, I can't actually remember what level we called "job"
06:09Etrien__ has joined #asahi-gpu
06:15Etrien has quit [Ping timeout: 480 seconds]
06:25Dcow has joined #asahi-gpu
06:25Etrien has joined #asahi-gpu
06:29SSJ_GZ has joined #asahi-gpu
06:30Etrien__ has quit [Ping timeout: 480 seconds]
06:33Dcow has quit [Ping timeout: 480 seconds]
06:49chadmed has joined #asahi-gpu
06:50ChaosPrincess has quit [Remote host closed the connection]
06:51uur has joined #asahi-gpu
06:52ChaosPrincess has joined #asahi-gpu
06:53uur has quit [Quit: Leaving.]
06:55uur has joined #asahi-gpu
07:06uur has quit [Ping timeout: 480 seconds]
07:12uur has joined #asahi-gpu
07:20chadmed has quit [Quit: Konversation terminated!]
07:20uur has quit [Remote host closed the connection]
07:29uur has joined #asahi-gpu
07:31uur has quit [Remote host closed the connection]
07:44chadmed has joined #asahi-gpu
07:55uur has joined #asahi-gpu
08:03al3xtjames9 has joined #asahi-gpu
08:04sorear___ has joined #asahi-gpu
08:04Manouchehri_ has joined #asahi-gpu
08:05jonmasters_ has joined #asahi-gpu
08:05chadmed has quit [Quit: Konversation terminated!]
08:05yuyichao has joined #asahi-gpu
08:06sa1_ has joined #asahi-gpu
08:06Z750 has joined #asahi-gpu
08:06nepeat_ has joined #asahi-gpu
08:06wicastC has joined #asahi-gpu
08:07yuyichao_ has quit [synthon.oftc.net larich.oftc.net]
08:07kov has quit [synthon.oftc.net larich.oftc.net]
08:07nepeat has quit [synthon.oftc.net larich.oftc.net]
08:07al3xtjames has quit [synthon.oftc.net larich.oftc.net]
08:07artemist has quit [synthon.oftc.net larich.oftc.net]
08:07wicast has quit [synthon.oftc.net larich.oftc.net]
08:07princesszoey has quit [synthon.oftc.net larich.oftc.net]
08:07Z751 has quit [synthon.oftc.net larich.oftc.net]
08:07Manouchehri has quit [synthon.oftc.net larich.oftc.net]
08:07sorear__ has quit [synthon.oftc.net larich.oftc.net]
08:07sa1 has quit [synthon.oftc.net larich.oftc.net]
08:07al3xtjames9 is now known as al3xtjames
08:07jonmasters has quit [synthon.oftc.net larich.oftc.net]
08:07sa1_ is now known as sa1
08:07Manouchehri_ is now known as Manouchehri
08:07artemist has joined #asahi-gpu
08:08princesszoey has joined #asahi-gpu
08:08dviola has joined #asahi-gpu
08:09uur has quit [Remote host closed the connection]
08:09kov has joined #asahi-gpu
08:15uur has joined #asahi-gpu
08:17Dcow has joined #asahi-gpu
08:20uur has quit [Remote host closed the connection]
08:25Dcow has quit [Ping timeout: 480 seconds]
08:33uur has joined #asahi-gpu
08:52Dcow has joined #asahi-gpu
08:55uur has quit [Remote host closed the connection]
08:56uur has joined #asahi-gpu
08:58uur has quit [Remote host closed the connection]
09:00uur has joined #asahi-gpu
09:00Dcow has quit [Ping timeout: 480 seconds]
09:02uur has quit [Remote host closed the connection]
09:28bluetail has quit [Ping timeout: 480 seconds]
09:29Etrien__ has joined #asahi-gpu
09:31Etrien has quit [Ping timeout: 480 seconds]
09:34Dcow has joined #asahi-gpu
09:39uur has joined #asahi-gpu
09:41uur has quit [Remote host closed the connection]
09:48Etrien has joined #asahi-gpu
09:52Etrien__ has quit [Ping timeout: 480 seconds]
09:55uur has joined #asahi-gpu
09:57uur has quit [Remote host closed the connection]
09:58uur has joined #asahi-gpu
10:32dviola has quit [Ping timeout: 480 seconds]
10:34chadmed has joined #asahi-gpu
10:34uur has quit [Remote host closed the connection]
10:38uur has joined #asahi-gpu
10:40kov has quit [Quit: Coyote finally caught me]
10:50uur has quit [Remote host closed the connection]
10:51uur has joined #asahi-gpu
10:57Etrien__ has joined #asahi-gpu
10:58Etrien__ has quit [Read error: Connection reset by peer]
10:59Etrien__ has joined #asahi-gpu
11:00mrkajetanp has joined #asahi-gpu
11:02Etrien has quit [Ping timeout: 480 seconds]
11:07mrkajetanp has quit [Remote host closed the connection]
11:17bluetail has joined #asahi-gpu
11:21chadmed has quit [Quit: Konversation terminated!]
11:31uur has quit [Remote host closed the connection]
11:33uur has joined #asahi-gpu
11:35dviola has joined #asahi-gpu
11:44uur has quit [Remote host closed the connection]
11:47uur has joined #asahi-gpu
11:48ethan has joined #asahi-gpu
11:52ethan is now known as amateurece
12:35compassion2 has joined #asahi-gpu
12:40compassion has quit [Ping timeout: 480 seconds]
12:40compassion2 is now known as compassion
13:41fmstrat has joined #asahi-gpu
13:49Gaspare has joined #asahi-gpu
14:02alyssa has joined #asahi-gpu
14:02alyssa"Apple Fabric"
14:02alyssathere a luxury clothing line I didn't hear about?
14:05svenit's their interconnect which connects most (all?) thing inside the SoC
14:05_jannau_they started selling square greyish samples for $19 a piece last year
14:06alyssa_jannau_: Ooh, got it, thank you
14:34uur has quit [Remote host closed the connection]
14:36compassion has quit [Quit: lounge quit]
14:36compassion has joined #asahi-gpu
15:14uur has joined #asahi-gpu
15:15uur has quit [Remote host closed the connection]
15:16uur has joined #asahi-gpu
15:24alyssa has quit [Quit: leaving]
15:26lonjil has quit [Quit: No Ping reply in 180 seconds.]
15:27lonjil has joined #asahi-gpu
15:28uur has quit []
15:30uur has joined #asahi-gpu
15:49uur has quit [Read error: Connection reset by peer]
15:49uur1 has joined #asahi-gpu
15:57chengsun_ has joined #asahi-gpu
16:00chengsun has quit [Ping timeout: 480 seconds]
16:00uur1 has quit [Quit: Leaving.]
16:08Gaspare has quit [Quit: Gaspare]
16:16maria has joined #asahi-gpu
16:26uur has joined #asahi-gpu
16:31SSJ_GZ has quit [Read error: Connection reset by peer]
16:32SSJ_GZ has joined #asahi-gpu
16:56maria has quit [Quit: WeeChat 3.5]
16:58Gaspare has joined #asahi-gpu
17:00maria has joined #asahi-gpu
17:07maria has quit [Quit: The Lounge - https://thelounge.chat]
17:07maria has joined #asahi-gpu
17:07maria is now known as Guest4362
17:08Guest4362 is now known as maria
17:22uur1 has joined #asahi-gpu
17:26uur has quit [Ping timeout: 480 seconds]
17:34uur1 has quit [Quit: Leaving.]
17:36uur has joined #asahi-gpu
17:55alyssa has joined #asahi-gpu
17:55alyssaIDK if this will fix glamor but it'll probably fix something!
17:55alyssaOr nothing. Completely untested :p
18:24Gaspare has quit [Quit: Gaspare]
19:34seam0s has joined #asahi-gpu
19:35seam0s has quit []
19:47r0ni has quit [Quit: Textual IRC Client: www.textualapp.com]
20:07millenialhacker has joined #asahi-gpu
21:40Gaspare has joined #asahi-gpu
21:50Gaspare has quit [Quit: Gaspare]
21:50Dcow has quit [Remote host closed the connection]
21:51Dcow has joined #asahi-gpu
21:54amarioguy has quit [Quit: Leaving]
21:59Gaspare has joined #asahi-gpu
22:06Gaspare has quit [Quit: Gaspare]
22:15millenia_ has joined #asahi-gpu
22:15millenia_ has quit [Remote host closed the connection]
22:15millenia_ has joined #asahi-gpu
22:16millenia_ has quit [Remote host closed the connection]
22:16millenia_ has joined #asahi-gpu
22:16millenia_ has quit [Remote host closed the connection]
22:17millenia_ has joined #asahi-gpu
22:18millenialhacker has quit [Read error: No route to host]
22:27millenia_ has quit [Ping timeout: 480 seconds]
23:01Dcow has quit [Remote host closed the connection]
23:02Dcow has joined #asahi-gpu
23:03uur_ has joined #asahi-gpu
23:04Dcow_ has joined #asahi-gpu
23:09uur has quit [Ping timeout: 480 seconds]
23:11Dcow has quit [Ping timeout: 480 seconds]
23:14uur_ has quit []
23:21Dcow_ has quit [Remote host closed the connection]
23:30phirehmmm... it would be nice if construct warned you when you named two fields the same thing
23:30phirelina: you have two pwr_filter_a_neg's
23:35Dcow has joined #asahi-gpu
23:36Dcow has quit [Remote host closed the connection]
23:36Dcow has joined #asahi-gpu
…