#asahi-gpu · 2023-09-08 13:50
linaThat has nothing to do with this#asahi-gpu · 2023-09-08 13:50
i509vcbmaxBufferSize from https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkPhysicalDeviceMaintenance4PropertiesKHR.html#asahi-gpu · 2023-09-08 13:49
linaI mean if we need to allocate at ioctl time it's not much harder to have it allocate-or-expand and take the buffer size from userspace#asahi-gpu · 2023-09-08 13:49
alyssaMaybe? shrug#asahi-gpu · 2023-09-08 13:49
alyssa13:17 lina | Then we could even do driconf overrides for known problem apps#asahi-gpu · 2023-09-08 13:48
linai509vcb: What limit are you thinking about? This is mostly for internal mesh/geom/etc emulation, not a buffer exposed to users#asahi-gpu · 2023-09-08 13:46
linaAnd using blocks larger than 64MiB seems... kind of wasteful#asahi-gpu · 2023-09-08 13:45
linaI don't think the GPU/firmware care about that, but there does seem to be what looks like a 64-bit bitmask of blocks involved, so whatever blocksize we pick we can only have up to 64 of them.#asahi-gpu · 2023-09-08 13:45
linaI think macOS just restricts it to 4GiB of address space#asahi-gpu · 2023-09-08 13:44
i509vcblina: Is the mentioned theoretical 4GiB maximum buffer size related to the usage of the buffer or is it more of a strict limit? (Want to understand this before blindly updating the limit in agxv)#asahi-gpu · 2023-09-08 13:38
lina(And since as far as I can tell the mechanism is uniform, it doesn't even have to be compute, I can just have a render pass request allocatinon)#asahi-gpu · 2023-09-08 13:37
linaalyssa: Do you want me to look at this more or is this enough to make up a plan? I *think* I can easily test this whole mechanism from the m1n1 python stuff, since it doesn't really have to involve shaders (I can just pre-populate the buffers with whatever I want and dump whatever the firmware did to them)#asahi-gpu · 2023-09-08 13:17
linaThen we could even do driconf overrides for known problem apps#asahi-gpu · 2023-09-08 13:17
linaalyssa: I know we don't know the max required blocks, but it seems more useful to allow that to be set from userspace (e.g. via env variable) than a single global kernel module parameter?#asahi-gpu · 2023-09-08 13:17
linaalyssa: Yes, "job" is a RunCompute#asahi-gpu · 2023-09-08 12:23
alyssa(If I did I'd just static allocate in userspace)#asahi-gpu · 2023-09-08 12:23
alyssaThe entire point of this mechanism is that I /don't/ know any reasonable bounds ahead-of-time#asahi-gpu · 2023-09-08 12:22
alyssaSo it would be set unconditionally to the size of the entire global heap#asahi-gpu · 2023-09-08 12:22
alyssaI don't know that information#asahi-gpu · 2023-09-08 12:22
alyssa10:08 <lina> Our ioctl could pass in a max required blocks size#asahi-gpu · 2023-09-08 12:21
alyssa(rather than an individual dispatch or a whole ioctl?)#asahi-gpu · 2023-09-08 12:20
alyssaTo be clear, "job" in this context is a thing kicked off by a RunCompute op?#asahi-gpu · 2023-09-08 12:20
alyssaThis seems deeply cursed#asahi-gpu · 2023-09-08 12:20
alyssa 08:57 <lina> For mesh, there are 3 compute jobs: the first one calculates the first allocation, the second one writes to it and calculates the second allocation, then the third one writes out the mesh data/stuff into the second allocation, then the final 3d/ta job uses that. #asahi-gpu · 2023-09-08 12:19
alyssaso that tracks#asahi-gpu · 2023-09-08 12:19
alyssaYeah, we're assuming the .TODO variants have different cache bits but we're not sure what any of the bits mean yet#asahi-gpu · 2023-09-08 12:19
alyssa 09:00 <lina> 1ca: 4591280510f83200 device_store.TODO 0, i32, xy, r18_r19, r12_r13, 2, signed, lsl 2, 1, 1, 0, 3, 0 #asahi-gpu · 2023-09-08 10:08
linaOur ioctl could pass in a max required blocks size, and then at submit time the kernel ensures the global shared allocation has at least that much space. Then we mutex job execution using the global buffers (considering this another resource, blocking via a fence), but lazily drag buffer blocks between VMs so we avoid remapping and zeroing out when the jobs actually use less than the max memory.#asahi-gpu · 2023-09-08 10:07
linaAnd the requirement is that everything is submitted as one ioctl, from alloc to free#asahi-gpu · 2023-09-08 10:06
linaI think for us, the ABI for this would look essentially the same as the firmware ABI, except we use a BO and offsets for the count/shared buffers with the firmware (then we can map it in firmware land without making it world writable like Apple does, they're so bad at this...)#asahi-gpu · 2023-09-08 09:06
linaSo we could use a different blocksize, but we'd lower our max usage if we do that.#asahi-gpu · 2023-09-08 09:06
linaOhh but I think the firmware may have a 64-bit free blockmap, so max 64 blocks. Blocks are 64MiB on macOS, which means 4GiB max buffer size, which makes sense as a max.#asahi-gpu · 2023-09-08 09:00
linaMy guess is that's an uncached store or something like that, since the firmware needs to be coherent with this stuff.#asahi-gpu · 2023-09-08 09:00
lina 1ca: 4591280510f83200 device_store.TODO 0, i32, xy, r18_r19, r12_r13, 2, signed, lsl 2, 1, 1, 0, 3, 0#asahi-gpu · 2023-09-08 09:00
linaThe only interesting thing in the shaders is those shared memory counts are written to like this:#asahi-gpu · 2023-09-08 08:59
linaAlso everything is done in discrete "block count" units but I think the firmware doesn't care about where we alloc or how big a block is, since that's just up to the shaders/kernel#asahi-gpu · 2023-09-08 08:58
linaIn typical Apple fashion, it's overcomplicated with two layers of accounting for buffer size/usage...#asahi-gpu · 2023-09-08 08:58
linaSo the "just in time" allocation mechanism is just there so the jobs don't have to pingpong through the kernel for allocs, but rather they can be queued at the firmware eagerly and only when the firmware is about to run a job does it poke the kernel to allocate its memory#asahi-gpu · 2023-09-08 08:57
linaFor mesh, there are 3 compute jobs: the first one calculates the first allocation, the second one writes to it and calculates the second allocation, then the third one writes out the mesh data/stuff into the second allocation, then the final 3d/ta job uses that.#asahi-gpu · 2023-09-08 08:56
linaEach job has an alloc request and a free request, and writes out the alloc/free request counts of the next job#asahi-gpu · 2023-09-08 08:55
linaInstead what they do is they have a compute shader calculate how much memory is needed, and write that out to some shared memory, then when the next shader runs that shared memory location is passed to the firmware as an alloc request, and then the firmware allocates memory before running it.#asahi-gpu · 2023-09-08 08:54
linaIt's all firmware and shared memory, there is no mechanism to interrupt a shader or anything like that.#asahi-gpu · 2023-09-08 08:53
linaalyssa: Soooo.... there's no mailbox ^^;;#asahi-gpu · 2023-09-07 09:08
alyssaand also the name of the descriptor passed to PBE instructions#asahi-gpu · 2023-09-07 09:08
alyssaPixel BackEnd, the hardware used to write to colour attachments & images#asahi-gpu · 2023-09-07 09:08
alyssayes for blit#asahi-gpu · 2023-09-07 02:44
i509vcbOn that topic what is PBE?#asahi-gpu · 2023-09-07 02:40
i509vcbhmm that wants load_layer_id, random crapshoot at the cmdbuf that appears to be something related to AGX_PBE?#asahi-gpu · 2023-09-07 02:03
i509vcb(I'm guessing yes because I saw the blit stuff already is there)#asahi-gpu · 2023-09-07 01:54
i509vcbalyssa: was the plan to implement vkCmdBlitImage with vk meta like image and buffer copies?