#
#asahi-dev · 2026-09-06 14:27
svenand you can just match on "M3-nvme" and only enabled 252 queue size at a later point then
#asahi-dev · 2026-09-06 14:26
yukaright
#asahi-dev · 2026-09-06 14:26
sventhe compatible will be something like "M3-nvme", "M1-nvme" or whatever anyway
#asahi-dev · 2026-09-06 14:26
yukaunless it's all of them
#asahi-dev · 2026-09-06 14:26
yukawhat I'm saying there would be a new compatible for the ones capable of queue size 252?
#asahi-dev · 2026-09-06 14:26
yukaOr I guess nobody knows if M2 or M2 Pro can already do it
#asahi-dev · 2026-09-06 14:25
yukasven: ack, but we should probably keep it in mind when upstreaming M3 nvme
#asahi-dev · 2026-09-06 14:25
yukathe main change is decoupling the queue sizes from tags and NVMMU_NUM_TCBS, since those still only go up to 63
#asahi-dev · 2026-09-06 14:24
svenmight increase performance but that can always be a follow-up
#asahi-dev · 2026-09-06 14:23
yukaor rather 252
#asahi-dev · 2026-09-06 14:23
yukaDo we think there is an advantage to running the nvme with queue size 256? This would require some more changes
#asahi-dev · 2026-09-06 14:22
yuka*have
#asahi-dev · 2026-09-06 14:22
yukaOr M1 Pro since that's the oldest I haive
#asahi-dev · 2026-09-06 14:18
yukaI should probably repeat the experiments on an M1
#asahi-dev · 2026-09-06 14:14
svenfeel free, the names are all made up anyway and if there's a name that fits better we should do it
#asahi-dev · 2026-09-06 14:13
yukaI wonder if we should rename it
#asahi-dev · 2026-09-06 14:05
yuka(what ivoszbg[m] already reported)
#asahi-dev · 2026-09-06 14:04
yukawhich explains why setting APPLE_ANS_MAX_PEND_CMDS_CTRL to max_queue_depth - 1 fixes it
#asahi-dev · 2026-09-06 14:04
yukaIt's actually APPLE_ANS_MAX_PEND_CMDS_CTRL
#asahi-dev · 2026-09-06 13:56
yukait does not seem to matter what I put in the qsize field
#asahi-dev · 2026-09-06 13:55
yukaI/O SQ size: 65, I/O CQ size: 65
#asahi-dev · 2026-09-06 13:47
yukalet me see what happens if I use qsize = max_queue_size - 2
#asahi-dev · 2026-09-06 13:45
yukaThis matches what Linux does. We assume max_queue_depth = 64 and set the qsize attr to 63 when creating the completion queue, but the ANS still prints it as size 65 when it crashes
#asahi-dev · 2026-09-06 13:44
yuka> Create IO completion queue [...] Command DWORD 10 contains the queue identifier in the low word, and the queue size in the high word. The queue size should be given as one less than the actual value.
#asahi-dev · 2026-09-06 13:41
yukaI wonder what even determines how many completion entries are used
#asahi-dev · 2026-09-06 13:41
yukaand this asymmetry is already present on M3
#asahi-dev · 2026-09-06 13:40
yukawhen we assumed 64 completion entries on the linux side
#asahi-dev · 2026-09-06 13:40
yukablaah, but yes I think this difference is what causes M4 to fail when it goes to completion entry 65
#asahi-dev · 2026-09-06 13:39
yukaso 254 completion and 252 submission entries
#asahi-dev · 2026-09-06 13:39
yukaM3 Max: IOCQ 0x0 - 0xfd, IOSQ 0x1 - 0xfc
#asahi-dev · 2026-09-06 13:38
yukaso effectively 65 IO completion entries and 63 IO submission entries
#asahi-dev · 2026-09-06 13:38
yuka*M3: IOCQ 0x0 - 0x40, IOSQ 0x1 - 0x3f
#asahi-dev · 2026-09-06 13:37
yukawhoops, the other way around
#asahi-dev · 2026-09-06 13:37
yukaM3: IOSQ 0x0 - 0x40, IOCQ 0x1 - 0x3f
#asahi-dev · 2026-09-06 13:33
yukahttps://github.com/AsahiLinux/m1n1/pull/675
#asahi-dev · 2026-09-06 13:30
yukathe code wouldn't work then I think
#asahi-dev · 2026-09-06 13:30
svensince that's an actual queue and whatever is running on the SoC and ANS have to agree about that value
#asahi-dev · 2026-09-06 13:30
sveni would expect it to refer to the CQ though
#asahi-dev · 2026-09-06 13:29
svenor i just typoed it once and copied that from m1n1 to linux then 😬
#asahi-dev · 2026-09-06 13:29
yukam1n1 nvme.c calls 0x100c NVME_DB_IOCQ too, so I guess the tracer is wrong
#asahi-dev · 2026-09-06 13:27
yukaOn my M3 Max it seems to actually use approximately 256 entries
#asahi-dev · 2026-09-06 13:25
yukalinux calls it APPLE_ANS_IOCQ_DB, m1n1 tracer calls it NVME_REG_DBS_IOSQ
#asahi-dev · 2026-09-06 13:25
yukaI'm also confused if 0x100c is related to the submission or the completion queue
#asahi-dev · 2026-09-06 13:24
sventhe io submission "queue" isn't really a queue, more like a vector, isn't it? so you could just get away with using less entries than available
#asahi-dev · 2026-09-06 13:24
yukanoopwafel: thanks for confirming
#asahi-dev · 2026-09-06 13:23
yukaIn the base M3 I see 65 (!) entries of the IO queue actually being used (0x0 - 0x40)
#asahi-dev · 2026-09-06 13:23
noopwafel(yes, that's from m3 with 14.x)
#asahi-dev · 2026-09-06 13:22
yukaboth set APPLE_ANS2_MAX_PEND_CMDS_CTRL = 0x1000100 (256)
#asahi-dev · 2026-09-06 13:21
yukaFirst thing I see is M3 seems to actually have a larger queue depth
#asahi-dev · 2026-09-06 13:21
yukaI looked at two traces, one from my M3 Max with 14.x fw and one from https://noopwafel.net/asahi/ (I think it's M3 with 14.x fw)

plus ›