#
✕ nick:sven
#asahi-dev · 2022-03-22 12:49
svenyeah
#asahi-dev · 2022-03-22 12:47
svensame for nvme, it also has a backdoor channel to SMC
#asahi-dev · 2022-03-22 12:33
svenjust more horribly because if you lock that processor up or do something wrong the entire SoC will reset *sigh*
#asahi-dev · 2022-03-22 12:32
sventhunderbolt uses a different HW mailbox interface but the ~same rtkit stuff as well
#asahi-dev · 2022-03-22 12:29
svenyeah, the mailbox subsystem assumes hardware channels with independent status bits
#asahi-dev · 2022-03-22 12:25
svenI also briefly looked into remoteproc, but that seemed to to mostly deal with virtio and firmware loading which we both don't have here
#asahi-dev · 2022-03-22 12:21
svenso if we had this inside mailbox we'd have some quirks for probably every single co-processor
#asahi-dev · 2022-03-22 12:20
svenmarcan knows more details about SMC, but there some parts are handled by the rtkit core (syslog, etc.) while others are handled by the SMC driver. that would be split across mailbox and SMC then. buffers are also mapped as if they are MMIO there and handed out by the co-processor
#asahi-dev · 2022-03-22 12:19
svenalright, rtkit. for nvme that thing is extra awkward because all we need there is boot and shutdown and no more communication after that. that already doesn't fit well with the mailbox API.
#asahi-dev · 2022-03-22 10:40
svenabout to grab lunch, can reply about rtkit later. but if we move rtkit to mailbox we have to keep SART separate because both rtkit buffers and nvme vendor-specific commands (which we may or may not need in the future) are filtered by SART
#asahi-dev · 2022-03-22 10:34
svenyes
#asahi-dev · 2022-03-22 10:34
svenit's the only thing that changed between M1 and M1 Pro/Max fwiw
#asahi-dev · 2022-03-22 10:34
svenI don't have any strong preference for keeping SART separate. it just felt like a different device.
#asahi-dev · 2022-03-22 10:32
sven*kernel-controlled
#asahi-dev · 2022-03-22 10:31
svenno need to do that inside linux ofc
#asahi-dev · 2022-03-22 10:31
svenand if it's more than two pages they setup the PRP structure twice, once inside kernel-controller memory and a second time inside fake-hypervisor-controlled memory
#asahi-dev · 2022-03-22 10:29
sventhey split it because they have this fake-hypervisor which setups the latter while the kernel sets up the first
#asahi-dev · 2022-03-22 10:29
sventhere's one array (which apple calls linear sq) where you setup the command and then another array where you set it up again (which apple calls NVMMU)
#asahi-dev · 2022-03-22 10:28
svenit's part of the NVMe driver because it's very tightly coupled with command submission
#asahi-dev · 2022-03-22 10:27
svenso far it's only been used for nvme
#asahi-dev · 2022-03-22 10:27
svenso i can't make it dma_map_ops because then everything would go through it
#asahi-dev · 2022-03-22 10:27
svenarnd: i wanted to reply later today but essentially the issue is that NVMe has two IOMMU-like things. once this tightly coupled NVMMU for most commands and then this SART thing for buffers like the syslog and some vendor-specific NVMe commands that don't use the normal PRPs
#asahi-dev · 2022-03-21 17:31
svenyes
#asahi-dev · 2022-03-21 17:09
sven:)
#asahi-dev · 2022-03-21 16:52
svenaaand sent.
#asahi-dev · 2022-03-21 16:44
svenotherwise i'll run into something and keep procrastinating that for another few months!
#asahi-dev · 2022-03-21 16:44
svenactually, let me send out nvme out first :D
#asahi-dev · 2022-03-21 16:43
svenbut doesn't hurt to enable it on my builds anyway i guess
#asahi-dev · 2022-03-21 16:42
svenif it's actually a race is in the controller firmware it won't help
#asahi-dev · 2022-03-21 16:38
sveni forgot that that branch doesn't have the device tree changes itself :D
#asahi-dev · 2022-03-21 16:37
svenoh, lol, i'm just dumb
#asahi-dev · 2022-03-21 16:36
svenbut apparently I also broke something else
#asahi-dev · 2022-03-21 16:36
svencheckpatch complained about some whitespace issues that clang-format didn't catch and I fixed them
#asahi-dev · 2022-03-21 16:35
svennvme-rfc (which is actually nvme-v1 now)
#asahi-dev · 2022-03-21 16:34
svenlet's see what i messed up
#asahi-dev · 2022-03-21 16:34
sveni thought I only did whitespace changes in that nvme thing, but now it doesn't seem to boot anymore *sigh*
#asahi-dev · 2022-03-21 16:30
sventhere's no trace file for ANS rtkit, it's just text output
#asahi-dev · 2022-03-21 16:15
svensure
#asahi-dev · 2022-03-21 16:14
svenok, so I guess wait until the merge window closes and submit it then?
#asahi-dev · 2022-03-21 16:00
svenalright, https://github.com/AsahiLinux/linux/commits/nvme-rfc should be good enough for the first submission
#asahi-dev · 2022-03-21 15:45
svenbut i expect that in either case the bug will still happen, possibly less frequent because the timing changes
#asahi-dev · 2022-03-21 15:45
sveneither leave it and put the dma_mb after it or replace it with the dma_mb
#asahi-dev · 2022-03-21 14:54
svencan't hurt i guess
#asahi-dev · 2022-03-21 14:52
svenbut we currently assume that the issue is actually concurrent writes to the SQ and CQ which then trigger a bug in the firmware. for that the lock would actually be the best solution
#asahi-dev · 2022-03-21 14:51
svenadd dma_mb(); just below the writel if you want to test that
#asahi-dev · 2022-03-21 14:50
svennot yet, just preparing everything
#asahi-dev · 2022-03-21 14:50
svenno
#asahi-dev · 2022-03-21 14:50
svenwhat nvme submit? and what patch?
#asahi-dev · 2022-03-21 14:50
svenhuh?
#asahi-dev · 2022-03-21 10:13
svenif it still reboots it can't be because of that thing then!

plus ›