#
…

2022-02-06

00:13phireoh this is cool, I can see it sending the gpu_va and size of my userspace allocations to the gpu firmware
00:15phireprobally a cache flush or memory clear command
00:40m6wiq has quit []
01:21phirebut this communication protocol is a bit weird
01:23phireit writes the VA to gfx-handoff+0xa0, the size to +0xa8 and then writes 2 to 0x98
01:23phirethen somehow the gpu firmware knows to read that shared memory
01:24phirethen I think an interupt comes back and the cpu sets gfx-handoff+0x98 back to 0
01:25phireI need to work out how to trace interrupts
01:33alyssaphire: Naively, I would assume Apple uses ASC messages in place of direct interrupts
01:34alyssa(Of course ASC fifo non-empty is an IRQ but Abstraction)
01:39phirethere is very little traffic going across the gpu firmware's ASC
01:39phireit's bugging me
01:40alyssato confirm you're looking both sgx and gfx-asc together?
01:41phiresgx has even less register traffic than gfx-asc
01:41phireand it's only reads
01:41alyssaright :|
01:42alyssawe're certain gfx-handoff is ordinary shared memory and not somehow special / MMIO?
01:43phireIt's possible the firmware gets an interrupt every time there is a write to it
01:43phirebut I do think it's real memory
01:45alyssawhat is the ASC traffic, then?
01:45phiregpu sometimes sends 0x43 on EP20
01:45phirecpu sometimes sends 0x83 with a 8bit value on EP21
01:46alyssawhat endpoints are those?
01:46phireI don't have names for them
01:47alyssamind pastebinning the output of `ioreg -l`?
01:47alyssabooted into linux right now
01:47phireha, I saved a copy of that
01:49alyssaforgot how thick it is
01:49phirealso, this is the M1N1 trace acound the gpu allocation stuff: https://gist.github.com/phire/9cad6e6bbe4add5f8f9dcc55d5a03dbc
01:50alyssahm i thought ioreg had endpoint stuff, boo
01:51phiremight be somewhere else?
01:54alyssaphire: that's a lot more ASC traffic than you made it sounds like ;)
01:55phirethe stuff at the top is just re-initilization. I think it happens whenever the gpu is bought out of sleep
01:56alyssathe 0xdead writes are ominous ;-p
01:56phirebesides, I'm mostly talking about entropy wise
01:56phiremight be because this is a debug kernel
01:57alyssaI just mean that every time the CPU sends the FW an ASC message, that's an opportunity for the ASC to check gfx-handoff, right?
01:57alyssaer.. are gfx-handoff writes traced here?
01:58phire(unknown, offset 0x10ffffb4nnn) are gfx-handoff
01:58alyssaokay yes i see V_V
01:58phirethe asc traffic itself is mostly just <ep:20 0042000000000000, >ep:21 0083000000000000
02:00phireif you clear the value at gfx-handoff+8, the kernel does the following panic:
02:00phirepanic(cpu 4 caller 0xfffffe0013c5d848): UAT PPL 0xfffffdf030af4160 (IOUAT): Invalid microPPL magic value found in the handoff region. Expected: 0x4b1d000000000002, Actual: 0x0
02:01phireI admit, things would make much more sense if gfx-handoff was an mmio region
02:01alyssasven: ^^
02:05alyssahttps://googleprojectzero.blogspot.com/2020/07/ came up in two google searches in a row
02:05phireseems to be memory, it holds any writes
02:06phire(it's also located in physical memory)
02:07phirethe cache coherrency protocol could theortically tell the gpu firmware that memory has been updated, but that's a really janky hack
02:07alyssathe table there is kinda interesting
02:07alyssa 0x239028000, 0x4000, 0x4007, 'DART' ; dart-usb
02:07alyssa0x267030000, 0x4000, 0x4007, 'DART' ; dart-ave
02:07alyssa 0x8FC3B4000, 0x4000, 0x40004016, 'GUAT' ; sgx.gfx-handoff-base
02:09phireI think that just comes from ADT
02:09alyssaI meant darts being grouped with the handoff
02:10phirethere are also carveouts there
03:10baddek has joined #asahi-gpu
03:10baddek` ्बहवDड i डd u` sड a t r डa iड n , & s uड p `pl y i डs ड` i s wi th w e. aड .p o डn s l i k e i t d i d w i t h a. l ड q. a डe d a to j. u sडt i f y c r e a डt i n g w. aड r s C. R Eड A Tड I V E C .H डAO S- d i d cड i` a d i d 9.डड11 . or, i
03:10baddekif . a. l ड q. a डe d a / ` d्i ्d .. it w h y , ` t o k `गi lग l 9 . m i जl` जl i on . i r` a q i s
03:10baddeke e a सs t
03:10baddekt j u डs t l e t ` it h aड pड pड en * .. a m e ड r i c` a n s ne eसd e d someसthiसng l i k e 9.स 1 1 to j /u sस t i f y i nस v `a dस i n g .i सr /a सq ` w h i c h/ has b e e n ...alrसea dy p l a n स n e d as a p art of c rस e/ a t सi v e c hस a` o s p l a n for . m i d सd l
03:10baddek has quit [Excess Flood]
03:10baddeka b o u t 9 m `il तi o n - iहr a ह`q i l o s t ` thल e i r li`ल vलes e i th e r be ca uz . o f di्r e ct ameri`can vi ्ol e`nce ` and .. . beca uz /of si ्e`ge . u दs दa ma de . a`g दa iदn s t i दrद aद q t h at pr eदveद nt ed iद r aद qi f or m es se nt ial ` l i v i n g ` ma
03:30bisko has quit [Server closed connection]
03:31bisko has joined #asahi-gpu
03:59PhilippvK has joined #asahi-gpu
04:01phiologe has quit [Ping timeout: 480 seconds]
04:42darkapex has quit [Ping timeout: 480 seconds]
04:42phireWeird... the only interrupt actually in use across both /arm-io/sgx and /arm-io/gfx-asc is the asc interrupt
04:46jbowen has quit [Ping timeout: 480 seconds]
06:15threerik[m] has quit [Server closed connection]
06:15threerik[m] has joined #asahi-gpu
06:37_jannau_ has quit [Server closed connection]
06:37_jannau_ has joined #asahi-gpu
06:38jbowen has joined #asahi-gpu
06:46jbowen has quit [Ping timeout: 480 seconds]
06:47sheepgoose has joined #asahi-gpu
06:53al3xtjames has quit [Quit: al3xtjames]
06:55al3xtjames has joined #asahi-gpu
07:07jbowen has joined #asahi-gpu
07:07mini has quit [Server closed connection]
07:07mini has joined #asahi-gpu
07:19mattmurr[m] has quit [Server closed connection]
07:19mattmurr[m] has joined #asahi-gpu
07:21jbowen has quit [Ping timeout: 480 seconds]
07:38darkapex has joined #asahi-gpu
08:07jbowen has joined #asahi-gpu
08:16jbowen has quit [Ping timeout: 480 seconds]
08:25jbowen has joined #asahi-gpu
08:33jbowen has quit [Ping timeout: 480 seconds]
09:10NotHere[m] has quit [Server closed connection]
09:10NotHere[m] has joined #asahi-gpu
09:12jbowen has joined #asahi-gpu
09:14nsklaus has joined #asahi-gpu
09:16rethematrix[m] has quit [Server closed connection]
09:16rethematrix[m] has joined #asahi-gpu
09:19ryanhrob[m] has quit [Server closed connection]
09:19ryanhrob[m] has joined #asahi-gpu
09:21jbowen has quit [Ping timeout: 480 seconds]
09:23nsklaus has quit [Quit: Textual IRC Client: www.textualapp.com]
09:27Rhys[m]12 has quit [Server closed connection]
09:27Rhys[m]12 has joined #asahi-gpu
09:40yrlf has quit [Server closed connection]
09:40yrlf has joined #asahi-gpu
09:58JoshuaAshton has quit [Server closed connection]
09:58JoshuaAshton has joined #asahi-gpu
10:13jbowen has joined #asahi-gpu
10:19nsklaus has joined #asahi-gpu
10:22jbowen has quit [Ping timeout: 480 seconds]
10:48Major_Biscuit has joined #asahi-gpu
10:56chengsun has joined #asahi-gpu
10:57chengsun_ has joined #asahi-gpu
11:04chengsun has quit [Ping timeout: 480 seconds]
11:08jbowen has joined #asahi-gpu
11:17jbowen has quit [Ping timeout: 480 seconds]
11:39chadmed has quit [Remote host closed the connection]
11:52chadmed has joined #asahi-gpu
11:57m6wiq has joined #asahi-gpu
11:58Major_Biscuit has quit [Ping timeout: 480 seconds]
12:09jbowen has joined #asahi-gpu
12:18jbowen has quit [Ping timeout: 480 seconds]
12:28Major_Biscuit has joined #asahi-gpu
13:10jbowen has joined #asahi-gpu
13:19jbowen has quit [Ping timeout: 480 seconds]
13:21kit_ty_kate has joined #asahi-gpu
13:44jbowen has joined #asahi-gpu
13:53alyssaA very sleepy Alyssa is about to stream M1 GPU driver dev ^^
13:53jbowen has quit [Ping timeout: 480 seconds]
14:09alyssaokay, I think OBS is reconfigured
14:09alyssa(...so sleepy)
14:17matthewayers[m]Is there a link to the stream?
14:17Major_Biscuit has quit [Ping timeout: 480 seconds]
14:21m6wiq1 has joined #asahi-gpu
14:27m6wiq has quit [Ping timeout: 480 seconds]
14:40m6wiq1 has quit [Ping timeout: 480 seconds]
14:45jbowen has joined #asahi-gpu
14:51m42uko has joined #asahi-gpu
14:54jbowen has quit [Ping timeout: 480 seconds]
14:55m42uko_ has quit [Ping timeout: 480 seconds]
15:30jbowen has joined #asahi-gpu
15:39jbowen has quit [Ping timeout: 480 seconds]
15:50Tuff has quit [Read error: Connection reset by peer]
15:57m6wiq has joined #asahi-gpu
16:19jbowen has joined #asahi-gpu
16:28jbowen has quit [Ping timeout: 480 seconds]
16:51m6wiq has quit [Remote host closed the connection]
16:51m6wiq has joined #asahi-gpu
17:09phireHey, this page table is actually 3 layers deep, and points at real-looking data
17:18Major_Biscuit has joined #asahi-gpu
17:20jbowen has joined #asahi-gpu
17:23blassphemy[m] has joined #asahi-gpu
17:26phireok... gpu-region is the context table
17:27phirecontains pointers to the page table for each context
17:28jbowen has quit [Ping timeout: 480 seconds]
17:45phirethe other page table (at gfx-shared-region + 0x10) might be for context -1, which shows up in logs
17:54alyssacontext -1, hm
17:55phireyeah, we can probally ignore that for now. right now I want to know how userspace command lists get executed
17:58alyssaOn Mali it's as easy as "write an address to a register and write the START command to the command register"
17:58alyssaI imagine AGX has a lot more indirection.
18:00phireMy current theory is that there is some kind of memory structure with a list of command lists
18:00phireand you send a message via ASC to tell it which one to execute
18:01alyssathat would be guess, yes
18:02phirewith the amount of entropy I've got, that's the best susgestion
18:06phirethat structure must be global, since the ASC message doesn't have a context id
18:12* alyssashould get back to coding instead of randomly perusing gitlab idly
18:13phireThis is probally another thing where there are registers on the sgx interface that control all this, but there is an ASC firmware in the middle
18:13Major_Biscuit has quit [Ping timeout: 480 seconds]
18:20* alyssapowers the mac back on
18:21jbowen has joined #asahi-gpu
18:22alyssai must say it'll be nice finally having 3D accel on linux
18:30jbowen has quit [Ping timeout: 480 seconds]
18:55jbowen has joined #asahi-gpu
19:00m6wiq1 has joined #asahi-gpu
19:03jbowen has quit [Ping timeout: 480 seconds]
19:06m6wiq has quit [Ping timeout: 480 seconds]
19:39alyssaokay i swear i'm starting again i'm just real good at not doing things
19:56m6wiq1 has quit []
19:56m6wiq has joined #asahi-gpu
20:22Tramtristwow .. i went grocery shopping and came back.. and you're still going at this pace.. quite amazing
20:55jbowen has joined #asahi-gpu
21:03jbowen has quit [Ping timeout: 480 seconds]
21:45phirethe gfx-shared-region + 0x10 page table is actually a L2 leaf, that gets inserted to cover the range starting at 0xffffffa000000000
21:46phireand the only address that gets sent over ASC (which must be our control structure) is 0xfa0000b0000
21:48phirethis pagetable also maps IO registers into it
21:48phire0xffffffa010000000-0xffffffa010003fff -> 0x404d00000
21:49phirethat is the sgx's register range
21:52akemin_dayo has quit [Ping timeout: 480 seconds]
21:55phireit also maps in some ranges I don't reconise: 0x20e100000, 0x28e494000, 0x204d60000, 0x200000000, 0x201000000, 0x202000000, 0x203000000, 0x204000000, 0x205000000, 0x206000000, 0x207000000, 0x2643c4000, 0x28e3d0000, 0x28e3c0000
22:09Tuff has joined #asahi-gpu
22:09Tuff has quit []
22:40akemin_dayo has joined #asahi-gpu
22:47msinkovics has joined #asahi-gpu
22:48msinkovics has left #asahi-gpu [#asahi-gpu]
22:49user982492 has joined #asahi-gpu
22:55m6wiq has quit []
22:55jbowen has joined #asahi-gpu
22:59Tramtristi was amazed at everything you were doing there alyssa from the dev environment on up
23:00Tramtristinspiring really
23:00yrlf has quit [Quit: The Lounge - https://thelounge.chat]
23:01yrlf has joined #asahi-gpu
23:03jbowen has quit [Ping timeout: 480 seconds]
23:59user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
…