#
✕ nick:marcan
#asahi-dev · 2023-09-10 12:13
marcanand then all KVM has to do is save/restore context for these things
#asahi-dev · 2023-09-10 12:13
marcanwhich means if we're doing that we might as well do what apple intended here, and just expose this raw to the guest and let it deal with it the same way my patch already does on bare metal
#asahi-dev · 2023-09-10 12:12
marcanone thing to keep in mind is that if we ever enable the APFLG feature especially, that one *definitely* has to be exposing this impdef stuff to EL0, and there's no trapping this
#asahi-dev · 2023-09-10 12:10
marcan(and nobody cares for our use case)
#asahi-dev · 2023-09-10 12:10
marcanI mean that's pretty obvious)
#asahi-dev · 2023-09-10 12:10
marcansure, but I think it's fair to say if you start enabling IMPDEF features migration is fscked
#asahi-dev · 2023-09-10 12:09
marcanyeah, and the point of upstreaming is... not having that problem :)
#asahi-dev · 2023-09-10 12:09
marcanmodules yes
#asahi-dev · 2023-09-10 12:08
marcanmaz: well, we all know how stable the Linux kernel APIs are and how sustainable modules are long-term...
#asahi-dev · 2023-09-10 12:07
marcanI'll keep my name then (some of the apple names are terribad)
#asahi-dev · 2023-09-10 12:06
marcannah it's definitely SPSR semantics, not PSTATE semantics
#asahi-dev · 2023-09-10 12:06
marcanwhich I think apple actually calls APSTATE_EL1? so maybe it's not exceptions? let me check...
#asahi-dev · 2023-09-10 12:05
marcanah, it's in NZCV bits 26-27 and then save/restore on exceptions is via ASPSR_EL1 bits 1-2 (which is also IMPDEF).
#asahi-dev · 2023-09-10 11:58
marcansupposedly NZCV which should map to CPSR bits unless they added state somewhere else...
#asahi-dev · 2023-09-10 11:54
marcansorry not CPSR, APSTATE I think? which I think is IMPDEF
#asahi-dev · 2023-09-10 11:49
marcannot sure what the rest of the bits do exactly. those 3 are the ones we care about for x86 emu, and TSO is by far the most important one.
#asahi-dev · 2023-09-10 11:48
marcanbit 4 sticks some extra x86 flags in CPSR (I think?)
#asahi-dev · 2023-09-10 11:47
marcanbit 5 is a pre-spec version of FEAT_AFP
#asahi-dev · 2023-09-10 11:47
marcanthere are some other ACTLR bits described here: https://github.com/AsahiLinux/docs/wiki/HW:ARM-System-Registers#actlr_el1-arm-standard-not-standard
#asahi-dev · 2023-09-10 11:44
marcanfor exposing TSO to the VM dynamically, we either disable trapping that reg (and AIDR) and signal it somehow, or we provide some kind of hypercall to poke it.
#asahi-dev · 2023-09-10 11:44
marcanand then ideally signal the VM somehow, so software running in the guest can know it's in a TSO environment
#asahi-dev · 2023-09-10 11:43
marcanso basically, for VM-wide TSO, we need to poke ACTLR_EL12 (e.g. copy it from ACTLR_EL2 to inherit whatever the host configured for that process, or something else)
#asahi-dev · 2023-09-10 11:40
marcanmaz: https://gist.github.com/marcan/9ab73ca0614864bea0eea9e953c074d3 it's pretty simple
#asahi-dev · 2023-09-10 11:25
marcaneiln: will look at camera again a bit later, going to sort out this TSO thing first :p
#asahi-dev · 2023-09-10 11:19
marcanre ACTLR, I just realized I do know how this works with EL1 since obviously I've tested this in m1n1 HV. I'll write a quick spec.
#asahi-dev · 2023-09-10 11:13
marcanI'm not particularly keen on the "this will be downstream forever because upstream won't take it in any form" outcome :/
#asahi-dev · 2023-09-10 11:12
marcanmaz: I mean, eventually this should be upstreamed in *some* form. Especially if we're going down the VM route, it will be incredibly silly to require downstream patches on the guest kernel when hardware support is otherwise not an issue.
#asahi-dev · 2023-09-10 09:26
marcanI'd be happy with enabling TSO globally in the VM (Apple already do this for Rosetta on Linux VMs on macOS anyway), we just need to signal it to the guest somehow so FEX can pick it up.
#asahi-dev · 2023-09-10 09:25
marcan(I know you're not going to like any of this, but I hope we can come up with at least a least-horrible solution because, really, using TSO is non-negotiable here, the perf boost for x86 emu is major)
#asahi-dev · 2023-09-10 09:23
marcanso we might at the very least need to copy ACTLR_EL12 (impdef) <= ACTLR_EL2 to make the prctl work "automatically" for kvm host processes
#asahi-dev · 2023-09-10 09:22
marcanactually I'm not sure if the guest ACTLR behaves as intended already, since apple came up with an IMPDEF ACTLR_EL12... I feel like I looked into this already but I'm not sure any more. it might be that right now EL1 always gets default ACTLR (ACTLR_EL12) even if EL2 set something in its ACTLR_EL2, regardless of what is enabled in HACR_EL2.
#asahi-dev · 2023-09-10 09:10
marcanfor ref, the TSO series is here: https://github.com/asahilinux/linux/tree/bits/220-tso
#asahi-dev · 2023-09-10 09:08
marcanalternatively we could have higher level hypervisor calls to forward this to prctl without dropping all the way to qemu or whatever, since I assume a vm exit (or more) to qemu for every context switch in the guest is a really bad idea
#asahi-dev · 2023-09-10 09:08
marcanbut of course it would be more efficient to only have TSO in the VM where needed, which would mean forwarding that control to the guest. in principle, AIUI the guest can directly twiddle that bit without any VM exits, but we have to explicitly allow that and it would be IMPDEF of course.
#asahi-dev · 2023-09-10 09:07
marcanbut, that means we need TSO for KVM. right now it's a prctl on the host (not sure if you saw that patch) and I assume it would work as-is to enable TSO globally in the VM if KVM just keeps that state untouched from process context.
#asahi-dev · 2023-09-10 09:06
marcanmaz: so we have another fun challenge now. turns out running x86 games in a 4K VM on a 16K host is actually doable and already proven to work well. that neatly sidesteps the whole 4K kernel pain for us, and might easily be the way to go at this point.
#asahi-dev · 2023-09-10 08:22
marcanjannau: probably yes, back when I did bringup on that I assume I saw it was breaking the display and did the quick fix.
#asahi-dev · 2023-09-10 06:01
marcanin a future where m1n1 can do atcphy stuff and point the display at multiple places, it probably makes sense for it to add the relevant PDs to the framebuffers dynamically
#asahi-dev · 2023-09-10 06:00
marcanyup
#asahi-dev · 2023-09-10 04:35
marcanjannau: with my genpd defer patches, it should work if you don't give the genpd to simpledrm as long as dcp owns it and *it* knows how to handle it. however, if you remove DCP from the device tree, just simpledrm will break without explicit multi pd handling.
#asahi-dev · 2023-09-08 13:14
marcanhttps://oftc.irclog.whitequark.org/asahi-dev/2023-04-10#32054931
#asahi-dev · 2023-09-08 13:14
marcanif no dma-ranges are specified
#asahi-dev · 2023-09-08 13:14
marcanthat is the broken code that runs before the probe() gets called, which means the DMA mask is unset, then defaults to 32bit
#asahi-dev · 2023-09-08 13:14
marcansven: see of_dma_configure_id
#asahi-dev · 2023-09-08 13:11
marcansomewhere in the OF/whatever code was defaulting to 32bit range if dma-ranges is not present
#asahi-dev · 2023-09-08 13:10
marcanto get it to (mostly) allow 64bit dma
#asahi-dev · 2023-09-08 13:10
marcanbecause end=0xffffffffffffffff will actually overflow into 0 in the code
#asahi-dev · 2023-09-08 13:10
marcanwe had to add this ridiculous hack: dma-ranges = <0 0 0 0 0xffffffff 0xffffc000>;
#asahi-dev · 2023-09-08 13:10
marcanwithout dma-ranges
#asahi-dev · 2023-09-08 13:10
marcanthere's a stupid 32bit limit that creeps in and was clamping everything to 4GiB, that I remember

plus ›