#
…

2022-06-19

00:53NickLu[m]1 has joined #asahi
00:57fionera[m]<sven> "but that’s possible on these..." <- this was what I wanted to ask ^^ Was curious why parallels on macos doesnt support it
01:04Lucy[m]Does macOS even allow that?
01:07darkapex has joined #asahi
01:08steevi don't have anything that's actually thunderbolt, but i thought it did fionera[m] - https://kb.parallels.com/en/124266 says you can use e.g. an eGPU
01:08steevahh directly
01:14fionera[m]yeah i would like to have the raw pcie device in the vm :D
01:14fionera[m]tho thats probably smth that macos blocks
01:24chadmed has joined #asahi
01:29nicolas17 has quit [Quit: Konversation terminated!]
01:29nicolas17 has joined #asahi
01:54nicolas17 has quit [Quit: Konversation terminated!]
01:54nicolas17 has joined #asahi
02:33PhilippvK has quit [Ping timeout: 480 seconds]
02:52nicolas17 has quit [Ping timeout: 480 seconds]
03:26ptudor_ is now known as ptudor
03:26marvin24_ has joined #asahi
03:29marvin24 has quit [Ping timeout: 480 seconds]
03:33nico_32 has quit [Ping timeout: 480 seconds]
03:57MichaelMesser[m]I assume that would apply to VMs as well.
03:57marcanthat parallels link is for intel machines
03:58Ry_Darcy has joined #asahi
03:58nico_32 has joined #asahi
03:58marcanM1 VMs on macOS do not support passthrough of anything and definitely not eGPUs
03:59MichaelMesser[m]macOS on Intel doesn't support passthrough either
04:00marcanah, that link is only about paravirt graphics, not passthrough
04:02marcanit's technically possible to make eGPUs work (with performance limitations) on M1 devices, but it's like a whole driver refactoring/development project to pull it off in a way that can work
04:03marcanso you'd have to find someone well versed in graphics and they'd likely have to spend months working on it, for each GPU driver/vendor, and then everything needs to be upstreamed, and upstream might not like the (rather intrusive) changes it brings
04:03marcanlikely needs userspace mesa changes too
04:03marcanand you'd still end up with pathologically bad performance for some workloads
04:06marcanVMs don't change any of that (the guest still needs the same driver changes)
04:07marcantechnically with a VM you could hook all BAR accesses and make it "work" with no changes to the guest, but performance would be hilariously bad
04:07marcanlike 100x slower bandwidth than native VRAM accesses bad
04:09marcanwould be trivial to test that with the m1n1 hypervisor once thunderbolt/pcie stuff is in asahi if someone wants to find out just how bad it would be
04:09marcanmostly just completing unaligned r/w support to make it work
04:43Ry_Darcy has quit [Remote host closed the connection]
04:46Ry_Darcy has joined #asahi
04:51MichaelMesser[m]Does anything other than eGPUs use this feature?
05:05marcannot to my knowledge
05:06marcanpotentially weird GPU-like things (like FPGA boards with lots of onboard RAM) could
05:06marcanbut the problem with GPUs is that making them work "properly" the sane way involved potentially modifying every app/game
05:07marcanGPUs are really the only device where this is exposed to a huge variety of applications
05:07marcanif you have something more niche you can just work around the problem in the driver/app much more easily
05:07marcan*involves
05:10MichaelMesser[m]So if Apple does not care about eGPUs, this is unlikely to be fixed in later hardware?
05:12Lucy[m]Any way to know if it's fixed in the M2? Probably not yet, right?
05:41chadmedi wonder if they even consider it a bug at all
05:42marcanthis isn't unique to the M1, it's the same on may other ARM systems
05:42qeeg has quit [Remote host closed the connection]
05:43chadmedyeah i doubt theyll fix it
05:43marcanit's not a "bug", it's software and hardware assuming everything is like x86
05:43marcanthey *could* make it like x86 to make this work, but it not being like x86 isn't a bug
05:43chadmednot having eGPU support also encourages software vendors to fix their applications to work better on AGX rather than direct customers to just go buy GPUs that
05:43marcanthat too
05:43qeeg has joined #asahi
05:45Lucy[m]Didn't you say that performance would be impacted though?
05:45marcanthe workaround you'd have to do to make it work impacts performance
05:45chadmedand even if they did make device memory look like x86, i doubt many PC consumer grade cards would work out of the box because of the weird Intel-specific DMA address filtering which itself would require software workarounds that would nuke performance in any case
05:45chadmedso theres no point fixing any of this
05:45Lucy[m]Right. Makes sense then
05:46chadmedthis has all already been experienced on ppc64 workstations
05:46chadmedvery very few pcie cards work properly on those, theres like 3 consumer grade cards in total that work as intended on talos boards for example
05:47chadmedits going to be an interesting tug of war once arm platforms start becoming commonplace on less "integrated" stuff than apple's
05:48chadmedwho will capitulate first? the add in cards or the soc vendors
05:49marcanI think we actually have yet to still 100% validate that this is not supported for PCIe with any memory type other than Device-nGnRE
05:49marcanI just looked up the ARM spec for PCIe integration, and it says Normal-NC is allowed (but not Normal)
05:49marcanwe've never tried Normal-NC, just Normal
05:50marcanI'm not holding my breath but it's worth checking
05:50marcanmight do it a bit later
05:51marcan(Normal-NC would make eGPUs work sanely)
05:53chadmedare cards expected to gracefully handle partial writes on x86?
06:19the_lanetly_052 has joined #asahi
06:41kettenisas far as I know NVIDIA uses Device-GRE mappings instead of Normal-NC in their (proprietary) graphics stack
06:44marcanthat won't work with userland that tries to make unaligned accesses to VRAM
06:44marcanwhich some userland does
06:46kettenisit'd be slow since the kernel would emulate the misaligned loads and stores
06:47kettenisand presumably NVIDIA's userland code doesn't do this
06:48kettenis(I mean unaligned accesses to VRAM from userland)
06:48marcanit's up to individual applications, not nvidia's userland code
06:48marcane.g. memcpy will often do unaligned accesses for performance
06:48marcanthat's why we can't just fix this in drivers
06:49marcanthere is no way unaligned emulation will perform well when literally memcpy hits it
06:50ketteniswell, does OpenGL/Vulkan/CUDA allow mapping VRAM directly into userland
06:50kettenisor is that an Intel/AMD extension?
06:51marcanI think every GPU driver allows that?
06:51kettenisthe usb ones certainly don't ;)
06:52marcando USB GPUs exist, besides that abomination Lina is working on?
06:52marcan(DisplayLink is not a GPU, it's a display controller)
06:53kettenisI was thinking of displaylink as an example of something that doesn't expose a VRAM framebuffer
06:54kettenisbut yeah, not really a GPU
06:55marcanyeah, this isn't about framebuffers, those end up being driver-managed; it's about things like texture buffers, VBOs, etc.
06:58kettenisI'm not familliar enough with the various graphics APIs, but at least some of the OpenGL functionality to directly map stuff like that from VRAM is an extension that drivers don't have to implement
07:01marcanwhich you're part of :-)
07:01kettenisah, well, KVM throws in a whole other complication
07:01kettenisand usually these discussions happen in the context of the open-source (Mesa) graphics stack
07:02ketteniswhich defenitely uses memcpy and assumes that non-aligned access works
07:04kettenisit'd be interesting to see what an Apple Silicon Mac Pro looks like and whether Apple is going to support PCIe GPUs in those
07:05marcanstill need to test that normal-NC really does not work
07:09bisko has joined #asahi
07:20marcannope, does not work, and in fact you get a nice AMCC panic from macOS
07:20marcanpanic(cpu 0 caller 0xfffffe0013443760): "AMCC PLANE3 PIO request with RO flag set error: INTSTS 0x0000000000400000 AFERRLOG0/1/2/3 0x101000/0x17f1406/0x2000000/0x40001 ADDR 0x600101000 CMD/SIZE/TYPE 0x14(CifNCWr)/0x7f/0x1 AID/TID 0x10/0" @AppleT8101PlatformErrorHandler.cpp:1323
07:20marcanpresumably RO=reorder
07:20marcannGnRnE and nGnRE both work for PCIe BARs
07:21marcannot sure how nGnRnE is supposed to actually work since PCIe writes are posted, but at least the fabric does not complain
07:21bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
07:21ketteniswith nGnRnE you simply don't get the benefits of posting at the CPU level
07:22marcanyeah, it's just still posting behind the scenes which seems a bit odd
07:22ketteniscompatible with x86 ;)
07:23marcanI really need to reverse engineer that AMCC error handler stuff, it'd be very useful
07:23ketteniswonder wether GRE or GnRE works
07:23marcan(I only have some stub stuff in m1n1 experiments)
07:23kettenissince G is what you really care about for mapping "prefetchable
07:23kettenis" PCI bars
07:26kettenisbut even then, it is probably not worth it spending time on making eGPUs work
07:26marcankettenis: tested they do
07:26marcanbut yeah, won't fix the alignment problem
07:27marcanall Device modes work for aPCIeC, but Normal modes do not
07:55kettenisso that means that all pci drivers that use ioremap_wc() to map a prefetchable BAR won't work on these machines
08:04mikoxyzzz has joined #asahi
08:14mikoxyzzz is now known as miko
08:28miko has quit [Quit: WeeChat 3.5]
08:29miko has joined #asahi
08:30miko has quit []
08:30miko has joined #asahi
09:11Ry_Darcy has quit [Remote host closed the connection]
09:19the_lanetly_052 has quit [Ping timeout: 480 seconds]
09:29c10l has quit [Quit: Bye o/]
09:33c10l has joined #asahi
09:53the_lanetly_052 has joined #asahi
10:20the_lanetly_052 has quit [Ping timeout: 480 seconds]
10:38miko has quit [Quit: WeeChat 3.5]
10:38gabuscus has joined #asahi
10:47fionera[m]Oh holy I really started a discussion about that :) I was interested because I was wondering if I can pass it to a VM. Either Windows or Linux and not only GPUs but also other PCIe Devices like Network Cards. (Yes I use my eGPU case to test enterprise NICs on my Laptop)
10:50c10l has quit [Quit: Bye o/]
10:54c10l has joined #asahi
11:09kettenisthe answer is pretty much no for linux (either passthrough or native)
12:07miko has joined #asahi
12:53milek7_ GL_MAP_COHERENT_BIT is required from gl 4.4
12:54milek7_before that I think it is possible to get away with intermediary buffer in the driver
13:17chadmed has quit [Ping timeout: 480 seconds]
13:18chadmed has joined #asahi
13:40MajorBiscuit has joined #asahi
15:08amarioguy has joined #asahi
15:09unrelentingtech<chadmed> "and even if they did make device..." <- what is the intel specific dma address filtering thing? amdgpu generally just works on non-broken aarch64 with no workarounds
15:24jakebot6 has quit [Quit: The Lounge - https://thelounge.chat]
15:30jakebot6 has joined #asahi
15:38___nick___ has joined #asahi
16:07MajorBiscuit has quit [Quit: WeeChat 3.5]
16:15miko has quit [Quit: WeeChat 3.5]
16:18miko has joined #asahi
18:05Moprius has joined #asahi
18:33osaka1990 has joined #asahi
18:34osaka1990 has quit []
18:35osaka1990 has joined #asahi
18:56Ry_Darcy has joined #asahi
19:15qeeg has quit [Ping timeout: 480 seconds]
19:18qeeg has joined #asahi
20:08___nick___ has quit [Ping timeout: 480 seconds]
20:37ptudor_ has joined #asahi
20:40ptudor has quit [Ping timeout: 480 seconds]
20:49konradybciohey, I was trying to figure out PMGR stuff for A(n) SoCs and got a little confused.. are the powerstates (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/arch/arm64/boot/dts/apple/t8103-pmgr.dtsi?h=next-20220531) a new representation of gate clocks? or are they a separate thing?
20:52svenwhat we assumed to be clock gates are actually those power states
20:53konradybciooh so they're votable?
20:53konradybcioas in, more than just on/off?
20:54svenyeah, there are at least three states (off, clock gated(?) and on)
20:54svenand they can do auto power management
20:54konradybciosounds convenient
20:55sventhey also usually don’t need explicit support in the drivers
20:55konradybciounlike some other SoC vendors' dvfs infra..
20:56svenas long as a device is only attached to a single power domain the genpd core will handle everything automatically
20:59j`eywhich only seems to be nvme and mca
21:01ptudor_ is now known as ptudor
21:02j`eywonder why sound/ is a separate subfolder and not in drivers/
21:02svenhuh, true. that’s weird
21:02sveni bet it’s for some obscure historical reason :D
21:02miko has quit [Quit: WeeChat 3.5]
21:42yuyichao has joined #asahi
21:43LunaFoxgirlVT has joined #asahi
21:51qeeg has quit [Ping timeout: 480 seconds]
22:07qeeg has joined #asahi
22:18LunaFoxgirlVT has quit [Quit: Leaving]
22:31manawyrm has quit [Quit: Read error: 2.99792458 x 10^8 meters/second (Excessive speed of light)]
22:31manawyrm has joined #asahi
22:37MichaelMesser[m]s/the/that/, s/hardware_families/hardware\_families/, s/implementing_a_pcie_kext_for_a_thunderbolt_device/implementing\_a\_pcie\_kext\_for\_a\_thunderbolt\_device/
22:42Moprius has quit [Ping timeout: 480 seconds]
22:58confusomu has joined #asahi
23:05Ry_Darcy has quit [Remote host closed the connection]
…