#
…

2021-08-06

00:41phireWell, you can already use Linux in a VM under macos
00:41phireWhich is almost the same as a hypervisor running both?
01:18dgc[m]Not almost enough for me :)
01:19dgc[m]I've done that for years, but I really would like the linux system not to be hostage to MacOS. And the inverse, running MacOS on a linux HV, suffers from HackOS not really being all quite there in the head.
01:20dgc[m]I truly just want a computer that can treat multiple OSes as first-class simultaneously. Not problem-solving here, just signing on to this as a great goal.
01:30chadmedthats something that sounds trivial in your head when you think about the basic problem definition but then you try to implement it and it ends up taking you 5 years and squillions of lines of code
01:31dgc[m]eh, I don't think it's trivial. But it's something I've not heard of anyone working on since Mach and the little I've heard on your project here sounds like it could be a lead.
01:33dgc[m]If that's nowhere near scope, my apologies. I'm really not asking for anything — just excited to hear of someone working on something interesting.
01:33chadmednah its obviously not trivial, hence why i said its something that can sound trivial in your head but really isnt once you try to implement it
01:34dgc[m]I guess maybe it can sound trivial in your head, I don't know.
01:35chadmedis there any real benefit to doing it that way rather than, say, an API translation layer a la wine?
01:35dgc[m]what I would enjoy is complete system independence. I'm not looking to run linux binaries on macos or vice versa. I don't care about that at all, I have linux machines and macos machines I can run those on.
01:36dgc[m]But I would like to have only one device to do it on
01:36dgc[m]when macos inevitably crashes after 8 or 9 days I'd like my opensolaris system with 190 days of uptime to still be running. that kind of thing.
01:37dgc[m](opensolaris was the first thing I ran as a VM on OSX)
01:37chadmedwell yeah thats kinda what im getting at. i appreciate the philosophical idea of independence that such an approach would bring, but what practical benefit does on-the-fly "kernel switching" have over a wine-style implementation?
01:38chadmedyou get the stability, uptime and freedom of (pick your *nix) but the ability to run all your macos applications as normal
01:38dgc[m]I don't know, that's not something I want so I can't speak toit
01:39dgc[m]OK. let me back up on this.
01:39chadmedbtw i am genuinely curious as to what youre thinking so i can better understand what you want out of the system, im not trying to put you down or anything like that :)
01:41dgc[m]Where I'm coming from is that I value being able to run the whole ecosystem. I want a Mac with the whole Mac environment on it. I also want a beos box (not really, this is just to abstract the capability from a specific use case, which I don't have). It's not about being able to run omnigraffle because it's better than any structured drawing tools on windows or linux. It's not about applications. I want to have both operating environments —
01:41dgc[m]VM-style — but not have to physically switch plastic to do it.
01:42chadmedright so the benefit would be to have the OSs "containerised" to a degree but allow on-the-fly switching between the two running close to metal
01:42dgc[m]Honestly having a beos machine and a mac on my desk with a full keyboard/video/mouse switch in front of it accomplishes my entire goal, except for being able to pack it into a suitcase and take it on a plane.
01:43dgc[m]I always hoped that Mach would accomplish this but in practice, nobody ever really took it there
01:43dgc[m]actual implementations of mach (next, DEC, even apple) were single-server
01:44dgc[m]Mainframes did this well
01:45dgc[m]nobody has figured out how to virtualize operating systems as well as IBM did in 1970.
01:45chadmedperhaps its issues in dealing with IPC prevented it from practically being taken further
01:45chadmedim sure you could find an airline willing to haul your system/z machine around with you :-P
01:46dgc[m]that would be pretty funny if I were elon musk
01:46chadmedthin hypervisor implemented entirely in PL/I, sounds like what you want to me lmao
01:47dgc[m]I've been meaning to install hercules on a raspi, I wonder how useful I could make that
01:48dgc[m]Well, you're right that this isn't actually useful to very many people.
01:49dgc[m]I suppose Iament that computing at large never committed to this road. I thought that maybe this idea I read about in the trade pulp about Asahi had something of that concept to it, but maybe not.
01:56chadmedi consider it a hypervisor in that it pretends to be XNU kernel so that iBoot will set up the hardware and jump to it, and allow it to set up a more "sane" environment for loading Linux and Linux-adjacent stuff
01:58chadmedi had this discussion with a mate the other day who was dismayed that he is unable to use Boot Camp on his new macbook. conceivably, m1n1 may at some point be useable to get a boot sequence like m1n1 --> u-boot --> edk2 --> windows boot manager --> nt kernel
01:58dgc[m]I see, so it's new/custom code with specific purpose
01:58dgc[m]the whatever I read sounded like "we ground down kvm to fit on this shelf"
01:59chadmedbut i doubt that is anywhere near the priority list at all, nor would microsoft bother to write or sign drivers for new mac hardware as it would cannibalise surface sales. nor would apple bother writing drivers for arm windows as it would stifle arm macos development
02:04yuyichao has joined #asahi
02:17cptcobalt has joined #asahi
02:22PhilippvK has joined #asahi
02:25phiologe has quit [Ping timeout: 480 seconds]
03:18marvin24 has joined #asahi
03:21marvin24_ has quit [Ping timeout: 480 seconds]
04:13boardwalk has quit [Quit: Ping timeout (120 seconds)]
04:17jkkm has joined #asahi
04:18boardwalk has joined #asahi
04:19opticron has quit [Ping timeout: 480 seconds]
04:22opticron has joined #asahi
04:28krbtgtwrt above i'm not really aware of any lightweight desktop hypervisors. for prior art on similar, you might be interested in the virt on IBM power systems
04:28krbtgtkinda-xen like except not really
04:28krbtgtwith a wacky custom OS underpinning it
05:44Z751 has joined #asahi
05:44Z750 has quit [Read error: Connection reset by peer]
06:16sveniirc the pongoOs/checkrain people (never_released, but i haven't seen him here in a while) wanted to write a small hv to allow running windows
06:16svenno idea if they ever started working on that though
06:18nkaretnikov has joined #asahi
07:13bisko has joined #asahi
07:49phiredgc[m]: m1n1 (at least it's current form/direction) isn't really what you want.
07:49phireIt's a lightwight hypervisor designed to run only a single operating system while giving it nearly full access to hardware
07:50dgc[m]@phire ok, thanks. I heard "lightweight hypervisor" and got my hopes up
07:50phireThough, I agree. The idea of a hypervisor of running two operating systems that feel first class is appealing
07:51j_eydgc[m]: its a hv for tracing MMIO accesses mostly
07:51dgc[m]right - I just had hoped it wouldn't be limited to that (valid!) purpose :)
07:51phireI have schemed about intergrating linux and windows tightly
07:52phiremaking them share a schedular and memory allocator
07:56phireThe real barrior to such ideas is that *most* hardware doesn't suport being controlled by two seperate driver stacks.
07:56phireeither they clash, or the introduce security flaws
07:57phireso your hypervisor is doomed to getting more and more heavy-weight
08:01bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
08:05bisko has joined #asahi
08:13arndOne thing that may be possible is to have a purely partitioning hypervisor that passes through (almost) everything to macos, while giving Linux access to half the CPUs, memory and one USB controller, and then put all the peripherals for the Linux guest on a USB hub.
08:14arndI don't see a real practical application for that kind of setup though, so I don't see who would spend the effort to implement that
08:15pipcet[m]arnd: I did something like that, communicating using "shared" memory (actually magic changing signatures in RAM that linux found and overwrote with its data)
08:15chadmedi think dgc's idea was, as he said, more like Mach where multiple userspaces have access to the entire hardware pool. though theres multiple reasons that approach never really went anywhere
08:16chadmednot least of which being the monstrous IPC overheads in Mach when used this way, and also the overheads incurred by the enormous amount of context switching the machine needs to do in order to make such a setup possible
08:16pipcet[m]arnd: even sharing just USB is hard because the interrupt controller isn't designed to be shared and the two USB ports (their PD chips) share an IRQ line..
08:18arndThere are a number of implementations along that line, in Linux we have 'jailhouse', and Intel is working on 'ACRN'
08:21arndpipcet[m]: I meant the Linux guest would either get the PCI XHCI (type-A on mac mini) ports, or the DCW3 (type-C) ports, or do you mean some bits are even shared between the two sets?
08:22pipcet[m]oh. that makes sense, sorry. I have a MBP :-)
08:26pipcet[m]arguably, it might make sense to let macos initialize the hardware, then kill it and run Linux instead using initialized hardware, but I think it's way more trouble than any benefit we might see...
08:26sveni'm still sad that they didn't hook up the "new usb connection" line to the dwc3 controllers :(
08:27svenfwiw, mac os will lock down *a lot* while initializing the hardware
08:27pipcet[m]sven: is it possible we get a "proper" SMC notification? I just got those to work...
08:27pipcet[m]yes, that's the trouble
08:28svenpossibly. but it's still annoying because we'll have to hook up that notification to the dwc3 driver :/
08:29pipcet[m](you set the "NTAP" flag in the SMC, then you get messages on EP20 with the low byte set to 0x18)
08:29svenhrm. or maybe we'll just need to hook it up to the usb phy driver
08:29sveni think that could actually work. make the usb phy listen for that SMC notification and then just flip the reset bits. dwc3 will actually get a "new device" event then
08:35pipcet[m](we should probably have a nice DT-based API for sharing the SMC between all the drivers that use it, anyway. Currently working on something but it's not quite ready yet.)
08:36chadmedisnt that what mailboxes are for?
08:37pipcet[m](mailboxes are designed to have a few channels. The SMC has ~4000 "properties" that can be read and set. but, anyway, it's quite possible we'll go with a mailbox-based hack in the end)
08:57bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
09:00svenwhat other drivers need to use the SMC in which ways? smc -> usb phy could e.g. just be an interrupt
09:03bisko has joined #asahi
09:04pipcet[m]the ones I know about are battery/power supply, usb, pci (for the power pin), power switch, lid switch, backlight (the on/off switch, not the PWM which goes through DCP), hwmon of course, gpio for when you really want to blow up your mac, system shutdown, (I think the way macos uses the WDT also makes use of the SMC)
09:06Vaughn has joined #asahi
09:17pipcet[m]they all read or write properties, some need interrupts which are provided by the SMC
09:31marcansven: is the different nvme queue thing new for M1? with the larger qes I was referring to the quirk that T2 macs needed
09:31marcanI hadn't realized there were deeper differences this time around
09:31marcan(other than it not being pci)
09:34bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
09:36sveni think so
09:37svennvme works on the T2 mac, doesn't it?
09:37svenif yes, than this queue thing is new
09:37svenapple calls it "nvme-linear-sq" in the ADT
09:38sveni thought previous macs were just that the tags are shared between queues, that there's only a single interrupt and that the queue size is limited
09:43svenoh.. there's also NVME_QUIRK_128_BYTES_SQES, but that's not required anymore on the m1
09:44svenpossibly because they moved the AES stuff to their weird iommu-like thing
09:52marcanah, makes sense then
09:56j_eymarcan: did you see sven's latest iommu pagesize mismatch code?
10:16svenmy plan is to send a RFC soonish anyway just to see if the iommu maintainers will even consider accepting such changes with their disadvantages
10:19eric_engestrom has joined #asahi
10:34chadmeddistros not keen on building 16k kernels i take it?
10:35sveni'm sure some of them will come around eventually, but if supporting 4k kernels with some limitations isn't too painful i'd like to do that
10:36sventhat would at least allow to use the official distro installer + then add a custom package for the 16k kernel after everything already works
10:39JTLI was going to ask about the current situation with IOMMU and page sizes, just out of self interest.
10:39chadmedoh yeah i fully get the rationale for doing it, just as you say the iommu guys may come down on it like a ton of bricks
10:39chadmedmaybe we just force everyone to use gentoo so they *have* to build their own kernel :-P
10:40sventhey weren't opposed to it so far fwiw, but that can change once some of the limitations are clear
10:40svenwhich is why i'll try to prepare that RFC series this weekend to see if it makes sense to continue working in that direction
11:12marcanj_ey: I saw the tweet :)
11:12marcansven: awesome :D
11:14* j_eyfollows sven on tweeter
11:14marcanbtw, for those that missed the installer stream: building a bootable partition "manually" works, copying files from another one. I believe I know how to construct all those files from ipsw contents, so now I just need to write a script to do it.
11:15marcanThe initial installer prototype will probably be a simple thing that you can point at an APFS container and it'll create everything needed to run m1n1, I might add some basic logic so you can just point it at a macOS partition and give it a target size and it'll make space for a secondary linux install next to it and set things up
11:15chadmedwere you able to stream in just the bytes of the ipsw that we need? i played around with it for a bit in python but i couldnt get any useable data out of it :( (i am exceedingly stupid though so maybe i missed something)
11:16marcanI haven't tried that yet but I know that will work. I've done streamed zips before.
11:16marcanIf standalone python bootstrap isn't an issue and I write this in python as I intend to, I expect just making a little cache layer between ZipFile and urllib will do what I want
11:17marcanalso, all of this can be tested/developed under macOS. in fact you don't need recovery at all for the first step of installation *if* you're using a base macOS version that Apple still signs in TSS.
11:18marcan(stage 2, i.e. bputil/kmutil, always has to happen in 1TR, but we know that already)
11:19marcanif you are using an older base macOS then you either need to run the installer from recovery first, or make it a three step process (macOS -> recovery -> 1TR) I believe
11:21chadmedas in, older than monterey? you mentioned some things that got fixed and changed with 12.0 on stream last night
11:22marcanno, Apple will only sign a subset of recent macOS versions for security reasons if you're in "full security" mode
11:22chadmedah right yeah i follow now
11:23marcanyou *can* do a "reduced security" mode install, I actually got a prompt for that while installing macOS once because I guess the network failed?
11:23marcanbut AIUI that can only happen from recovery mode
11:28tophevich[m] has joined #asahi
12:19eichin has joined #asahi
13:38Mary has quit [Quit: The Lounge - https://thelounge.chat]
13:43Mary has joined #asahi
13:44Mary has quit []
13:47yuyichao has quit [Ping timeout: 480 seconds]
13:57alyssapipcet[m]: Do you have system shutdown/reboot patches I can cherrypick on top of mainline? (SMC stuff?)
13:58alyssasven: Also do they work with your ASC driver?
13:59svenuhhh... who knows. maybe?
13:59sventhanks for volunteering to test and fix them!
14:00sventhat asc driver will probably required some changes either way. i'm still not sure the rtkit handling belongs in drivers/mailbox
14:01sven*require
14:31bps has quit [Remote host closed the connection]
14:32bps has joined #asahi
14:37Mary has joined #asahi
15:03alyssasven: Are dma_alloc_noncoherent/dma_alloc_wc expected to work on m1?
15:03alyssa(do I need specific CONFIG options? or more patches? or..)
15:03alyssaor.. maybe I shouldn't be going down this path at all, since I don't actually want CMA memory but rather rando memory mapped to the disp0-dart..?
15:07sventhe framebuffer is coherent fwiw. either by default or because the DART works its magic
15:08svenno idea about dma_alloc_noncoherent
15:11svenlooks like it should work though
15:16svenalyssa: so to have memory mapped to the disp0-dart you need to declare that dart in the DT, then add iommus = <&dart stream-id>; to whatever device you want to use DMA with
15:17svenand then dma_alloc_coherent should just work
15:17sven(internally, the device's dma_map_ops will be set to the ones from dma-iommu.c which will eventually call into apple-dart.c)
15:26bps has quit [Ping timeout: 480 seconds]
15:35marcanalyssa: do not use any of that nonsense
15:35marcanthat is mostly x86 specific
15:35marcanon M1 all memory is coherent, and you want it like that
15:36marcandma_alloc_coherent is probably what you want
15:37marcandon't model your memory management after x86 drm drivers; that is the ~one place weird stuff happens on x86 that has no place anywhere else
15:38marcanon M1 dma_alloc_coherent should be a normal memory allocation
15:38arndIIRC dma_alloc_noncoherent() is a hack that is used for some of the old pa-risc/mips/ia64 numa systems, it has nothing to do with x86
15:39arndotherwise as marcan says, don't use that
15:40marcanI was thinking _wc
15:40marcanwhich ISTR being used to map framebuffers
15:40krbtgtDRM drivers are like the 50th circle of hell per the people i know trying to get them working on non-x86
15:40krbtgtwhich basically means AMD's DRM driver, lol
15:42marcanheh, a bunch of arm drivers also use dma_alloc_wc I see
15:42arndmarcan: You are probably thinking of ioremap_wc() vs ioremap(), which is the right idea for frame buffers to speed up consecutive stores
15:42marcanarnd: yeah but not here, here you just want flat out cached
15:42marcansince we know it's coherent
15:43marcanand do we even do _wc? it probably falls back to uncached if it even works, which would be bad
15:43marcan(I forget how this ended up :))
15:43arndthere is no need to ioremap() here, since it's just normal memory, not mmio
15:43marcan/home/marcan/asahi/git/linux/arch/arm64/include/asm/io.h:#define ioremap_wc(addr, size) __ioremap((addr), (size), __pgprot(PROT_NORMAL_NC))
15:43marcanthought so
15:44arndoh, I wonder why that's different from arm32
15:45arndanyway, if the gpu can see the cache, there is no need for write-through or uncached write-combining mapping of regular memory
15:46marcanso far we have no evidence of *anything* being noncoherent on M1
15:48marcanI think there's a trap bit for cache ops in the VM stuff, maybe I should turn that on and see how much cache management macos does
15:48marcanmight reveal some stuff
15:49marcanand also if I can somehow determine if it's for icache/dcache or something else (maybe always ignore dcache ops and always turn icache invals into dcache flush/icache inval?) I can just disable all device cache management... and see what breaks
16:09bisko has joined #asahi
17:03pipcet[m]well, isb is sometimes necessary, but that's not really noncoherent
17:09pipcet[m]alyssa: I think the SMC stuff is a mess, frankly, they go digging around in nvmem and the device tree for the poweroff sequence. Reboot is easier, there's apple_wdt_restart here https://github.com/pipcet/linux/blob/543af4a24f5561522e8a52aa6bd6bec692d2e93b/drivers/watchdog/apple_wdt.c#L72. Literally four writel()s :-)
17:14bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
17:30j_eypipcet[m]: why did you have to register a restart handler? looks like the core wdt code already deos that
17:31arnidg[m] has joined #asahi
17:33pipcet[m]j_ey: I think you're right, thanks!
17:33gomikun has joined #asahi
17:33gomikun has quit []
17:36pipcet[m]j_ey: I think it was a restart handler that grew a WDT "driver" (not much of one, admittedly), if there's any interest in it I'll obviously clean it up :-)
20:49alyssasven: ACk
20:49alyssamarcan: those are two function calls that different code paths for gem fb alloc use
20:50alyssapresumably _wc is the one to use, just need to make sure that works
20:50alyssawill try adding the disp0 dart to the dt as sven suggested and see what has to happen to make that work
23:25alyssaOk, so can I igure out how to get ssh onto the mac without having wired connectivity hMMM
23:25alyssaI can probably share a connection from my linux machine? maybe?
23:25alyssabetter question is if I have a second usb ethernet adaptor..
23:27roxfan2 has joined #asahi
23:30alyssaMaybe not. Is this fate telling me to cherry pick the PCI patches
23:30alyssamumble. ok
23:30alyssasven: halp
23:31alyssapipcet[m]: horrifying
23:31pipcet[m]i'm confused, where are you trying to get ssh? if you have USB the ethernet-over-usb stuff used to work :-)
23:32roxfan has quit [Ping timeout: 480 seconds]
23:32alyssapipcet[m]: I have only one USB ethernet adaptor with me rn
23:33alyssaand my laptop is a chromebook (debianized) so ..
23:33alyssapipcet[m]: re the watchdog driver, have you sent that upstream? scared of LKML?
23:34pipcet[m]sorry, what I meant was making the USB gadget pretend to be an ethernet adapter
23:35alyssaer wait what? :P
23:36pipcet[m]sorry, I'm not sure I'm thinking of the right situation. If you have the M1 running Linux with Sven's patches you should be able to configure in eem support, and then you can run TCP/IP over the same cable you presumably use to talk to m1n1?
23:37alyssaHuh. this is new to me
23:37alyssaand yes I have a m1n1 cable it's just inert right now after boot
23:38alyssawill give that a try thank you!!
23:38pipcet[m]it might be easier to do it the other way around, if your chromebook has usb c and gadget support...
23:38alyssathey both are type-c
23:40pipcet[m]and you're using a C-C cable? that setup always scares me because I'm afraid they'll start charging each other and all of my computers blow up at the same time ;-)
23:40alyssam1 charges the chromebook
23:42opticronmy friend has a 16" MBP and plugged a C-C cable into two of its ports...it started "charging" and the screen got brighter as if it were plugged in
23:42pipcet[m]it'll go blind
23:45alyssapipcet[m]: so how do I make this gadget support come on
23:45pipcet[m]did you select the preconfigured gadget or the configfs one?
23:46alyssahmmm is configuring it important? ;p
23:48alyssaI have CONFIG_USB_ETH set which is in `USB Gadget precomposed configurations`
23:48alyssaso that sounds promising
23:49alyssaah but i'm in host only mode umm
23:49alyssaerrr why does it only have host-only or gadget-only ....
23:49alyssaIs there really no dual mode? :|
23:50alyssadual role uhh
23:50pipcet[m]in the kernel config? yes, there is, it just depends on usb role switching IIRC
23:51alyssa │ Depends on: <choice> && (USB [=y]=y || USB [=y]=USB_DWC3 [=y]) && (USB_GADGET [=y]=y || USB_GADGET [=y]=USB_DWC3 [=y]) && (EXTCON [=n]=y || EXTCON [=n]=USB_DWC3 [=y]) │
23:51alyssaso need EXTCON
23:52alyssaok. let's try this again. I wonder if it'll recompile the universe.
23:54pipcet[m]hmm. I think I used the non-preconfigured eem driver, and I still ended up using twenty lines of weird hex constants: https://github.com/pipcet/pearl/blob/main/local/hardware/pearl/bin/gadget#L3 through line 23. maybe this wasn't the best idea :-)
…