#
…

2022-04-11

01:34marcantoo many cores probably
01:34marcanshould slow down the HV tick on at least the secondaries
01:34marcanthat plus the BHL means it doesn't scale
02:45marcanI forget if I take the BHL to do local FIQ handling... I should probably move at least that to finer-grained locking, so the HV ticks don't fight each other. that'll at least make it scale.
02:46kevans91big hairy lock?
02:52PhilippvK has joined #asahi-dev
02:55phiologe has quit [Ping timeout: 480 seconds]
03:02marcanbig hypervisor lock
03:07nicolas17 has quit [Quit: Konversation terminated!]
03:08kode54 has quit [Quit: The Lounge - https://thelounge.chat]
03:08KDDLB has quit [Quit: The Lounge - https://thelounge.chat]
03:28KDDLB has joined #asahi-dev
03:28KDDLB has quit []
03:28KDDLB has joined #asahi-dev
03:35KDDLB has quit [Quit: The Lounge - https://thelounge.chat]
03:37KDDLB has joined #asahi-dev
03:39kode54 has joined #asahi-dev
04:45gladiac is now known as Guest1648
04:45gladiac has joined #asahi-dev
04:51Guest1648 has quit [Ping timeout: 480 seconds]
05:12user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
05:25user982492 has joined #asahi-dev
05:42kevans91hmm, am I reading these pci nodes correctly? https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/apple/t8103.dtsi#n446 -> my really limited understanding is that reg should encode the b:d:f (which I think we (FreeBSD) are doing wrong), but these reg values also don't seem to line up with the iommu-map if I'm reading that right
05:44kevans91i.e. there's no way a masked rid from <0x0 0x0 0x0 0x0 0x0> could possibly fall into [0x100, 0x101), [0x200, 0x201), or [0x300, 0x301)
05:45kevans91so it feels like either i'm reading something wrong, or this is inconsistent and one of those (presumably the iommu-map) are wrong
05:47kevans91otoh, linux presumably gets this right
06:03viridian_ has joined #asahi-dev
06:18JayasPJacob[m] has joined #asahi-dev
06:20JayasPJacob[m]hi i would like to contribute for this awesome project
06:59ketteniskevans91: there are nu PCI bus master capable devices on bus 0
07:12MajorBiscuit has joined #asahi-dev
07:13the_lanetly_052 has joined #asahi-dev
07:16user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
07:27MajorBiscuit has quit [Ping timeout: 480 seconds]
07:29jluthra_ has joined #asahi-dev
07:29viridian_ has quit [Quit: -yes]
07:30jluthra has quit [Ping timeout: 480 seconds]
07:49MajorBiscuit has joined #asahi-dev
07:50zoeyrae has quit [Quit: The Lounge - https://thelounge.chat]
08:01the_lanetly_052 has quit [Remote host closed the connection]
08:02the_lanetly_052 has joined #asahi-dev
08:07Major_Biscuit has joined #asahi-dev
08:09MajorBiscuit has quit [Ping timeout: 480 seconds]
08:20riker77 has quit [Quit: Quitting IRC - gone for good...]
08:33riker77 has joined #asahi-dev
08:58M76cw2gnqjj[m] has joined #asahi-dev
09:15Major_Biscuit has quit []
09:19MajorBiscuit has joined #asahi-dev
09:23kameks has joined #asahi-dev
09:26AnalogDigital[m] has left #asahi-dev [#asahi-dev]
09:42arndI ran into some funny performance issue on the second half of the T6002. I noticed that a BLAS/GEMM benchmark is a lot slower on 10 cores of the M1 Ultra than an M1 Max MBP. With some more investigation I found that the first 10 cores are about the same, but the second set of firestorm cores appears slower. Doing the test individually on each core shows 30 GFLOPS on any icestorm, 90 GFLOPS on the first eight firestorm cores and 60
09:42arndGFLOPS on the other eight
09:45j`eydid jannau fix cpufreq in linux?
09:48j`eyok maybe it doesn't actually need any fixes, after a quick look
09:49arndj`ey: what was the issue? I see the firestorm cores running at 3228000 HZ in /sys/devices/system/cpu/cpu12/cpufreq/cpuinfo_cur_freq during the test
09:49_jannau_looks like cpufreq doesn't work on the second die. m1n1 initializes all cores to 2 GHz, (max for icestorm, 2/3 of max for firestorm)
09:49arndok, that explains it then
09:50arndI guess cpuinfo_cur_freq shows what the kernel has requested, not what the hardware actually does
09:52_jannau_yes, 3.228 GHz is even wrong for the cores of the 1st die. that's only reached when the other cores of the cluster are in deep sleep
10:00arnd$ cat /sys/devices/system/cpu/cpu12/cpufreq/affected_cpus
10:00arnd2 3 4 5 12 13 14 15
10:02arndso it thinks that the cores in the second cluster always run at the same clk as the corresponding ones in the first one. looking at cur_freq confirms that
10:11_jannau_that's wrong, looks like I botched the cpufreq dt
10:15jannaumissed to update the core's "apple,freq-domain = <&cpufreq_hw x>;" for the second die
10:15jannaushould be 3, 4, 5 instead of 0, 1, 2
10:16arndjannau, right, I just tried that locally
10:17arndI had to change the reg-names as well, but now it seems to work
10:18arndjannau: should I send you a proper patch, or do you just want to fold it in?
10:18arndnot sure if marcan ends up redoing it all anyway
10:19_jannau_I'll fold it in, we will probably end up redoing it anyway
10:20arndI got a typo in cpu_p00_d1, that needs to be 4, not 3
10:21arndwith the wrong number, CPU 12 dropped down to 18 GFLOPS...
10:21_jannau_thanks. strange that I still saw a 2x performance improvement in kcbench compared to m1 max
10:23j`eyredo the measurements later :D
10:25arndjannau: what is the maximum frequency under constant load? If the m1 max gets throttled to the same 2GHz after a while, that still works out
10:32_jannau_3 GHz are the expected all core full load frequency for the performance cores. even in the macbook pro it should be sustainable for a longer period of time. we need to look for the actual core frequency
10:32riker77 has quit [Ping timeout: 480 seconds]
10:33robinp_ has quit [Ping timeout: 480 seconds]
10:34arndit scales almost perfectly now: 591 GFLOPS for either set of 8 firestorm cores, 1171 GFLOPS for all 16
10:34robinp has joined #asahi-dev
10:35arndinterestingly it goes down to 886 GFLOPS if I use all 20 cores including the small ones, but I think that's just blis/gemm not being aware of big/little cores, and making the big ones wait
10:47kov has joined #asahi-dev
10:52riker77 has joined #asahi-dev
10:59jannau10% faster, 70s instead of 78s for make vmlinux with linux-5.15 arm64 defconfig. I guess I have to redo it on the m1 max as well
11:00c10l has quit [Quit: Bye o/]
11:01j`eyit was 5mins on my m1 air D: (for 5.18ish defconfig)
11:06c10l has joined #asahi-dev
11:18riker77 has quit [Quit: Quitting IRC - gone for good...]
11:22svenyay, nvmem (not to be confused with NVMe) which was even more boring than watchdog got merged :D
11:26j`eysven: woot
11:30riker77 has joined #asahi-dev
11:39MajorBiscuit has quit [Ping timeout: 480 seconds]
11:46arndjannau: on this many cores, the total runtime is not all that meaningful because half the build time is spent in single-threaded work like linking or parsing the Makefiles, it's often better to compare CPU time (user+system from /bin/time, or output from perf stat) to see how much work the CPUs got done
11:57MajorBiscuit has joined #asahi-dev
12:04mazj`ey: 5 minutes for 'make vmlinux'? that's odd. it takes half that time on my mini (make -j9 vmlinux).
12:05j`eyI cant remember what -j I used now, less than 9 for sure. I also only have the 8GB ram model
12:06maz8GB is more than enough. the compiler used may have a significant impact though (I use GCC 10.2.1)
12:07j`eyI was having OOM issues with LLVM, so felt like being convservative :P will run more tests at some point. I was using llvm/clang
12:07j`ey(OOM issues *building* LLVM)
12:12j`eymaz: any news on your studio?
12:13mazj`ey: landing expected first half of May...
12:16___nick___ has joined #asahi-dev
12:16___nick___ has quit []
12:17atsalyuk has joined #asahi-dev
12:18___nick___ has joined #asahi-dev
12:21mpsj`ey: `real 2m 26.38s` on mbp with `busybox time make -j9 vmlinux`, though not defconfig but current linux-asahi for alpine config
12:23j`eyhmmm real 4m 4.61s
12:24j`eyI cant imagine clang could be that much slower.. retesting with gcc
12:25mpswhat is PAGE_SIZE of running kernel
12:26mpsto complete pkg build (apk for alpine) take about 4m 40s
12:26mpss/to/to me/
12:31j`eyreal 5m 51.31s, with gcc...
12:36arndthe time it takes for building a kernel can differ hugely based on a single Kconfig option, such as CONFIG_DEBUG_INFO, the exact toolchain version, or how the compiler was built
12:39j`eyI'll try mps's config later, since we have the same gcc (from alpine)
12:40arndj`ey: clang is usually some 10% to 20% slower than gcc for a defconfig build, but it's more sensitive to options that lead to larger indirect header inclusions
12:41j`eyclang was 2mins faster for me
12:41j`eyI need to check how alpine's gcc is built
12:42arndhttps://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/arm64/ has toolchains that should work on any distro, as long as you have glibc, using those should make it easier to compare across systems
12:42j`eyyeah, alpine has musl :)
12:42j`eyI was going to ask for your commands to build those, so I could get a build as similar as possible..
12:43arndit might be enough to copy libc.so.6 from any other distro, but I haven't tried
12:44arndthe toolchains are based on segher's 'buildall' scripts, I need to see if I can find a copy
12:46arndI don't know who jmesmon is, but it looks like nathanchance has a newer version at https://github.com/nathanchance/buildall
12:46j`eyah cool
12:49arndin the gcc source tree, run ./contrib/download_prerequisites to make it build a local copy of the mpc/mpfr/isl/gmp libraries instead of the distro version
13:20alexsv has joined #asahi-dev
13:30kameks has quit [Ping timeout: 480 seconds]
13:39yuyichao has quit [Ping timeout: 480 seconds]
13:40j`ey(so that points to alpine being slower, for some reason)
13:43mpsj`ey: hm, on which distro you run this
13:44j`eyalpine
13:45mpsI don't understand, you run on alpine and say it is slower than your previous test?
13:52Axenntio has joined #asahi-dev
13:52Axenntio has quit []
13:53mazj`ey: FWIW, I get 3:20 with LLVM-11 on the same defconfig, v5.18-rc2.
13:55j`eymaz: thanks, maybe a musl vs glibc thing then..
13:55timokrgr has quit [Quit: User left the chat]
13:57timokrgr has joined #asahi-dev
13:58psykoseadd gawk and measure again
13:58j`eypsykose: I think that helps more for x86
13:58yuyichao has joined #asahi-dev
14:08user982492 has joined #asahi-dev
14:16winter has quit [Quit: Ping timeout (120 seconds)]
14:16linuxgemini95 has quit []
14:17nico_32 has quit [Read error: Connection reset by peer]
14:17user982492 has quit [Read error: Connection reset by peer]
14:17nullroute has quit [Quit: bai]
14:17user982492 has joined #asahi-dev
14:17Method has quit [Remote host closed the connection]
14:17nullroute has joined #asahi-dev
14:19Method has joined #asahi-dev
14:19winter has joined #asahi-dev
14:19linuxgemini95 has joined #asahi-dev
14:22nico_32 has joined #asahi-dev
14:45kevans91kettenis: ahh, that makes sense, thanks!
15:01user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
15:49user982492 has joined #asahi-dev
16:03user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
16:21c10l has quit [Quit: Bye o/]
16:29c10l has joined #asahi-dev
16:42rikkaa has joined #asahi-dev
16:45user982492 has joined #asahi-dev
16:48MajorBiscuit has quit [Ping timeout: 480 seconds]
16:58kovj`ey, mps are you both running on the Air? it is much slower than the Mini due to thermal throttling in my testing
16:59kov(thought not sure it would have such a big impact on a quickish build)
16:59j`eykov: mps is mbp, im on the air
17:00kovah, then it makes sense yeah... fewer perf cores + thermal throttling will do it for sure hehe
17:54coversine has joined #asahi-dev
18:04boardwalk has quit [Quit: Ping timeout (120 seconds)]
18:04boardwalk has joined #asahi-dev
18:09MajorBiscuit has joined #asahi-dev
18:09MajorBiscuit has quit []
18:09MajorBiscuit has joined #asahi-dev
18:15c10l has quit [Quit: Bye o/]
18:16c10l has joined #asahi-dev
18:51rikkaa has quit [Quit: Connection closed for inactivity]
18:59possiblemeatball has joined #asahi-dev
19:06possiblemeatball has quit [Quit: Leaving.]
19:06possiblemeatball has joined #asahi-dev
19:14MajorBiscuit has quit [Ping timeout: 480 seconds]
19:33atsalyuk has quit [Ping timeout: 480 seconds]
19:37user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
19:40user982492 has joined #asahi-dev
19:59possiblemeatball has quit [Quit: Leaving.]
20:00possiblemeatball has joined #asahi-dev
20:05bisko has joined #asahi-dev
20:08___nick___ has quit [Ping timeout: 480 seconds]
20:18bisko has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
20:18bisko has joined #asahi-dev
20:47MajorBiscuit has joined #asahi-dev
21:14user982492 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
21:35possiblemeatball has quit [Quit: left]
21:36possiblemeatball has joined #asahi-dev
21:47MajorBiscuit has quit [Ping timeout: 480 seconds]
21:59user982492 has joined #asahi-dev
22:01kode54clang is 100% slower for me building a mostly generic Arch x86_64 kernel fulllto
22:01possiblemeatball has quit [Quit: left]
22:02possiblemeatball has joined #asahi-dev
22:02nathanchancewell, full LTO is the real problem there :)
22:02nathanchancealthough clang is absolutely slower than gcc
22:02nathanchancehopefully soon there will be some work in that area...
22:06possiblemeatball has quit []
22:07possiblemeatball has joined #asahi-dev
22:18j`eywhat happened to the days when clang was fast!
22:19psykosedon't think it ever was faster
22:20j`eythere was a lot of claims of that.. many years ago
22:21mpsmarketing
22:22psykosemaybe it was, then, but i certainly can't say that's the case anymor
22:26nicolas17 has joined #asahi-dev
…