affinity: Expose coregroup id/die id to Linux OS
Power10 and above systems have 2 dies in a chip. This has already been
exposed in PowerVM LPARs. Expose P10 hemispheres as a die in PowerNV
systems too.
Previously primary domain index was at depth 4. Each higher depth
indicates smaller set. Since coregroup is smaller than primary domain
index, from Power10 onwards, primary domain index is moved to 3 and
coregroup would be at 4.
Now device tree property ibm,associativity-reference-points is now in
sync with PowerVM.
$ lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 480
On-line CPU(s) list: 0-479
Thread(s) per core: 8
Core(s) per socket: 15
Socket(s): 4
NUMA node(s): 4
Model: 2.0 (pvr 0080 0200)
Model name: POWER10, altivec supported
CPU max MHz: 3249.0000
CPU min MHz: 3249.0000
L1d cache: 32K
L1i cache: 48K
L2 cache: 1024K
L3 cache: 4096K
NUMA node0 CPU(s): 0-119
NUMA node1 CPU(s): 120-239
NUMA node2 CPU(s): 240-359
NUMA node3 CPU(s): 360-479
with-out patched firmware and/or Linux-kernel
---------------------------------------------
$ grep -h -r . /sys/devices/system/cpu/*/topology/die_id |sort | uniq -c | sort -n -r
120 27
120 18
120 9
121 0
$ grep -h -r . /sys/devices/system/cpu/*/topology/die_cpus_list |sort | uniq -c | sort -n -r
120 360-479
120 240-359
120 120-239
120 0-119
with patched firmware and Linux-kernel
--------------------------------------
grep -h -r . /sys/devices/system/cpu/*/topology/die_id |sort | uniq -c | sort -n -r
64 6
64 4
64 2
64 0
56 7
56 5
56 3
56 1
grep -h -r . /sys/devices/system/cpu/*/topology/die_cpus_list |sort | uniq -c | sort -n -r
64 360-375,392-407,424-439,456-471
64 240-255,272-287,296-311,328-343
64 120-135,152-167,184-199,208-223
64 0-15,32-47,64-79,88-103
56 376-391,408-423,440-455,472-479
56 256-271,288-295,312-327,344-359
56 16-31,48-63,80-87,104-119
56 136-151,168-183,200-207,224-239
Observation:
Without the patched kernel and/or skiboot, die-id and numa were same.
With patched kernel and/or skiboot, we see 2 die-id per node.
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
Firmware for OpenPower systems.
Source: https://github.com/open-power/skiboot
Mailing list: skiboot@lists.ozlabs.org
Info/subscribe: https://lists.ozlabs.org/listinfo/skiboot
Archives: https://lists.ozlabs.org/pipermail/skiboot/
Patchwork: http://patchwork.ozlabs.org/project/skiboot/list/
Documentation: http://open-power.github.io/skiboot/doc/index.html
OPAL firmware (OpenPower Abstraction Layer) comes in several parts.
A simplified flow of what happens when the power button is pressed is:
Here, the OPAL image is three parts:
They may be all part of one payload or three separate images (depending on platform).
The bootloader will kexec a host kernel (probably linux). The host OS can make OPAL calls. The OPAL API is documented in doc/opal-api/ (there are missing parts, patches are welcome!)
See doc/overview.rst for a more in depth overview of skiboot.
Any host OS can build and test skiboot provided it has a C cross compiler for big endian powerpc64. All good Linux distributions (and several bad ones) provide a packaged compiler that can be installed through the usual package management tools.
To build on Ubuntu:
apt-get install gcc-powerpc64le-linux-gnu gcc valgrind \ expect libssl-dev device-tree-compiler make \ xz-utils libmbedtls-dev CROSS=powerpc64le-linux-gnu- make -j`nproc`
To build on Fedora:
dnf install gcc-powerpc64le-linux-gnu binutils-powerpc64-linux-gnu gcc make \
diffutils findutils expect valgrind-devel dtc openssl-devel xz \
mbedtls-devel
CROSS=powerpc64le-linux-gnu- make -j`nproc`
(The little-endian powerpc64le compilers in Ubuntu and Fedora are actually bi-endian and can compile skiboot even though it's big-endian. We recommend installing a little-endian toolchain if you plan on building other projects.)
On any POWER system with a bi-endian system compiler:
CROSS="" make -j`nproc`
Alternatively, pre-built cross compilers for x86 systems can be downloaded from here: https://www.kernel.org/pub/tools/crosstool/ When using these compilers add /opt/cross/gcc-4.8.0-nolibc/powerpc64-linux/bin/ to your PATH. Once this is done skiboot can be compiler by just running make
We use Sphinx to produce various documentation from reStructuredText (preferred) and Markdown. The Sphinx documentation has a useful primer for reStructuredText here. And the docutils website has a nice quick reference for the basic constructes.
Building on Fedora
dnf install python3-sphinx python3-recommonmark.noarch pip install -r doc/requirements.txt make -C doc/ html SPHINXBUILD=sphinx-build-3
On Ubuntu:
Patches welcome!
View the output using doc/_build/html/index.html
Skiboot comes with a set of unit tests that can be run on your desktop. They can can be run with:
make check
To test in a simulator, install the IBM POWER8 Functional Simulator from: http://www-304.ibm.com/support/customercare/sas/f/pwrfs/home.html Also see external/mambo/README.md
Qemu as of version 2.8 implements the ‘powernv’ machine model and is sufficient to run skiboot:
qemu-system-ppc64 -M powernv -m 3G -nographic -L /path/to/skiboot/
To run a boot-to-bootloader test you need a Linux kernel image ‘zImage.epapr’. Build one using the opal_defconfig config for op-build. See https://github.com/open-power/op-build/ on how to build, or download one from https://github.com/open-power/op-build/releases/ .
Drop zImage.epapr in the skiboot directory and the skiboot test suite will automatically pick it up. You can also run a combined skiboot and Linux test in Qemu (version 3.0+):
qemu-system-ppc64 -M powernv -m 3G -nographic -kernel zImage.epapr -L /path/to/skiboot/
See opal-ci/README for further testing instructions.
To test on real hardware, you will need to understand how to flash new skiboot onto your system. This will vary from platform to platform.
You may want to start with external/boot-tests/boot_test.sh as it can (provided the correct usernames/passwords) automatically flash a new skiboot onto ASTBMC based OpenPower machines.
All patches should be sent to the mailing list with linux-kernel style ‘Signed-Off-By’. The following git commands are your friends:
git commit -s git format-patch
You probably want to read the linux https://kernel.org/doc/html/latest/process/submitting-patches.html as much of it applies to skiboot.
The Skiboot build process produces a bunch of different outputs. This is what they are, and where you should use them:
skiboot.elf: The output of the linker. Don't flash to a system, but useful when debugging
skiboot.lid: The raw binary object, named .lid because IBM. Flash this on really old P8 systems, the POWER Functional Simulator (mambo), or FSP systems
skiboot.lid.stb: Lid wrapped with secure boot header. Use on FSP systems
skiboot.lid.xz: Compressed raw binary. Use this on a OpenPower P8
skiboot.lid.xz.stb: Compressed raw binary wrapped with a secure boot header. Use this on OpenPower P9 systems
See LICENSE