Build 64-bit kernel for Raspberry Pi, using native tools

How to build a custom 64-bit ARM aarch64 kernel for the Raspberry Pi 3, 3+ and 4 boards, on device, using native cross-compilation tools binutils and gcc.

 

Changed

Note: For a long time the Raspberry Pi foundation didn't provide 64-bit kernel or userland even if they are advertising the Pi 3 and 4 as 64-bit platforms. This has now changed, as they provide both a 64-bit kernel and userland combination (beta 64-bit Raspberry OS), and also a 64-bit kernel combined with a 32-bit userland option, so you don't need to build a 64-bit kernel yourself anymore unless you really want to.

There are 3 ways to get a 64-bit kernel compiled:

  1. Native build on another 64-bit ARM platform
  2. Cross-compilation on another platform
  3. Cross-compilation on the Pi itself

This tutorial will explain how to do option number 3. (And will actually work just fine for option number 2 as they are pretty much the same).

Note: You will need plenty of space when building as the sources and build results for both the tools and kernel takes quite a lot of space. Also a heat sink is highly recommended if building in parallel.

Time: Speed is relative. The Raspberry Pi is fast or slow, depending on what you compare it to. Even so, it will take a while to build everything, for example C-compiler only gcc will take about 85 minutes to build using non-parallel make on a Pi 3.

Install essential build tools and development packages

As we need to build the tools we need, that is aarch64 binutils and gcc, we need to install the required build tools first. Start by first installing required dependencies for compilation of these tools:

sudo apt-get install build-essential libgmp-dev libmpfr-dev libmpc-dev libisl-dev libncurses5-dev bc git-core bison flex

Build the cross compilation tools

To be able to cross-compile we need to build a couple of tools ourselves as they are not provided. These are aarch64 versions of binutils (assembler, linker) and gcc (C compiler). We will install these tools in their own prefix or path,

/opt/aarch64

We will also use out-of-source build directories to keep the sources clean. This is handy if you need to re-configure, re-build or start over without needing to clean the source tree.

Build and install Binutils

We will start with building the low-level tool, binutils. Binutils is basically an assembler and linker.

Download the latest binutils, (Tested with 2.34 at time of writing, feel free to try any later versions when available)

wget https://ftp.gnu.org/gnu/binutils/binutils-2.34.tar.bz2

Untar the binutils archive

tar xf binutils-2.34.tar.bz2

Configure build and install

mkdir binutils-obj
cd binutils-obj
../binutils-2.34/configure --prefix=/opt/aarch64 --target=aarch64-linux-gnu --disable-nls
make -j4
sudo make install

Binutils is now installed, to be able to use it directly, add "/opt/aarch64/bin/" to your path:

export PATH=$PATH:/opt/aarch64/bin/

Build and install GCC

Next up is gcc, the C compiler. We will build a minimal compiler for C only and no userland support. This is enough to get a aarch64 kernel compiled.

Download latest stable gcc (Tested with 8.4.0 at time of writing, feel free to try any later version when available):

wget https://ftp.gnu.org/gnu/gcc/gcc-8.4.0/gcc-8.4.0.tar.xz

Untar the gcc archive

tar xf gcc-8.4.0.tar.xz

Next configure gcc and build gcc. We will configure the build for only a minimal C compiler, that is enough for building a kernel.

mkdir gcc-out
cd gcc-out
../gcc-8.4.0/configure --prefix=/opt/aarch64 --target=aarch64-linux-gnu --with-newlib --without-headers \
 --disable-nls --disable-shared --disable-threads --disable-libssp --disable-decimal-float \
 --disable-libquadmath --disable-libvtv --disable-libgomp --disable-libatomic \
 --enable-languages=c
make all-gcc -j4
sudo make install-gcc

Test that the cross gcc runs ok, run:

/opt/aarch64/bin/aarch64-linux-gnu-gcc -v

It should report something close to this is everything is ok:

Using built-in specs.
COLLECT_GCC=./aarch64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/opt/aarch64/libexec/gcc/aarch64-linux-gnu/8.3.0/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../gcc-8.3.0/configure --prefix=/opt/aarch64 --target=aarch64-linux-gnu --with-newlib 
--without-headers --disable-nls --disable-shared --disable-threads --disable-libssp --disable-decimal-float 
--disable-libquadmath --disable-libvtv --disable-libgomp --disable-libatomic --enable-languages=c --enable-lto
Thread model: single
gcc version 8.3.0 (GCC)

If all is well, then you can continue to building the kernel itself!

Optional: libgcc

The C-only compiler we built above is enough for building a Linux kernel and is enough for this guide. But, if you would like to for example build and use u-boot, then you will need to build libgcc, a static library that contains shared code and various helper functions. Building and installing libgcc is fortunately very easy, just make the libgcc target and install using the following commands after building gcc:

make all-target-libgcc
make install-target-libgcc

Build the Linux kernel

Now you have a toolchain that is able to build 64-bit ARM kernels. So the next step is to download the Raspberry Pi kernel sources, configure them and build a 64-bit kernel and modules.

Download Linux kernel sources

You have a couple of choices depending on what you like to do. If you just wanna get the latest Raspberry Pi kernel release, download a release archive. If you would like to keep up with any changes fast, then clone the repository.

Release archive

The fastest way to get the kernel sources are by downloading a release archive from the Raspberry Pi GitHub repository. Just pick the latest release, 1.20200212-1 at time of writing, and use that.

Extract the archive with

tar xf raspberrypi-kernel_1.20200212-1.tar.gz
From git repository

Get the most up to date stable branch kernel sources (4.19 branch at time of writing) directly from the Raspberry Pi GitHub repository with

git clone --depth=1 -b rpi-4.19.y https://github.com/raspberrypi/linux.git
Keeping sources fresh

The Linux kernel is a moving target and evolves almost daily. To keep updated with latest changes, run the following command in the directory you checked out above:

git pull

Configure Linux kernel for 64-bit Raspberry Pi

We use output directory for kernel build so we can use the same source tree for other configuration (32-bit for example)

mkdir kernel-out
cd linux
For Raspberry Pi 3, use bcmrpi3_defconfig
make O=../kernel-out/ ARCH=arm64 CROSS_COMPILE=/opt/aarch64/bin/aarch64-linux-gnu- bcmrpi3_defconfig
For Raspberry Pi 4, use bcm2711_defconfig
make O=../kernel-out/ ARCH=arm64 CROSS_COMPILE=/opt/aarch64/bin/aarch64-linux-gnu- bcm2711_defconfig

The kernel build is now configured with a default configuration for a 64-bit Raspberry Pi kernel.

In case you need to customize the configuration (Add support for some specific hardware, trim out things you don't need or otherwise mess around with the configuration) run

make O=../kernel-out/ ARCH=arm64 CROSS_COMPILE=/opt/aarch64/bin/aarch64-linux-gnu- menuconfig

and adjust the configuration for your needs.

Build the kernel and modules

Next build the kernel and modules with:

make -j4 O=../kernel-out/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-

If all goes well, as it should, you should now have a 64-bit kernel, modules and device tree built.

Keep up with kernel development

The Linux kernel evolves and so does the Raspberry Pi kernel too. If you used the repositroy clone then to keep up with development changes you can pull changes with git to download changes to your kernel source tree. To do that run the following command in the kernel source locations:

git pull

This will download any changes made to the kernel source.

Update kernel configuration

After pulling in changes I personally like to always run, just in case:

make -j4 O=../kernel-out/ ARCH=arm64 CROSS_COMPILE=/opt/aarch64/bin/aarch64-linux-gnu- oldconfig