Background

Recently, there are plenty of introductory articles about the WireGuard VPN on Phoronix as the first stable Linux kernel with WireGuard (5.6) is released. At first I thought it was nothing but yet another VPN protocol like conventional PPTP or L2TP. However, after digging into it, I found that it is different by design. It is quite simple. WireGuard tools only create the VPN by adding a network interface and assign the IP and the gateway. There is neither NAT nor DHCP. So, I decided to give it a try.

Using WireGuard on Android

A first taste

The official App already includes a Go backend, so enabling WireGuard on Android is as simple as installing an application and deploying an configuration file. However, there are some limitations with the default Go backend. It utilizes Android's VPN framework to tunnel traffic. As a result, there is always a VPN icon in the notification area, and I cannot connect to another VPN while keeping the WireGuard VPN connected. For phones with root access, a solution is using the Linux kernel module backend, so that limitations imposed by Android frameworks are gone.

Building the kernel module

zx2c4's article on XDA includes descriptions for how to build the WireGuard kernel module for Android devices. That post mentioned the android_kernel_wireguard repo, which is apparently an AOSP project as there is a top-level Android.mk file. As per patch_kernel.sh in that AOSP repo, I need to inject wireguard-linux-compat into the kernel source tree for my device as an in-kernel module. After the injection, I can build the whole kernel to get the WireGuard kernel module. To simplify things, I decided to try out building a vanilla kernel image first.

Building a vanilla kernel

Fortunately, HTC obeys GPL obligations and makes source code tarballs for kernels used on devices available on htcdev.com. First, let's download and extract kernel sources provided by HTC.

$ curl 'http://dl4.htc.com/RomCode/Source_and_Binaries/ocndtwl-4.4.153-perf-g0041d80.tar.gz' -H 'Referer: https://www.htcdev.com/devcenter/downloads' -O
$ tar xf ocndtwl-4.4.153-perf-g0041d80.tar.gz

Among extracted files, there is a Readme.txt with build instructions. Basically, I should be able to build the whole kernel by following those instructions. However, there are two tricks. First, the kernel is built with GCC, while Android NDK has phased out GCC. As a result, the latest revision of https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9 contains only binutils but not GCC. To work around the issue, I used an older branch for the compiler repo. Second, the kernel provided by HTC is too old, so that host scripts cannot be compiled with GCC 10 or newer as the latter forbids common symbols from different source files. Thanks to the hint given by Nathan Chancellor, I can apply a simple patch to make things build. Here are all steps:

$ mkdir kernel
$ tar xf ocndtwl-4.4.153-perf-g0041d80.tar.gz
$ git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9 -b ndk-r13-release --depth 1
$ cd kernel
$ mkdir -p out
$ curl "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/patch/?id=ce513359d8507123e63f34b56e67ad558074be22" > gcc10.patch
$ patch -Np1 -i gcc10.patch
$ make ARCH=arm64 CROSS_COMPILE="$PWD"/../aarch64-linux-android-4.9/bin/aarch64-linux-android- O=out htcperf_defconfig
$ make ARCH=arm64 CROSS_COMPILE="$PWD"/../aarch64-linux-android-4.9/bin/aarch64-linux-android- O=out -j3

Build the WireGuard module with the vanilla kernel

After building the kernel, I started to integrate steps for building my kernel module into the scripts for building WireGuard kernel modules. By some googling, I found https://github.com/WireGuard/android-wireguard-module-builder, which contains scripts for building hosted kernel modules on the download server used by the WireGuard Android application. In that repository, build-one.bash uses repo to build the kernel module using do.bash and manifest.xml in kernels/<codename>. Here manifest.xml is an AOSP platform manifest just like those mentioned on official build instructions for Android kernels. In existing scripts, do.bash calls build.sh in the AOSP kernel/build project. However, as HTC provides only tarballs instead of a complete AOSP tree, I decided to put all commands, include injecting the wireguard-linux-compat module into the kernel source tree and building the whole kernel, directly into do.bash. I still need a git repository as AOSP manifest.xml does not seem to accept tarballs. By some googling, I found the maintainer behind the TWRP port for my device publishes a git repo with mostly the same files as downloaded from htcdev.com. After some trials, I managed to create manifest.xml that specifies an older branch of the compiler repo and use a non-Google git repository. The overall results are submitted as a pull request and it got merged. Now I can reproduce the kernel module with a simple step:

$ ./build-one.sh ocn

A few months later, I was hit by the GCC 10 issue mentioned above, so I created another pull request to get it fixed.

Deploy the built kernel module

The kernel module built by scripts mentioned above has a hash string in its name. That hash can be computed by running sha256sum /proc/version|cut -d ' ' -f 1 on the device as hinted on the module-builder repo. Moreover, if the kernel module exists in the download path used by the Android application, it will load that module as if the latter is downloaded from the official download server. Above all, I can place the locally-built kernel module to where the application expected, and click in the Android application to use the module:

$ adb push out/wireguard-80ee34126cd97c9a15bc3b970a6f38ce30852d0b8547dbcc43eee22956aa1934.ko /data/local/tmp/
$ adb shell
$ su
$ cd /data/data/com.wireguard.android/cache
$ mkdir -p kmod
$ cp /data/local/tmp/wireguard-80ee34126cd97c9a15bc3b970a6f38ce30852d0b8547dbcc43eee22956aa1934.ko kmod/

After my pull request is merged, the kernel module for my device appeared on the official download server. To verify it, I cleared the application cache and then click the "Download and install kernel module" again without pushing the locally-built module beforehand. Confirmed working!

Random feelings

It is not my first time to build a Android kernel, but it is my first time to interact with professional Android kernel developers for such a bleeding-edge and exciting feature. I learned a lot, and I am looking forward to contributing more to AOSP!