Fuzzing has achieved tremendous success in discovering bugs and vulnerabilities in various software systems. However, fuzzers require an environment in which they can execute code with full flexibility and be restricted in a controlled manner. When analyzing processor vulnerabilities and bugs, a hypervisor is needed that fulfills these characteristics. To meet this need, this project implements a minimal bare-metal hypervisor for ARM processors that enables the safe and controlled execution of fuzzing code on the processor.
A bare-metal hypervisor kernel for the Raspberry Pi 4B was implemented for this purpose. It takes fuzzing input via UART, executes the code on one of the sub-cores, and saves the state of the processor before and after the execution of the given input. The saved state is then sent back to the client as fuzzing feedback via UART and can be used to generate new fuzzing inputs.
This repository contains the code for the hypervisor for the Raspberry Pi 4B written in C, a LibAFL-based fuzzer written in rust to generate input for the processor, and a debug client written in python. The debug client allows users to send custom processes to debug the hypervisor and receive feedback to evaluate the hypervisor's correctness.
-
fuzzer: Contains the LibAFL-based fuzzer, which is used to generate fuzzing input and evaluate fuzzing coverage. Additionally, this folder includes a Raspberry Pi Pico W-based autorebooter, which restarts the Raspberry Pi 4B at the command of the fuzzer if the fuzzing input crashes the hypervisor. For further information on setting up the rebooter, please refer to the end of the this file.
-
raspberry_pi: Contains the source code of the hypervisor, as well as all the files necessary to build the firmware and the boot image.
-
debug: Contains the debug client and provides more detailed instructions on how to use it.
The following section explains how the hypervisor can be set up.
To send and receive UART messages, a uart2usb converter must be connected to the RP4. In my setup, I use the Raspberry Pi Debug Probe as a uart2usb converter. However, any other similar device can be used analogously. The cables of the Debug Probe are connected as follows:
_________________
Pin 2: 5V | ___________________________
Pin 4: 5V | | |
Pin 6: GND |--------| GND (Black) |
RP4 PIN 8: TXD |--------| RX (Yellow) Debug Probe |====> USB
PIN 10: RXD |--------| TX (Orange) |
PIN 12: PCM | |__________________________|
... |
_________________|
First, the kernel must be compiled with an ARM compiler:
- Download the GNU compiler toolchain: AArch64 bare-metal target (aarch64-none-elf)
- Copy the archive to the raspberry_pi folder and extract its contents and, if necessary, change the
GCCPATHto the bin folder of the compiler chain in the Makefile. - Run
makein baremetal_hypervisor$ cd raspberry_pi/baremetal_hypervisor && make
Next, the kernel with the necessary files for booting must be written to the SD card. For this purpose, a finished ISO image is created, which can later be written to the SD card. The steps follow the great tutorial from Hardik Srivastava
- To create the boot partition, we need some files from the Raspberry Pi OS (64-bit). The image can be downloaded here and the following files can be copied from the boofs of the image:
├── bcm2711-rpi-4-b.dtb
├── bootcode.bin
├── overlays
│ ├── miniuart-bt.dtbo
│ └── overlay_map.dtb
└── start4.elf
- Copy the files with the structure shown below into the bootfs folder.
bootfs/
├── bcm2711-rpi-4-b.dtb
├── bootcode.bin
├── cmdline.txt
├── config.txt
├── overlays
│ ├── miniuart-bt.dtbo
│ └── overlay_map.dtb
└── start4.elf
- Run the generate-img.sh script.
$ ./raspberry_pi/generate-img.sh - Write the generated iso image on the SD card.
The hypervisor can also store the saved state of the processor on the SD card. For this we have to create a rootfs partition.
- Create a second partition as W95 FAT32 (0x0B)
- Create a folder named
subprocess_logswithin the new file system on the new partition. The logs will later be saved in this folder.
The SD card is now ready and can be plugged into the Raspberry Pi.
Before the LibAFL Fuzzer can be started, Rust and LibAFL must first be downloaded. Instructions for installation can be found here.
After the installation was successful, the Cargo.toml file still needs to have the path to libafl and libafl_bolts set.
libafl = { path = "your_LibAFL_path/LibAFL/libafl" }
libafl_bolts = { path = "your_LibAFL_path/LibAFL/libafl_bolts" }
Additionally, the path to the serial port must be set. The path for the connection to the Raspberry Pi and Rebooter can be set in hypervisor_client/mod.rs.
static HYPERVISOR_PATH: &str = "/dev/serial/by-id/usb-Raspberry_Pi_Debug_Probe__CMSIS-DAP__E6614103E7741C25-if01";
static REBOOTER_PATH: &str = "/dev/serial/by-id/usb-Raspberry_Pi_Pico_E66164084322852B-if00";
Then the LibAFL Fuzzer is ready and can be started. If no rebooter is used, then first start the LibAFL Fuzzer, and then start the Raspberry Pi.
Usage: cargo run [observer_type] [feedback_type] [seed] [input_length]
Options:
observer_type Type of observer to use (ConstObserver, BitObserver, ValueObserver)
feedback_type Type of feedback (Map, Value)
seed Seed value for randomization (positive integer)
input_length Length of fuzzing input data (positive integer)
Note:
- If no arguments are provided, default values will be used.
- If fewer than 4 arguments are provided, default values will be used.
$ cargo run ValueObserver Value 0 100
[HOST] main: Start Fuzzer: Observer type: ValueObserver, feedback type: RegisterValueFeedbackMapPacked, seed: 0, fuzzing input size: 1000!
The idea of the project was to evaluate how well the fuzzer can modify certain registers in the processor using random bytes as instruction input, across various observers. For this purpose, three different observers were used:
- ConstObserver: The constant observer receives only one constant bit of feedback consistently, thus providing no feedback regarding the generated input.
- BitObserver The Bit Observer receives information about whether a register has been modified or not. However, it disregards how many bits in the register were altered by the input.
- ValueObserver: In contrast, the Value Observer receives complete bits of the registers as feedback. This provides additional information about which bits within the register were altered.
The following table provides an overview of which registers were used for the ValueFeedback:
| Register | ValueObserver | BitObserver | ConstObserver |
|---|---|---|---|
| X0 | 64 Bit | 1 Bit | 0 Bit |
| X1 | 64 Bit | 1 Bit | 0 Bit |
| X2 | 64 Bit | 1 Bit | 0 Bit |
| ... | ... | ... | ... |
| X28 | 64 Bit | 1 Bit | 0 Bit |
| N Flag | 1 Bit | 1 Bit | 0 Bit |
| Z Flag | 1 Bit | 1 Bit | 0 Bit |
| C Flag | 1 Bit | 1 Bit | 0 Bit |
| V Flag | 1 Bit | 1 Bit | 0 Bit |
| D Flag | 1 Bit | 1 Bit | 0 Bit |
| A Flag | 1 Bit | 1 Bit | 0 Bit |
| I Flag | 1 Bit | 1 Bit | 0 Bit |
| F Flag | 1 Bit | 1 Bit | 0 Bit |
| Exc. Thrown | 1 Bit | 1 Bit | 0 Bit |
| - | 1865 Bit | 38 Bit | 0 Bit |
Since the goal of the fuzzer is to change as many bits as possible in all registers, the Value Observer was used for evaluating the coverage results across all observers. The Coverage Evaluator runs independently from LibAFL within the fuzzer, enabling the evaluation of results across all bits in all registers with different observers.
The coverage result is output to the console every 100 fuzzing inputs and also written to a CSV file to allow comparison between different observers later on. The seed is provided at the start of the fuzzer to ensure the initial state remains identical across different experiments.
$ cargo run ValueObserver Value 700 100
[HOST] main: Start Fuzzer: Observer type: ValueObserver, feedback type: RegisterValueFeedbackMapPacked, seed: 700, fuzzing input size: 100!
[HOST] Feedback: input_count: 100, total_covarage: 40 (2.14%), avarage_coverage: 6.44 (0.35%)
[HOST] Feedback: input_count: 200, total_covarage: 45 (2.41%), avarage_coverage: 9.10 (0.49%)
[HOST] Feedback: input_count: 300, total_covarage: 68 (3.65%), avarage_coverage: 13.12 (0.70%)
[HOST] Feedback: input_count: 400, total_covarage: 128 (6.86%), avarage_coverage: 15.75 (0.84%)
[HOST] Feedback: input_count: 500, total_covarage: 129 (6.92%), avarage_coverage: 15.79 (0.85%)
[HOST] Feedback: input_count: 600, total_covarage: 147 (7.88%), avarage_coverage: 15.81 (0.85%)
[HOST] Feedback: input_count: 700, total_covarage: 170 (9.12%), avarage_coverage: 16.63 (0.89%)
[HOST] Feedback: input_count: 800, total_covarage: 170 (9.12%), avarage_coverage: 18.05 (0.97%)
[HOST] Feedback: input_count: 900, total_covarage: 229 (12.28%), avarage_coverage: 18.88 (1.01%)
[HOST] Feedback: input_count: 1000, total_covarage: 237 (12.71%), avarage_coverage: 18.40 (0.99%)
[HOST] Feedback: input_count: 1100, total_covarage: 250 (13.40%), avarage_coverage: 18.09 (0.97%)
[HOST] Feedback: input_count: 1200, total_covarage: 264 (14.16%), avarage_coverage: 17.92 (0.96%)
[HOST] Feedback: input_count: 1300, total_covarage: 269 (14.42%), avarage_coverage: 18.03 (0.97%)
[HOST] Feedback: input_count: 1400, total_covarage: 270 (14.48%), avarage_coverage: 19.10 (1.02%)
[HOST] Feedback: input_count: 1500, total_covarage: 292 (15.66%), avarage_coverage: 18.85 (1.01%)
[HOST] Feedback: input_count: 1600, total_covarage: 293 (15.71%), avarage_coverage: 18.63 (1.00%)
[HOST] Feedback: input_count: 1700, total_covarage: 340 (18.23%), avarage_coverage: 18.43 (0.99%)
[HOST] Feedback: input_count: 1800, total_covarage: 358 (19.20%), avarage_coverage: 18.35 (0.98%)
[HOST] Feedback: input_count: 1900, total_covarage: 358 (19.20%), avarage_coverage: 18.12 (0.97%)
[HOST] Feedback: input_count: 2000, total_covarage: 370 (19.84%), avarage_coverage: 17.97 (0.96%)
Since no memory protection has yet been implemented in the hypervisor, kernel memory can be overwritten by a fuzzing process, leading to the hypervisor crashing. To address this, an auto-power cycle design was also implemented.
If the hypervisor does not respond after a few seconds, an Raspberry Pi Pico W will automatically power cycle it. The Raspberry Pi 4B can be restarted by connecting the GLOBAL_EN and GND pins together. The Raspberry Pi Pico W is connected to a transistor, which can connect these two pins at the command of the fuzzer, thereby restarting the Raspberry Pi 4B.
A BC548C transistor was used along with a 22 kilo ohm resistor. However, other components can also be used. The following figure shows the circuit:
- Install CMake (at least version 3.13), and GCC cross compiler
$ sudo apt install cmake gcc-arm-none-eabi libnewlib-arm-none-eabi libstdc++-arm-none-eabi-newlib - Set up the PicoSDK
-
Download the Raspberry Pi Pico SDK repository
$ git clone https://github.com/raspberrypi/pico-sdk.git -
Copy pico_sdk_import.cmake from the SDK into the fuzzer/pico_rebooter folder.
-
Set
PICO_SDK_PATHto the SDK location in your environment, or pass it (-DPICO_SDK_PATH=) to cmake later.
-
Build the Firmware
$ cd fuzzer/pico_rebooter $ mkdir build $ cd build $ cmake -DPICO_BOARD=pico_w .. $ make -
Flash the firmware on the Raspberry Pi Pico W:
- The firmware file
main.uf2is located in thebuildfolder. To flash it onto the Raspberry Pi Pico W, simply drag and drop the file. Press and hold theBOOTSELbutton while connecting the USB cable to initiate the flashing process.