The RV64ILP32 ABI (referred to as “New 32-bit”) holds significant value in the field of high-performance embedded applications. Building upon existing applications (Reference: Kubuds Tech Facilitates the First Application Landing of RV64ILP32 ABI), Kubuds Technology continues to explore the possibilities of the “New 32-bit” ecosystem within Hypervisors. On the SiFive P550 hardware platform, the open-source EasyXMen has successfully run on KVM in VS-mode, and the performance overhead introduced by virtualization has been measured. The test results demonstrate the feasibility of running open-source EasyXMen with virtualization, indicating that the performance impact of virtualization is within an acceptable range.

About Embedded Virtualization

Open-Source EasyXMen Virtualization Solution

In the RISC-V architecture, applications can run on hardware in three ways: based on the ABI (bare-metal programs), based on an operating system (requiring SBI specification implementation), and based on a Hypervisor virtual machine (requiring a high-performance CPU). This open-source EasyXMen virtualization solution is implemented using the third approach.

About the HiFive P550 Hardware Platform

The P550 is a high-performance RISC-V development board equipped with an ESWIN EIC7700X SoC and a SiFive quad-core P550 64-bit out-of-order CPU. The hardware supports KVM.

Exploration Strategy

First, Dhrystone was compiled natively for P550 Linux, and bare-metal performance data was obtained by running with random core binding. Then, the same Dhrystone code was embedded into a looping Task within the open-source EasyXMen using the “New 32-bit” toolchain. Finally, multiple KVM virtual machines were run on the P550 with core binding. Each virtual machine loaded an instance of the open-source EasyXMen, and Dhrystone performance data was collected under different scenarios. Charts were generated to compare performance differences before and after virtualization.

Verify that the OS kernel supports the H extension and has KVM enabled

Ensure qemu-system-riscv64 supports KVM

Porting Dhrystone to Open-Source EasyXMen

Prepare the source code: git clone https://github.com/iwannatto/dhrystone.git. Several functions need to be implemented manually: strcmp, strcpy, malloc, and functions for calculating elapsed time.

The code can be debugged locally.

qemu-system-riscv64ilp32 -s -S -cpu rv64 -M virt -m 1G -nographic -bios ~/bins/riscv/qemu-linux/fw_dynamic.bin_m64lp64 -kernel riscv_helloworld.bin

riscv64-unknown-elf-gdb riscv_helloworld.elf

Generate the Open-Source EasyXMen Image Using the RISC-V New 32-bit Toolchain

The kernel entry address in the open-source EasyXMen needs to be modified.

Collect Bare-Metal Dhrystone Scores on P550

cd Examples/riscv_helloworld/bench/
make
#text
begin=0
while \[ $begin -le 12 \]
do

cid=\`expr $begin % 4\`
  echo "####################"
  echo begin=$begin,cid=$cid
  echo "####################"
  time taskset -c $cid ./dry2nr 20000000
  ((begin++))
done

Record Performance Data of Dhrystone Running in One and Multiple KVM Virtual Machines

Multi-core Binding Scenario (4 Guests)

taskset -c 0 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 1 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 2 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin

Multi-core Binding Scenario (8 Guests)

taskset -c 0 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 0 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 1 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 1 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 2 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 2 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin

Single-core Binding Test (1 Guest)

taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin

Single-core Binding Test (4 Guests)

taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin
taskset -c 3 sudo qemu-system-riscv64 --nographic --enable-kvm -M virtilp32 -cpu host -m 1024M -smp 2 -kernel riscv_helloworld.bin

Performance Charts

Using taskset to bind to 1 CPU core, running 1-4 EasyXMen Guest OS instances within KVM yields the following Dhrystone performance data:

Using taskset to bind to 4 CPU cores, running 1-8 EasyXMen Guest OS instances within KVM yields the following Dhrystone performance data:

Note: The yellow dashed line represents the performance data of running Dhrystone directly on P550 Linux. The blue bars represent the performance data after virtualization. Both datasets use the same Dhrystone source code and the same compilation parameters.

Future Plans

Kubuds Technology will continue to explore memory isolation, security, and stability among multiple open-source EasyXMen instances within KVM.