Virtualization framework, linux environment

Im learning about Virtualization framework from Apple, and Im currently trying to create an Arch Linux environment to run on top of it. Im still a bit confused about few concepts regarding how this works, so I gathered them into 3 sections.

  1. Kernel and Initial Ram Disk (initrd) usage.

From what I learned to start Virtual Machine, I need to have extracted linuz and initrdfiles, and use them to create a bootloader. This is kind of different approach from other VMsolutions, but I think I get why it is. But to make sure few questions: Since Im loading the kernel and initramdisk at this step, I can extract exact files from my linux distribution disk image, and remove them from linux /boot folder right? And going forward since this is a bootloader, I don’t really need to install Grub to run it, right?

  1. Kernel and inited capabilities and format.

Until now I was only able to run a single kernel and initrd - some Ubuntu cloud 20.04images downloaded from the internet. I tried also other versions of Ubuntu, and my own Arch build, but it usually ends with either some error or sometimes virtualMachine.start()returns success but after that nothing is happening. I think I need either to compile some required features into the kernel (but have no idea which) or have these files with some specific format. I noticed that when using file command on initrd I get different information about these files. For example, the one from Ubuntu that works for me is: LZ4 compressed data (v0.1-v0.9). The other one I tried, but didn’t work was Zstandard compressed data (v0.8+), Dictionary ID: None. Does that play a role here? If so how to prepare initrd in correct format?

  1. Virtual disk format.

After booting into the environment I would like to perform Arch Linux installation on separate virtual disk, and later switch to use this disk as the main one. How should I prepare .img empty file to use? From Disk utility Im only able to prepare .dmg format. Can I use some other formats as well? And later how should I partition this disk? I was planning to create GTP partition table with just a single Linux ext4 partition. On other Virtual Machine I was also creating a 1MB partition for Grub, but I think this can be skipped here. Is that correct?

  1. Do I need initrd?

I think that I will need to compile my own kernel to achieve nicely working environment. And this might be the good solution for me. If that's the case do I really need initrd at all? Would it be possible to create fully prepared kernel that boots without it? If so what features do I need to use in kernel config?

Accepted Reply

Many good questions :)

Before going into the kernel and the Ramdisk: there is an other choice in macOS Ventura. Virtualization framework adds support for EFI with the VZEFIBootLoader. This can be an easier option to run a full distribution like Arch Linux.

Some documentation to get started:


When to use which boot loader?

  • VZEFIBootLoader provides a full EFI and is compatible with any modern boot loaders like Grub.
  • VZLinuxBootLoader is specifically written to boot Linux kernel directly on the virtual hardware, without any firmware like EFI. It's the fastest way to go from zero to the Linux kernel executing code. The downside is it more difficult to configure.

For Arch, unless you are building a system that boots in a few milliseconds, VZEFIBootLoader is likely the easier option.


Since Im loading the kernel and initramdisk at this step, I can extract exact files from my linux distribution disk image, and remove them from linux /boot folder right?

That's right. They won't be used from the disk image if you boot directly with them. No harm in keep them either if you need them later :)

And going forward since this is a bootloader, I don’t really need to install Grub to run it, right?

If you use EFI, you will need Grub or something else compatible with EFI. With the VZLinuxBootLoader, Grub won't run.

I tried also other versions of Ubuntu, and my own Arch build, but it usually ends with either some error or sometimes virtualMachine.start()returns success but after that nothing is happening. I think I need either to compile some required features into the kernel (but have no idea which) or have these files with some specific format.

When starting from scratch like that, the best way to get a starting point is to look at the kernel serial output. You can start by making sure you kernel is compiled with PCI support and Virtio Console within the kernel (not as a module). Then with the right boot options, the serial can go on the console and you can see any error from Linux.

An other way of attacking the problem is to start from a configured kernel and strip it down to your needs.

How should I prepare .img empty file to use?

A RAW disk image is just a flat file that represent the whole disk. The blocks are 512 bytes each. A block at offset X is just the content of X * 512 in the file.

So to create an empty RAW disk image, you can just use truncate with the desired size (multiple of 512).

Can I use some other formats as well?

Only RAW disk images on APFS are supported.

And later how should I partition this disk? I was planning to create GTP partition table with just a single Linux ext4 partition.

That's up to you. The disk appears like a normal block device for Linux. You are free to set up any layout.

If that's the case do I really need initrd at all? Would it be possible to create fully prepared kernel that boots without it? If so what features do I need to use in kernel config?

You are right, it is possible to boot without any Ramdisk. You would need to compile the drivers for all the devices you use inside the kernel rather than as modules. The drivers you need depends on your configuration. You will need PCI support in any case as the Virtio devices are on the PCI bus.

  • Thank you for this detailed response, it helps a lot. I'll start from compiling new kernel, with mentioned required functionalities built in, and experiment a bit with EFI support. I think I would like to stick to VZLinuxBootLoader, as Im planning to prepare a very specific distribution build, just for my needs, so this seems like the best way to go. But anyway, good to know there is an EFI option as well.

Add a Comment

Replies

Many good questions :)

Before going into the kernel and the Ramdisk: there is an other choice in macOS Ventura. Virtualization framework adds support for EFI with the VZEFIBootLoader. This can be an easier option to run a full distribution like Arch Linux.

Some documentation to get started:


When to use which boot loader?

  • VZEFIBootLoader provides a full EFI and is compatible with any modern boot loaders like Grub.
  • VZLinuxBootLoader is specifically written to boot Linux kernel directly on the virtual hardware, without any firmware like EFI. It's the fastest way to go from zero to the Linux kernel executing code. The downside is it more difficult to configure.

For Arch, unless you are building a system that boots in a few milliseconds, VZEFIBootLoader is likely the easier option.


Since Im loading the kernel and initramdisk at this step, I can extract exact files from my linux distribution disk image, and remove them from linux /boot folder right?

That's right. They won't be used from the disk image if you boot directly with them. No harm in keep them either if you need them later :)

And going forward since this is a bootloader, I don’t really need to install Grub to run it, right?

If you use EFI, you will need Grub or something else compatible with EFI. With the VZLinuxBootLoader, Grub won't run.

I tried also other versions of Ubuntu, and my own Arch build, but it usually ends with either some error or sometimes virtualMachine.start()returns success but after that nothing is happening. I think I need either to compile some required features into the kernel (but have no idea which) or have these files with some specific format.

When starting from scratch like that, the best way to get a starting point is to look at the kernel serial output. You can start by making sure you kernel is compiled with PCI support and Virtio Console within the kernel (not as a module). Then with the right boot options, the serial can go on the console and you can see any error from Linux.

An other way of attacking the problem is to start from a configured kernel and strip it down to your needs.

How should I prepare .img empty file to use?

A RAW disk image is just a flat file that represent the whole disk. The blocks are 512 bytes each. A block at offset X is just the content of X * 512 in the file.

So to create an empty RAW disk image, you can just use truncate with the desired size (multiple of 512).

Can I use some other formats as well?

Only RAW disk images on APFS are supported.

And later how should I partition this disk? I was planning to create GTP partition table with just a single Linux ext4 partition.

That's up to you. The disk appears like a normal block device for Linux. You are free to set up any layout.

If that's the case do I really need initrd at all? Would it be possible to create fully prepared kernel that boots without it? If so what features do I need to use in kernel config?

You are right, it is possible to boot without any Ramdisk. You would need to compile the drivers for all the devices you use inside the kernel rather than as modules. The drivers you need depends on your configuration. You will need PCI support in any case as the Virtio devices are on the PCI bus.

  • Thank you for this detailed response, it helps a lot. I'll start from compiling new kernel, with mentioned required functionalities built in, and experiment a bit with EFI support. I think I would like to stick to VZLinuxBootLoader, as Im planning to prepare a very specific distribution build, just for my needs, so this seems like the best way to go. But anyway, good to know there is an EFI option as well.

Add a Comment