Hello,
I'm building a virtualization app on Hypervisor.framework (Apple silicon). Guest RAM is anonymous memory mapped with
hv_vm_map. Activity Monitor and task_info(TASK_VM_INFO).phys_footprint show the VM process using much more memory than
the guest is assigned. For example, a 6 GB Windows ARM64 guest shows about 8.6 GB.
Here's what I've found:
Pages written only from the host count once.
Pages written only by the guest count once.
A page written by both the host and the guest counts twice, even though it is the same physical page. The host reads
back the guest's writes, so there's no second copy.
In a real VM this happens all the time. The VM process writes into guest RAM for firmware loading, disk reads, virtio
buffers and so on, and the guest writes the same pages. So the reported footprint keeps growing past the guest's RAM
size.
Minimal reproducer (needs the com.apple.security.hypervisor entitlement):
// clang -O1 -o hvfootprint hvfootprint.c -framework Hypervisor
// codesign -s - --entitlements hv.entitlements -f hvfootprint
#include <Hypervisor/Hypervisor.h>
#include <mach/mach.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#define MB (1024ull * 1024)
#define GPA 0x40000000ull
#define SIZE (2048 * MB)
#define PAGE 16384
static uint64_t phys_footprint_mb(void) {
task_vm_info_data_t info;
mach_msg_type_number_t count = TASK_VM_INFO_COUNT;
task_info(mach_task_self(), TASK_VM_INFO, (task_info_t)&info, &count);
return info.phys_footprint / MB;
}
// Guest (EL1, MMU off): x0 = start GPA, x1 = value, x2 = page count.
// Stores x1 to each 16 KB page, then exits with HVC.
static const uint32_t guest_code[] = {
0xf9000001, // str x1, [x0]
0x91401000, // add x0, x0, #0x4000
0xf1000442, // subs x2, x2, #1
0x54ffffa1, // b.ne 0b
0xd4000002, // hvc #0
};
static void guest_write_all(hv_vcpu_t vcpu, uint64_t value) {
hv_vcpu_set_reg(vcpu, HV_REG_PC, GPA);
hv_vcpu_set_reg(vcpu, HV_REG_CPSR, 0x3c5);
hv_vcpu_set_reg(vcpu, HV_REG_X0, GPA + PAGE);
hv_vcpu_set_reg(vcpu, HV_REG_X1, value);
hv_vcpu_set_reg(vcpu, HV_REG_X2, SIZE / PAGE - 1);
hv_vcpu_run(vcpu);
}
int main(void) {
uint8_t *ram = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0);
hv_vm_create(NULL);
hv_vm_map(ram, GPA, SIZE, HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC);
memcpy(ram, guest_code, sizeof(guest_code));
hv_vcpu_t vcpu; hv_vcpu_exit_t *exit_info;
hv_vcpu_create(&vcpu, &exit_info, NULL);
printf("start: phys_footprint %5llu MB\n", phys_footprint_mb());
for (uint64_t off = PAGE; off < SIZE; off += PAGE) // host writes every page
ram[off] = 1;
printf("host wrote 2 GB: phys_footprint %5llu MB\n", phys_footprint_mb());
guest_write_all(vcpu, 0x55); // guest writes the same pages
printf("guest wrote the same 2 GB: phys_footprint %5llu MB\n", phys_footprint_mb());
printf("host sees guest data (same pages): %s\n", ram[PAGE] == 0x55 ? "yes" : "no");
guest_write_all(vcpu, 0x66); // and again
printf("guest wrote the same 2 GB again: phys_footprint %5llu MB\n", phys_footprint_mb());
return 0;
}
Output (macOS 27.0.1, build 26A434, Apple M5):
start: phys_footprint 2 MB
host wrote 2 GB: phys_footprint 2050 MB
guest wrote the same 2 GB: phys_footprint 4099 MB
host sees guest data (same pages): yes
guest wrote the same 2 GB again: phys_footprint 4099 MB
2 GB of shared guest RAM is reported as 4 GB.
Other observations
The result is the same whether guest RAM comes from mmap(MAP_PRIVATE | MAP_ANON), mmap(MAP_SHARED | MAP_ANON),
mach_vm_allocate, or a named memory entry (mach_make_memory_entry_64 with MAP_MEM_NAMED_CREATE) mapped twice, with
one view passed to hv_vm_map and the other used by the host.
footprint region totals ("Footprint:" / Dirty) count only pages the host touched. When only the guest wrote 2
GB, the regions showed about 3 MB while phys_footprint showed 2,051 MB. So neither number shows real usage once both
sides touch memory.
System-wide anonymous memory (host_statistics64, internal_page_count + compressor_page_count) does not grow when the
guest writes pages the host already wrote. That suggests this is accounting only, not a second physical copy.
Parallels describes the same symptom for its VMs (https://kb.parallels.com/en/128437).
Questions
Is this double accounting expected for memory mapped with hv_vm_map, or is it a bug? If it's a bug, I'll file it in
Feedback Assistant with this reproducer.
Is there a supported way to allocate or map guest RAM so each physical page is charged to the process once? For
example, a ledger tag, memory-entry ownership, or an hv_vm_map / hv_vm_allocate option.
Is phys_footprint the right input for memory-pressure decisions (jetsam) for a VM process? Could the double count
make such a process look larger than it is to the system?
What is the recommended way to report a VM process's real memory use to users?
Thank you.