This tutorial for installing macOS Big Sur Final using OpenCore has been adapted for Proxmox from Kholia’s OSX-KVM project and Leoyzen’s OpenCore configuration for KVM. You can get the full sourcecode on my GitHub here.
I’ll assume you already have Proxmox 6 installed. You also need a real Mac available in order to fetch the OSK key.
Your Proxmox host computer’s CPU must support SSE 4.2, so for Intel your CPU must be at least as new as Nehalem, which was the first CPU generation to bear the “Core” i5/i7 branding. Older CPUs will cause the finder to repeatedly crash after installation completes (with an Illegal Instruction exception in the graphics code).
Modern AMD CPUs also support SSE 4.2 and will work with this guide.
First step: Create an installation ISO
Download my copy of the OSX-KVM repository using the download button, and unzip it:
First we need to install some build requirements. If you will be building the installer ISO on macOS, open up the Terminal and run this command to install the commandline tools:
If you’re building the ISO on Linux, you instead need to run this command (these are the package names for Ubuntu or similar distributions, they may need adjustment on other distributions):
sudo apt install g++ git qemu-utils libxml2-dev libssl-dev zlib1g-dev cmake libbz2-dev libfuse-dev fuse autoconf unzip
Now in the Terminal, from the root of OSX-KVM, run:
cd scripts/bigsur make BigSur-recovery.img
This will download the Big Sur installer from Apple’s software distribution servers and build a
BigSur-recovery.img file for you. Upload this file to your Proxmox’s ISO store directory (typically
/var/lib/vz/template/iso). Although we’re putting it in the ISO directory so that we can use it with Proxmox’s ISO picker later, this a raw disk image rather than a true ISO.
If you’re building the installer on macOS, you can build a full installer instead of just a recovery, which will mean that macOS won’t have to download Big Sur files during installation, and so won’t require an Internet connection. Simply ask it to build BigSur-full.img instead:
cd scripts/bigsur make BigSur-full.img
This option is not available when building the installer on Linux.
Prepare an OpenCore image
Download the OpenCore.iso.gz file from the newest release in my repository (that says it’s Big Sur compatible), double click it to unpack it, and upload it to Proxmox’s ISO store at
/var/lib/vz/template/iso. Although it has a .iso file extension, this is actually a hard disk image. (You need v10 or newer)
Fetch the OSK authentication key
macOS checks that it is running on real Mac hardware, and refuses to boot on third-party hardware. You can get around this by reading an authentication key out of your real Mac hardware (the OSK key). Save the first block of C code from this page as smc_read.c. In a command prompt, change into the same directory as that file and run:
xcode-select --install # If you don't already have gcc gcc -o smc_read smc_read.c -framework IOKit ./smc_read
It’ll print out the 64 character OSK for you. Make a note of it.
Every Mac uses the same OSK, so don’t be surprised that it doesn’t look like a random string!
Create the VM
From the Proxmox web UI, create a new virtual machine as shown below.
In the Options page for the VM, ensure that “use tablet for pointer” is set to “Yes”.
In the Hardware page for the VM, add a second DVD drive at IDE0, set it to use your BigSur-full.img or BigSur-recovery.img.
Don’t try to start the VM just yet. First, SSH into your Proxmox server so we can make some edits to the configuration files.
Edit /etc/pve/qemu-server/YOUR-VM-ID-HERE.conf (with nano or vim). Add this line, being sure to substitute the OSK you extracted earlier into the right place:
args: -device isa-applesmc,osk="THE-OSK-YOU-EXTRACTED-GOES-HERE" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2
A USB keyboard is added here because macOS doesn’t support QEMU’s default PS/2 keyboard. Ensure the args are all on a single line!
We also need to add a -cpu argument. If your host CPU is Intel, add this to the end of the “args” line:
This will pass through all of the features that your CPU supports. OpenCore’s config will pretend to macOS that the CPU’s model name is Penryn for compatibility.
If your host CPU is AMD, or the above argument doesn’t work for you, use this more-compatible alternative:
This pretends that your CPU is Penryn, which will keep macOS happy even if your host CPU is AMD, and adds a bunch of newer required and optional CPU features on top. Features that your host CPU doesn’t support will be ignored (a warning will be printed to the console during launch with
qm start 1xx), but note that macOS won’t work without SSE4.2 support.
Now find the lines that define the two “ISOs” (ide0 and ide2), and remove the “,media=cdrom” part from them. Add “,cache=unsafe” in its place. This will treat these as hard disks rather than DVD drives.
Save your changes, return to the Options tab, and change the boot order to put IDE2 (the OpenCore image) first. Your final VM configuration file should resemble this:
args: -device isa-applesmc,osk="..." -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -cpu host,kvm=on,vendor=GenuineIntel,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc balloon: 0 bios: ovmf bootdisk: ide2 cores: 4 cpu: Penryn efidisk0: vms:vm-100-disk-1,size=1M ide0: isos:iso/BigSur-full.img,cache=unsafe,size=2094688K ide2: isos:iso/OpenCore-v10.img,cache=unsafe,size=150M machine: q35 memory: 4096 name: bigsur net0: vmxnet3=...,bridge=vmbr0,firewall=1 numa: 1 ostype: other virtio0: vms:vm-100-disk-0,cache=unsafe,discard=on,size=64G scsihw: virtio-scsi-pci smbios1: uuid=... sockets: 1 vga: vmware
On Proxmox, run “echo 1 > /sys/module/kvm/parameters/ignore_msrs” to avoid a bootloop during macOS boot. To make this change persist across Proxmox reboots, run:
echo "options kvm ignore_msrs=Y" >> /etc/modprobe.d/kvm.conf && update-initramfs -k all -u
Install Big Sur
Now start up your VM, it should boot to the OpenCore boot picker:
Press enter to boot the “Install macOS Beta” entry and the installer should appear.
If you built a recovery installer, the icon will instead be an image of a hard disk and be labelled “MacOS Base System”.
Our virtual hard drive needs to be erased/formatted before we can install to it, so select the Disk Utility option. Follow the steps below to format the disk:
Now we’re ready to begin installation!
After the first stage of installation, the VM will reboot 3 or 4 times in quick succession, and each time you must pick the “macOS Installer” entry (the second one here, with the hard disk icon) to continue installation. It will not be selected for you automatically:
If your keyboard isn’t responding on this screen, exit the Console tab in Proxmox and re-enter it. If you get a “prohibited” sign like this appearing, hit the Reset button on the VM to try again:
Now the installation is complete and the macOS Installer entry disappears, so pick the name of your main disk to boot (mine’s called Main).
Answer the initial install questions, and you’ll be logged on! Note that you might want to hold off on logging into your Apple ID until you’ve configured your Mac’s serial number in OpenCore.
Note that it will be really sluggish for a few minutes after the first boot while the system performs housekeeping tasks.
Make the OpenCore install permanent
We’re currently booting using OpenCore from the attached OpenCore ISO. Let’s install that to the hard drive instead. Pop open Terminal and run “diskutil list” to see what drives we have available.
Use “sudo dd if=<source> of=<dest>” to copy the “EFI” partition from the OpenCore CD and overwrite the EFI partition on the hard disk. The OpenCore CD is the small disk (~150MB) that only has an EFI partition on it, and the main hard disk is the one with the large (>30GB) Apple_APFS “Container” partition on it.
In my case these EFI partitions ended up being called disk2s1 and disk0s1 respectively, so I ran “sudo dd if=/dev/disk2s1 of=/dev/disk0s1” (note that if you get these names wrong, you will overwrite the wrong disk and you’ll have to start the installation over again!).
Now shut down the VM, and remove both the OpenCore and the Big Sur installer drives from the Hardware tab. On the Options tab, edit the boot order to place your virtio0 disk as the first disk. Boot up. If everything went well, you should see the OpenCore boot menu, and you can select your “Main” disk to boot Big Sur:
I found that I was unable to wake Big Sur from sleep using my mouse or keyboard. If you encounter the same problem, you can either disable system sleep in Big Sur’s Energy Saver settings to avoid the issue, or you can manually wake the VM up from sleep from Proxmox by running:
qm monitor YOUR-VM-ID-HERE
Editing your OpenCore/EFI settings
The Configuration.pdf that explains the OpenCore config.plist file can be found along with the OpenCore release on my GitHub.
To mount your EFI partition in macOS so you can edit your config.plist, first check the “identifier” of your EFI partition in the terminal:
~$ diskutil list /dev/disk0 (external, physical): #: TYPE NAME SIZE IDENTIFIER 0: GUID_partition_scheme *512.1 GB disk0 1: EFI EFI 209.7 MB disk0s1 2: Apple_APFS Container disk1 511.9 GB disk0s2
Then you can mount it like so:
sudo mkdir /Volumes/EFI
sudo mount -t msdos /dev/disk0s1 /Volumes/EFI
Now you can edit
/Volumes/EFI/OC/config.plist with your favourite text editor to make your changes. (TextEdit is not a great choice because it likes to insert curly quotes into the file and otherwise break things, there are some dedicated plist editors available such as XCode).
If you’re unable to boot macOS, you can edit the config.plist using the “UEFI Shell” option in the OpenCore boot menu instead.
Enter “FS0:” and press enter to open up the first available filesystem, then run “edit EFI\OC\config.plist” (if the file isn’t found, try switching to another filesystem like fs1:). When you’re done editing, press control+Q to exit, “Y” to save, then run “exit” to return to the OpenCore menu. You need to reboot for your changes to take effect.
In config.plist, you can set Misc/Boot/Timeout to a non-zero value to allow the default boot option be chosen automatically after that delay in seconds. I’ve disabled this by default because it causes the installer ISO to re-enter its main menu instead of continuing the second stage of installation.
To boot macOS in Verbose mode to diagnose boot problems, edit config.plist to change the “bootargs” in the NVRam section from “keepsyms=1” to “keepsyms=1 -v”.
Changing screen resolution
To change macOS’ screen resolution, you need to edit the UEFI/Output/Resolution setting in config.plist, the default is 1920×1080@32.
You should be able to change this to any of the modes that the system OVMF menu offers (hit F2 at the start of guest boot and choose “Device Manager/OVMF Platform Configuration” to see which resolutions are available).
Because there is no guest video acceleration available for macOS, video performance is poor.
In Google Chrome in the guest you will need to toggle off the setting to “use hardware acceleration when available” to improve issues with elements not being drawn or flickering (especially video). Safari may be a better choice.
macOS’s built in “Screen Sharing” feature offers dramatically better framerates and latency than Proxmox’s browser-based VNC console, so if you have a real Mac to act as a viewing console, you can enable that in the VM’s “Sharing” settings and connect to the VM using the Screen Sharing app from your Mac instead:
Apparently Screen Sharing is also compatible with VNC clients like RealVNC, so you should be able to connect to it from Linux or Windows consoles using RealVNC.
The real magic bullet for video performance is to pass through a compatible video card using PCIe passthrough (though note that Big Sur, like Catalina, does not support most NVidia cards). This offers near-native performance. You can read more about how I’m using PCIe passthrough on my own installation here.
Since I want to use this as my primary computer, I want to use a USB keyboard and mouse plugged directly into Proxmox, rather than sending my input through the web VNC console.
Proxmox has good documentation for USB passthrough. Basically, run “qm monitor YOUR-VM-ID-HERE”, then “info usbhost” to get a list of the USB devices connected to Proxmox:
qm> info usbhost
Bus 3, Addr 12, Port 6, Speed 480 Mb/s
Class 00: USB device 8564:1000, Mass Storage Device
Bus 3, Addr 11, Port 5.4, Speed 12 Mb/s
Class 00: USB device 04d9:0141, USB Keyboard
Bus 3, Addr 10, Port 5.1.2, Speed 12 Mb/s
Class 00: USB device 046d:c52b, USB Receiver
Bus 3, Addr 9, Port 14.4, Speed 12 Mb/s
Class 00: USB device 046d:c227, G15 GamePanel LCD
Bus 3, Addr 8, Port 14.1, Speed 1.5 Mb/s
Class 00: USB device 046d:c226, G15 Gaming Keyboard
In this case I can add my keyboard and mouse to USB passthrough by quitting qm, then running:
qm set YOUR-VM-ID-HERE -usb1 host=04d9:0141
qm set YOUR-VM-ID-HERE -usb2 host=046d:c52b
This saves the devices to the VM configuration for you. You need to reboot to have the new settings apply. Note that the emulated USB3 device doesn’t work with macOS, so don’t set “usb3=1”.
You can also pass through USB devices by passing through an entire USB controller using Proxmox’s PCIe passthrough feature, which gives much better compatibility.
FileVault now works with OpenCore, so you can encrypt your boot disk by using the option in macOS’ system security preferences. But be certain to keep a copy of your recovery key and keep your backups up to date.
Fixing “guest boots to UEFI shell”
If your guest ends up booting to the UEFI shell instead of showing the OpenCore boot menu, especially if you’ve just updated OpenCore to a new version, you’ll need to edit the guest’s UEFI boot entries to fix this.
At the very start of guest boot, hit F2 to enter guest UEFI settings.
First we’ll remove the old entries. Choose the Boot Maintenance option, then Boot Options -> Delete Boot Option. Use the spacebar to tick any old Clover or OpenCore entries (avoid ticking the EFI Internal Shell option, you want to keep that!). Select “Commit Changes and Exit”.
Now we’ll add the correct entry for OpenCore back in. Select Add Boot Option. Navigate through the device tree to EFI/BOOT/BOOTx64.efi and select it, name this new option “OpenCore” or similar. Again Commit Changes and Exit.
Go to the Change Boot Order and move OpenCore to the top. Commit Changes and Exit.
Now back out to the main menu and choose Reset, and you should successfully boot into OpenCore this time.
Upgrading from macOS Catalina
First make a backup or snapshot of your system! Being able to roll back when the upgrade goes wrong is a real lifesaver.
You’ll need to update OpenCore to my v10 release before the upgrade. You can follow the instructions in the “Editing your OpenCore settings” section above to mount your EFI disk. Then you can replace the OpenCore files in the mounted “EFI” disk with the ones from the OpenCoreEFIFolder.zip file in my newest OpenCore release.
Reboot to make sure that you can still boot Catalina.
If you’re using any PCIe passthrough devices (particularly video cards) you’ll want to disable those and set “vga: vmware” instead, so you can install using Proxmox’s web console from a different machine during the upgrade. This avoids installer problems triggered by flaky video card passthrough, especially host lockups caused by the AMD Reset Bug.
Now you can upgrade to Big Sur using the App Store like you would on a real Mac.
Disabling SIP (System Integrity Protection)
You can disable SIP by selecting the Recovery option from the OpenCore boot menu, then use the top menu to open the Terminal and run “csrutil disable”. Then reboot.
This may be needed to run unsigned kexts or perform other hacks (it looks like I needed this for WhateverGreen to work with my passthrough graphics card).