Installing macOS Mojave 10.14 on Proxmox 5.4

With the release of macOS Catalina 10.15, this tutorial is now obsolete! Please check out the new Catalina tutorial here.

This tutorial for installing macOS Mojave has been adapted for Proxmox 5.4 from Kholia’s GitHub project for installing into vanilla KVM. There is more documentation there which will help out with enabling extra features and diagnosing problems!

Requirements

I’ll assume you already have Proxmox 5.4 installed. You also need a real Mac available in order to fetch the OSK key.

Your Proxmox host computer’s CPU must support SSE 4.2, so for Intel your CPU must be at least as new as Nehalem, which was the first CPU generation to bear the “Core” i5/i7 branding. Older CPUs will cause the finder to repeatedly crash after installation completes (with an Illegal Instruction exception in the graphics code).

Apparently modern AMD CPUs also support SSE 4.2 and can be used with this guide without any modification (maybe Bulldozer and certainly Ryzen), but I haven’t tested this myself.

Continue reading Installing macOS Mojave 10.14 on Proxmox 5.4

Patch OVMF to support macOS in Proxmox 5 and 6

Proxmox 5 and 6’s version of the OVMF firmware includes two commits (2ac1730 and 147fd35) that are intended to mark the pagetables as read-only during startup. This was first seen in Proxmox 5.1. This conflicts with the OsxAptioFixDrv drivers in Clover, which expect to be able to modify the pagetables to remap memory:

https://sourceforge.net/p/cloverefiboot/tickets/439/

I’ve patched OVMF to revert the effect of these two commits, which allows macOS to boot again (I also tested it by booting Windows 10, which worked fine). If you just want to download the fixed .deb, skip to the end of the article, otherwise if you want to build it yourself, follow along with the instructions in the next section:

Continue reading Patch OVMF to support macOS in Proxmox 5 and 6

Installing macOS High Sierra on Proxmox 5

With the final release of Mojave, this tutorial is now out of date, see the new tutorial for installing Mojave instead!

This tutorial for installing macOS Sierra has been adapted for Proxmox 5 from Kholia’s GitHub project for installing into vanilla KVM. There is more documentation there which will help out with enabling extra features and diagnosing problems!

Requirements

I’ll assume you already have Proxmox 5.1 installed. You also need a real Mac available in order to download High Sierra from the App Store and build the installation ISO. Your Proxmox host computer must have an Intel CPU at least as new as Penryn (I believe you would need a a custom Mac kernel in order to use an AMD CPU). Continue reading Installing macOS High Sierra on Proxmox 5

Passthrough of advanced CPU features for macOS [High] Sierra guests

When emulating macOS on Proxmox, it seems that we are forced to set the guest’s CPU type to “Penryn”. This is a very old architecture, and is missing some features that could unlock higher CPU performance. In particular, I wanted to use AVX (for accelerated stream processing) and AES-NI (for encryption), but macOS panics on boot if I set the CPU to Sandy Bridge, which would match my CPU which includes those features.

Luckily, kholia over at the OSX-KVM project has discovered that we can keep using Penryn, but enable the passthrough of individual advanced CPU features and have Sierra use them, even though Penryn never supported these features.

Continue reading Passthrough of advanced CPU features for macOS [High] Sierra guests

Upgrading a Proxmox 5 macOS Sierra guest to High Sierra

macOS 10.13 High Sierra has finally been released, and the good news is that it works with Proxmox 5!

Here’s how I upgraded my Proxmox 5 Sierra installation, which has been previously updated to use Clover/UEFI boot and is stored on a passthrough NVMe SSD. Your setup may differ and your upgrade steps may need to change. I doubt these instructions would work for enoch/chameleon boot.

Take a snapshot of Sierra

I cannot stress this enough! If your filesystem gets completely trashed by the installer, you really need to be able to roll it back to a snapshot!

Continue reading Upgrading a Proxmox 5 macOS Sierra guest to High Sierra

Fixing ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED on macOS

After reinstalling Mac OS Sierra, I found that Chrome could no longer use my HTTPS client certificates. Instead, after choosing my certificate from Chrome’s pop-up certificate picking menu, I just got a fatal “ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED” error. The HTTPS client certificates worked fine in Safari, so it seemed to be a Chrome-specific problem.

I was able to fix this by opening the Keychain Access program, right-clicking my HTTPS private key and selecting Get Info, then on the Access Control tab I changed it from “allow all applications to use this item” to “confirm before allowing access”. The next time I tried to view the website in Chrome, Mac OS popped up to confirm that I wanted to allow it to use the key, and it worked perfectly after clicking Allow! I guess the Keychain’s application permissions got messed up at some point, and this reset it.

Emulating MIPS guests in Proxmox 5

I wanted to emulate MIPS guests on my Proxmox hypervisor so that I could do some security research on router firmware. Unfortunately, Proxmox has customised some of the QEMU packages and their dependencies, which makes it difficult to install the standard Debian qemu-system-mips package. In particular, Proxmox provides its own pve-libspice-server1 package which conflicts with the libspice-server1 package that vanilla QEMU depends on, so attempting to install it will complain:

Some packages could not be installed.

The following packages have unmet dependencies:
 qemu-system-mips : Depends: libspice-server1 (>= 0.12.5)

To solve this, we need to build a modified version of the package from source.

Continue reading Emulating MIPS guests in Proxmox 5

Login bypass in Ubiquiti airMAX/airOS before 8.0.2, 7.2.5, 6.0.2, 5.6.15 if airControl web-UI was used

After seeing this arbitrary command execution vulnerability in Ubiquiti equipment, discovered by SEC Consult, I was intrigued. In that bug, code that would have been secure on a more recent version of PHP was rendered vulnerable because of the ancient PHP version used (2.0.1, which is nearly 20 years old). I wanted to see what other bugs might be caused by PHP that works in unexpected ways.

My friend owns a “NanoBeam AC” running firmware WA_v8.0.1, so I downloaded that firmware from Ubiquiti’s website and unpacked it with binwalk. I found a bunch of PHP scripts, a custom patched PHP 2.0.1 binary, and a custom patched Lighttpd server which handles session management and serves the files.

Continue reading Login bypass in Ubiquiti airMAX/airOS before 8.0.2, 7.2.5, 6.0.2, 5.6.15 if airControl web-UI was used

Passthrough more than 4 PCIe devices to Proxmox 4 and 5 guests

Update: Proxmox 6 now has built-in support for this! If you’re getting “Duplicate ID ‘ich9-pcie-port-5’ for device”, be sure to remove any additions you’ve made to /usr/share/qemu-server/pve-q35-4.0.cfg

By default in Proxmox 4.4 and 5, you are unable to pass through more than 4 PCIe devices to the guest. If you try, you’ll get an error when attempting to start the VM which reads:

vm 100 - unable to parse value of 'hostpci4' - unknown setting 'hostpci4'

Passed-through PCIe devices are attached to the four ports called “ich9-pcie-port-{1,2,3,4}” which are defined in /usr/share/qemu-server/pve-q35.cfg. These ports occupy PCIe function numbers 0-3, leaving function numbers 4-7 unused.

It’s a simple matter to add definitions for an extra 4 ports to use up those spare function numbers in /usr/share/qemu-server/pve-q35.cfg (Proxmox 4 and 5): Continue reading Passthrough more than 4 PCIe devices to Proxmox 4 and 5 guests

Fix for macOS [High] Sierra 10.12.4+ “don’t steal mac OS” error on boot on Proxmox 4

In Sierra 10.12.4, macOS added some extra copy protection which is able to tell that the SMC emulation that QEMU provides is not a real Mac. This causes a fatal error during boot on Proxmox 4 and earlier. Proxmox 5.1 now includes the fix for this problem in its regular QEMU package so a patch for 5.1 is no longer necessary.

One way of fixing this would be to remove the SMC device from the virtual machine’s arguments, and use FakeSMC.kext instead, like a regular Hackintosh, but this is inelegant.

Instead, we can patch QEMU to fix the SMC support, using the fixes from here: Continue reading Fix for macOS [High] Sierra 10.12.4+ “don’t steal mac OS” error on boot on Proxmox 4