Hotplug support for eGPU on Linux

As noted previously, I have an eGPU set up with my laptop. Today I figured out how to successfully unplug it without the system freezing up.

Initially, I was quite confused as to why I was unable to just unplug the eGPU. It works on Windows after all. It turns out that a Thunderbolt disconnection (at least with my fully updated Arch Linux) does not gracefully remove the PCI devices from the system. Everything that was relying on the eGPU and any other peripherals in the eGPU box, such as the wired ethernet, fail horribly. Threads lock up waiting for a reply to arrive on the PCI bus, but nothing does.

So the fix is really quite simple. Use the pcihp module’s functionality to gracefully remove the entire Thunderbolt chain from the PCI tables before unplugging the cable. This is easiest done by echoing “1” to the “remove” file present in the lowest-numbered thunderbolt PCI-device. (Note: Dependent on your system, this might not be a clever idea if you happen to have more than one Thunderbolt port… It might be worth removing the attached devices instead. Check your lspci output first.)

I wrote up a little script to automate this:

#!/bin/bash

secs=5
tbt_chain=/sys/bus/pci/devices/0000:01:00.0

echo "Unplug eGPU script started."
if [ "$(id -u)" != "0" ]; then
	echo "Please run using sudo. Exiting."
	exit 1
fi
if [ -e $tbt_chain/remove ]
then
	echo 1 > $tbt_chain/remove
	echo "Thunderbolt chain removed from PCI tree. Please unplug eGPU now."
	while [ $secs -gt 0 ]; do
   		echo -ne "$secs to rescan...\033[0K\r"
   		sleep 1
   		: $((secs--))
	done
	echo 1 > /sys/bus/pci/rescan
	echo "Rescanned the PCI bus. Completed."
	exit 0
else
	echo "eGPU does not appear to be attached. Exiting."
	exit 1
fi

Advertisement

Six-way comparison of my eGPU setup

I like thin and light notebooks. But I also like having the graphics horsepower available when at my desk to play the occasional game. In the last few months, external GPUs connected by Thunderbolt 3 are starting to appear, and I was lucky enough to get my hands on a PowerColor Devil Box. And unlucky enough to experience an initial incompatibility between the Devil Box and my laptop, a recent Kaby Lake Dell XPS 13 (9360).

Luckily though, I have received a firmware update for the Devil Box, which allows compatibility with my laptop, so I thought I would finally do some testing of the setup.

System:

Laptop: Dell XPS 13 9360
CPU: i7-7500U
RAM: 8GB
internal GPU: Intel HD graphics 620

eGPU enclosure: PowerColor Devil Box
eGPU: Sapphire AMD Radeon RX470 4GB

Now, I use Arch Linux as my daily driver, and I’m pleased to report that the eGPU is detected, and can be used to render any application by simply setting the environment variable DRI_PRIME=1 (when using Wayland). The only issue I’ve had with Linux is that the system doesn’t really like when unplugging the eGPU while running, so I have to turn the laptop off before unplugging. I might investigate this some other day, but it is no big deal, since the eGPU can be connected while the computer is running with no issues.

Anyway, on to the benchmarks. I am running Unigine Heaven on Extreme settings, and rendering on the laptops internal screen:

Quality:       Ultra
Tesselation:   Extreme
Anti-aliasing: x8
Fullscreen:    no
Resolution:    1600x900

The scores were:

OS          (API)      eGPU   iGPU
Windows 10  (DX11)     1322    211
Windows 10  (OpenGL)   1230    199
Arch Linux  (OpenGL)    816    146

So what can we gather from this? Firstly, Linux OpenGL drivers are not as good as Windows OpenGL drivers. Secondly, DirectX 11 performs better than OpenGL. Finally, an eGPU is a compelling upgrade in my case.

So what’s left? Well, I’ve ordered a replacement fan for the front of the Devil Box. I was so tired of the constant high-RPM spinning, that I have unplugged both fans. But this leads to the PSU fan spinning up when under load, and the little fan on the PSU is very loud (although with quite a pleasant smooth sound). I hope a lower RPM and better fan in the front of the Devil Box will prevent this. Also, I need to investigate ways of improving the Linux experience: fix the problems when unplugging, and work out how to change the fan profile of the card when running in Linux.

GPG and Enigmail

Recently, I decided to set up a setup to send and receive encrypted email. It was remarkably easy using Arch Linux. GPG was already installed, so I installed Crypter to give me a nice GUI to set up my keys. I then installed Enigmail into Thunderbird, went through the setup, and was done within 10 minutes. Most of this was actually waiting for the random number generator to seed the key generation.

Linux life: using the built in encryption of solid state disks

Introduction

This post has been sitting around in my drafts for about a year and a half. I have since abandoned this solution, preferring to encrypt my home partition only. But if someone finds it useful, I’ve published it here.

Background

Most solid state disks nowadays proudly claim that data is encrypted in the controller before being written to disk. But if no ATA password is set on the disk, then that encryption is not really of much use. All it does is prevent someone from disassembling your SSD and reading the data off the flash modules without the controller, and ensuring that a secure erase can be performed quickly. This is not much use for me, as I was trying to guard my data on my laptop against theft, while not compromising on performance.

One vital part of a proper security system is protecting “data at rest”, i.e. preventing data on your laptop from being accessed if it is lost or stolen while turned off. For this, Full Disk Encryption (FDE) is generally the best solution, but if your CPU does not support AES-NI, then implementing FDE on the operating system level can cause throughput to be painfully slow (around 40 MB/s through the encryption pipeline on my AMD E2-1800), and CPU intensive. Not ideal for a laptop, and having spent all that money on an SSD, very annoying to have to compromise between speed and security. Step forth the SSD controller, a small chip on your SSD that manages the interface between SATA and flash modules. Samsung’s newest SSDs use a triple-core ARM controller, so wouldn’t it be cool to offload responsibility for FDE to this controller, since it is already doing it?

Leaving aside issues of whether to trust your SSD vendor’s claims about a proper keychain of trust, I at least wanted to use the provided encryption of my solid state disk. The big problem I faced though was that my BIOS did not support setting an ATA password, nor did it support unlocking an already set ATA password (ATA passwords can be set and unlocked using hdparm on linux. But make sure you can boot a live USB disk to unlock it again if your BIOS turns out not to be able to unlock it, otherwise your system may be unbootable).

Solution

After a bit of thinking, my solution turned out to be:

-Ensuring my computer could boot a USB disk if the SSD was locked (check boot order in BIOS)

-Prepare a USB disk which would boot, run hdparm, ask for the password, unlock the SSD, and hand over to the linux installation on the SSD. Henceforth called the drive-unlock-stick (DUS).

-Set the ATA password on my SSD.

-Verify that the setup worked (it does)

-Test for additional problems caused (one big one: Suspend to RAM does not currently work properly, described below)

Preparing the “drive-unlock-stick” (DUS)

Firstly, consider that you will need this disk every time you boot your laptop. Either, you could use this opportunity to implement a form of two-factor authentication (to boot you need both the DUS and your password), or find a USB disk small enough that you don’t mind leaving it in your computer. I chose the latter, and for the added performance benefits of USB 3.0, bought a compact USB disk off amazon.

The DUS must be bootable, so set up syslinux. This involves making sure that the partition on the DUS has the bootable flag set (check with fdisk for example), copying the syslinux MBR onto the DUS, installing the syslinux bootloader, and writing the configuration file.

Next thing you need is a linux kernel and an initial ramdisk environment in which to run hdparm. I’m using Arch linux, so I scripted a hook that would load the drivers to interface with the disk (ahci in my case, but check using lsmod), load the drivers to eventually read the filesystems of my boot and root partitions on the SSD, ask for the password from the user, unlock the drive, do a bit of magic by unloading the driver again and reloading it, freeze the drive security, to prevent setting another password in the operating system, mount the boot and root partitions of the SSD, and hand over control to the original linux kernel on the SSD through a kexec call, from which point normal booting continues. This requires some work to get right, and my current solution still spews a lot of errors when the ahci module gets confused, and I think it has a 5 second timeout somewhere that slows it down quite a bit, but it works.

Since writing this post back in 2013, I seem to have lost the configuration files. But the above descriptions seem to sum the method up nicely.

Linux life: Syncing iPhone with iTunes on a virtual machine

This post is a bit different from my previous posts, but I thought it was interesting enough to post, and it might help someone in the future.

The issue at hand is to get music onto my iPhone 4 running iOS 6, using only a Linux machine. Unfortunately, the iTunesDB on the phone is encrypted on iPhone 4 and up, and the excellent libgpod  team have not been able to reverse-engineer it yet, so USB syncing of music is not possible natively on Linux. From this there are two obvious workarounds: jailbreak your phone, and use some of the programs available on Cydia to enable adding music to the iTunesDB, or somehow get your iPhone talking with a real iTunes installation.

I chose the second option, but iTunes does not work very well on wine currently, so I chose to install windows on a virtual machine. I had a Windows 7 OEM license bundled with the laptop, so I set up Virtual Machine Manager and Qemu, and installed iTunes on top of this. This works very well with the RedHat virtIO drivers for networking and virtual storage, highly recommended.

To allow the iTunes on my virtual machine to see my music stored on my linux box, I set up a Samba share, restricted to the virtual network. iTunes 11 no longer has the option to add an entire folder to your library from the menu, but you can still drag and drop folders from Windows Explorer into iTunes, which took a while, but worked.

I was able to get USB passtrough to the virtual machine working  for a short while, long enough to get the iPhone recognised in iTunes, but this was painfully slow. I ticked the box for enabling wifi sync in iTunes, and left it for the next day.

The wifi sync feature requires a Bonjour connection between iTunes and the iPhone, but unfortunately Bonjour cannot bridge from the virtual machine over virtIO, and the other virtual network devices I tried did not work well. Finally, i found mdns-repeater a small program that does exactly what I want: it runs on my Linux machine, and forwards Bonjour packets from the virtual network between the host and virtual machine, and the wireless network on which my iPhone and Linux host are both connected. Using this, iTunes and the iPhone can see each other, and I am syncing with around 10 Mbit/s, much faster and more reliably than with the USB passtrough solution.

If anyone finds this useful, feel free to ask questions if you need a step or two fleshed out. I’ll be happy to help. 🙂