HowTo: Xen 4.1.3 Windows 8 HVM domU with Intel HD4000 VGA Passthrough on Debian Wheezy

Update 05/07/2013:
Despite the HowTo being close to 1 year it applies perfectly to this day. At the time Wheezy was Debian's Testing distribution and has since moved to Stable. Much of it can be used to setup other HVM domU such as Linux. 

Important notice: I've experienced some problems with xen-hypervisor-4.1-amd64 versions 4.1.4-3+deb7u1_amd64 and 4.1.4-3_amd64 which implement some security advisory patches . As a result to have sucessful passthrough I was forced to go back to the previous working version: 4.1.4-2_amd64 which is available at http://www.xen.org/downloads/XCP/debian/repo/amd64/xen-hypervisor-4.1-amd64_4.1.2-2.1_amd64.deb. Again, do notice that this package is missing some security patches available in more recent versions.
 
In this HowTo I'll present the steps required to install Xen 4.1.3 using the xm toolstack on a Debian Wheezy (kernel 3.2.0-3-amd64) dom0, create a Windows 8 HVM domU config and setup VGA/PCI Passthrough for the integrated GPU, USB 2.0 controller and audio.

This HowTo assumes that the reader is comfortable with Linux and Windows operating systems namely Debian GNU/Linux and Windows 7/8 as such it doesn't cover the operating systems installation.

For easier reference the procedure will broken down in the following steps:
1) Hardware requirements
2) Install Xen on Debian Wheezy
3) Configure networking
4) Configure Xen
5) Prioritise Xen boot
6) Create and install Windows 8 HVM domU
7) Assign devices for PCI Passthrough
8) Install GPLPV drivers
9) Advices and impressions

1) Hardware requirements

For PCI passthrough both motherboard and CPU must support VT-d also know as IOMMU IO virtualization.

The hardware used to write this HowTo setup is composed of:

• Intel Core i7-3770 CPU,
• Intel DQ77MK Motherboard,
• 32GB GEIL DDR3 1600 MHz,
• 200GB Maxtor SATA HDD,
• Samsung SyncMaster 940BW Monitor.

It should be noted that VT-d and VT-x have been enabled in the motherboard and that the i7-3770 integrated GPU is the computer's sole GPU.

In addition to the above setup I've also used another networked computer so I could SSH into the dom0 and perform the steps identified bellow.

2) Install Xen on Debian Wheezy

The Xen hypervisor is provided by the xen-linux-system package:

1. $ su
2. # apt-get update
3. # apt-get install xen-linux-system

3) Configure networking

There are several ways you can provide network access to domU guest domains, the most common being setting up a network bridge which I'll be covering.

To assign a static IP to the dom0 and define a network bridge named eth0, disable NetworkManager (if installed) and edit /etc/network/interfaces to contain a bridge:

1. # /etc/init.d/network-manager stop
2. # update-rc.d network-manager disable
3. # aptitude install bridge-utils
4. # vim /etc/network/interfaces auto lo br0 iface lo inet loopback allow-hotplug eth0 iface eth0 inet manual iface br0 inet static bridge_ports eth0 address 192.168.1.5 broadcast 192.168.1.255 netmask 255.255.255.0 gateway 192.168.1.254
5. # vim /etc/resolv.conf domain my.domain.com nameserver 192.168.1.254

192.168.1.5 is the IP chosen to be assigned to the host, 192.168.1.255 and 255.255.255.0 are the typical broadcast and netmask values for a 192.168.1.x network and in my case the gateway is 192.168.1.254. Replace this values according to your own network settings and desires.

4) Configure Xen

The xend daemon employs xend-config.sxp to determines the parameters that Xen should use.

Personally I choose to disable dom0 ballooning, define the dom0 assignable memory and change the keyboard layout (I've changed mine to pt):

1. # vim /etc/xen/xend-config.sxp (dom0-min-mem 2048) (enable-dom0-ballooning no) (keymap 'pt')

I've restricted the amount of memory assigned to dom0 to 2048 MB. In my case the dom0 is headless and all the hard work is to be done by the non-privileged virtual machines as such I've opted for a comfortable amount of memory to be assigned to the dom0, 2048 MB (2 GB). To this end GRUB needs to pass the appropriate command as the hypervisor boots:

1. # echo 'GRUB_CMDLINE_XEN="dom0_mem=2G,max:2G"' >> /etc/default/grub
2. # update-grub2

5) Prioritise Xen boot

By default Wheezy's GRUB lists and boots regular kernels and afterwards the Xen hypervisor.

Assuming that the computer is to be running Xen all the time it advisable to change this behaviour and increase Xen's GRUB boot priority so that it's the first on the list and boots by default.

The Debian way to do this is to used dpkg-divert like so:

1. # dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
2. # update-grub2

To undo this necessary:
 

1. # dpkg-divert --rename --remove /etc/grub.d/20_linux_xen
2. # update-grub2

6) Create and install Windows 8 HVM domU

The xm toolstack uses configuration files that define the domain meaning that we need to create a configuration file for our guest VM:

1. # vim /etc/xen/win8-x64.cfg kernel = 'hvmloader' builder = 'hvm' vcpus = '4' memory = '4096' disk = ['file:/srv/xen/domains/win8-x64.img,hda,w',
'file:/srv/xen/images/Windows8-ReleasePreview-32bit-English.iso,hdc:cdrom,r'] name = 'win8-x64' vif = [ 'mac=00:16:3E:51:20:4C,bridge=br0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' sdl = '0' vnc = '1' vnclisten = '0.0.0.0' vncpasswd = '' stdvga = '0' usb = '1' usbdevice = 'tablet'

Do note that a MAC address must be assigned to the virtual interface. The 00:16:3e MAC block is reserved for Xen domains, do the last three digits may be randomly filled in (hex values 0-9 and a-f only).

In this HowTo I'm using file based storage which implies using the dd command to create what will be the domU hard drive. To create a 40GB .img file:

1. # dd if=/dev/zero of=/srv/xen/domains/win8-x64.img bs=1M count=40960

If you're using LVM use 'phy:/dev/mapper/win8-x64,hda,w' (change according to your own target logical volume) instead of 'file:/srv/xen/domains/win8-x64.img,hda,w'.

For more on the options that the domain configuration file accepts refer to xmdomain.cfg.

There are 2 options when it comes to actually installing Windows 8 on the virtual machine. One method consists in using VNC to connect to the guest virtual machine and installing the operating system from whatever computer you have with a graphical desktop environment. In alternative, one can use VGA Passthrough for the install process altogether.

Choose one of the methods, though the VNC method is preferable as it eases troubleshooting and it's the one documented bellow. To use the VGA Passthrough method jump to step 7 of the HowTo and issue xm create win8-x64.cfg.

After defining Windows 8 domU configuration file execute it and connect through VNC to install Windows 8:

1. # xm create win8-x64.cfg
2. $ vncviewer 192.168.1.5

If running a GUI on dom0 simply vncviwer 127.0.0.1, however if running from a networked computer replace the localhost with the IP of the said networked computer (192.168.1.5 for example).

Proceed to do a Windows install, shutdown the guest VM and backup the .img for future use. To shutdown the Windows 8 HVM domU either use guest's shutdown button or issue:

1. # xm destroy win8-x64

xm list can be used to find out the domain Id and use it as argument for xm destroy, for example:

1. xm listName         ID Mem VCPUs State   Time(s)Domain-0     0 4096 8     r-----   34476.9win8-x64     4 4096 4     -b----   301.0
2. xm destroy 4

Also comment out the cdrom line so that the virtual machine doesn't boot into the Windows installation cdrom every time it boots. For security reasons it is best to disable VNC.

1. # vim /etc/xen/win8-x64.cfg kernel = 'hvmloader' builder = 'hvm' vcpus = '4' memory = '4096' disk = ['file:/srv/xen/domains/win8-x64.img,hda,w',
#'file:/srv/xen/images/Windows8-ReleasePreview-32bit-English.iso,hdc:cdrom,r'] name = 'win8-x64' vif = [ 'mac=00:16:3E:51:20:4C,bridge=br0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' sdl = '0' vnc = '0' vnclisten = '0.0.0.0' vncpasswd = '' stdvga = '0' usb = '1' usbdevice = 'tablet'

7) Assign devices for PCI Passthrough

A domU can be made aware and directly access and use PCI devices with full privileges. To accomplish that the PCI devices need to be hidden from the dom0 and not be forwarded to any other domUs.

Using the xm toolstack this is achieved loading the pci_stub kernel module, identifying the PCI devices that are to be forwarded, unbinding the device from dom0 and bind it to pci_stub thus allowing it to be assigned in the domU config file.

1. # lspci 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09) 00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09) 00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04) 00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04) 00:16.3 Serial controller: Intel Corporation 7 Series/C210 Series Chipset Family KT Controller (rev 04) 00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connection (rev 04) 00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04) 00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04) 00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4) 00:1c.6 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 7 (rev c4) 00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a4) 00:1f.0 ISA bridge: Intel Corporation Q77 Express Chipset LPC Controller (rev 04) 00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04) 00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04) 02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection 03:03.0 FireWire (IEEE 1394): LSI Corporation FW322/323 [TrueFire] 1394a Controller (rev 70)

I'll be forwarding 00:02.0 VGA compatible controller, 00:1b.0 Audio device and 00:1d.0 USB controller. To know the exact numbering of the devices run lspci -n:

1. # lspci -n 00:00.0 0600: 8086:0150 (rev 09) 00:02.0 0300: 8086:0162 (rev 09) 00:14.0 0c03: 8086:1e31 (rev 04) 00:16.0 0780: 8086:1e3a (rev 04) 00:16.3 0700: 8086:1e3d (rev 04) 00:19.0 0200: 8086:1502 (rev 04) 00:1a.0 0c03: 8086:1e2d (rev 04) 00:1b.0 0403: 8086:1e20 (rev 04) 00:1c.0 0604: 8086:1e10 (rev c4) 00:1c.6 0604: 8086:1e1c (rev c4) 00:1d.0 0c03: 8086:1e26 (rev 04) 00:1e.0 0604: 8086:244e (rev a4) 00:1f.0 0601: 8086:1e47 (rev 04) 00:1f.2 0106: 8086:1e02 (rev 04) 00:1f.3 0c05: 8086:1e22 (rev 04) 02:00.0 0200: 8086:10d3 03:03.0 0c00: 11c1:5811 (rev 70)

For each PCI to be forwarded create a pci-stub Id, unbind it from the dom0 and bind to pci-stub. xm pci-list-assignable-devices is useful in confirming if the device has been added to the pool of devices that can be assigned to a guest domain.

1. # modprobe pci_stub
2. # echo "8086 1e26" > /sys/bus/pci/drivers/pci-stub/new_id
3. # echo "0000:00:1d.0" > /sys/bus/pci/devices/0000\:00\:1d.0/driver/unbind
4. # echo "0000:00:1d.0" > /sys/bus/pci/drivers/pci-stub/bind
5. # xm pci-list-assignable-devices 0000:00:1d.0
6. # echo "8086 0162" > /sys/bus/pci/drivers/pci-stub/new_id
7. # echo "0000:00:02.0" > /sys/bus/pci/devices/0000\:00\:02.0/driver/unbind
8. # echo "0000:00:02.0" > /sys/bus/pci/drivers/pci-stub/bind
9. # xm pci-list-assignable-devices 0000:00:02.0 0000:00:1d.0
10. # echo "8086 1e20" > /sys/bus/pci/drivers/pci-stub/new_id
11. # echo "0000:00:1b.0" > /sys/bus/pci/devices/0000\:00\:1b.0/driver/unbind
12. # echo "0000:00:1b.0" > /sys/bus/pci/drivers/pci-stub/bind
13. # xm pci-list-assignable-devices 0000:00:02.0 0000:00:1b.0 0000:00:1d.0

Do note that the devices won't be available in the dom0, that's why typical VGA Passthrough setups involve 2 or more graphics cards forwarding the more powerful to the domU alongside an USB controller and audio (I'll cover Secondary Display Adapter PCI passthrough in a future post). In this case only the CPU's integrated GPU is present so as soon has 00:02.0 VGA compatible controller is hidden for the dom0 it can't be used by it and thus the only way to be access is via another computer using SSH for example.

Update the domU's configuration file with the devices that are to be used and start the Windows 8 domU by issuing xm create win8-x64.cfg.

1. # vim /etc/xen/win8-x64.cfg kernel = 'hvmloader' builder = 'hvm' vcpus = '4' memory = '4096' disk = [ 'file:/srv/xen/domains/win8-x64.img,hda,w', #'file:/srv/xen/images/Windows8-ReleasePreview-32bit-English.iso,hdc:cdrom,r' ] name = 'win8-x64' vif = [ ',mac=00:16:3E:51:20:4C,bridge=br0,model=e1000' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' boot = 'dc' acpi = '1' apic = '1' viridian = '1' xen_platform_pci='1' sdl = '0' vnc = '0' vnclisten = '0.0.0.0' vncpasswd = '' stdvga = '0' usb = '1' usbdevice = 'tablet' pci = [ '00:1d.0', '00:1b.0' , '00:02.0' ]
2. # xm create win8-x64.cfg

Also consider creating a script to automate loading pci-stub and unbinding/binding the  PCI devices.

8) Install GPLPV drivers

Developed by James Harper, GPLPV drivers allow swapping the QEMU emulated devices for paravirtualized devices. With these new devices I/O speeds are improved as Windows will use the network and block backend drivers present in the dom0.

Signed GPLPV drivers are available at http://wiki.univention.de/index.php?title=Installing-signed-GPLPV-drivers. Windows 8 32-bit can be downloaded at http://apt.univention.de/download/addons/gplpv-drivers/gplpv_Vista2008x32_signed_0.11.0.356.msi while 64-bit drivers are available at http://apt.univention.de/download/addons/gplpv-drivers/gplpv_Vista2008x64_signed_0.11.0.356.msi.
Vista2008 in the drivers' installer filename means that it applies to the recent Windows releases (e.g. Vista/2008/7/8).

9) Advices and impressions

I've written some pieces of advices and impressions over using XEN over at opensource-impressions.blogspot.com with some specific Intel HD4000 VGA Passthrough so make sure to read them.

Wheezy's stock kernel doesn't contain xen-acpi-processor as it was introduced during Linux kernel 3.4.x as such power management is lacking. In this scenario, consider compiling the kernel or in alternative installing Debian Experimental's kernel. The latter option implies editing /etc/apt/sources.list to cover the experimental branch and instructing apt-get to install the kernel package from the experimental distribution.

Xen is at version 4.1.3 in Debian Wheezy's repository however the recent 4.2.0 release is already available in the experimental branch. And with the 4.2.0 comes xl, the new toolstack set to replace xm.

In the coming weeks I'll update this HowTo with the required steps to combine Debian Experimental Xen and Linux kernel packages with the testing distribution.

Update 05/07/2013:
I've moved on from HD4000 VGA passthrough and am currently using PCI passthrough on 2 AMD cards (HD7850 and HD5450) with much success. The HD5450 is assigned to a Fedora 19 domU while the HD7850 is assigned to a Windows 8 domU (Catalyst drivers aren't installed) for my gaming fix. 
Stability of the setup: rock solid. dom0 is typically online for 1 to 2 months without rebooting, the Fedora 19 is hardly rebooted or shutdown while the Windows 8 domU is shutdown at night. Aside from these domU another Windows 8 domU is allways on.

Tip: Assign Xen dom0 memory on Debian Wheezy

Though Xen can manage memory with the ballon driver it is considered good practice to limit/restrict the amount of memory assigned to dom0. Doing so will stop the  Linux kernel from wasting memory.

To do so you'll need to pass the appropriate boot parameter for both the Xen hypervisor and Linux.

As an example, let's assume you want to assign 2GB to your dom0 here's how:

1. $ su
2. # echo 'GRUB_CMDLINE_XEN="dom0_mem=2G,max:2G"' >> /etc/default/grub
3. # update-grub2

The above steps were executed on Debian Wheezy running Xen 4.1.3 but should translate as well to any other distribution running Xen 4.1.2 or latter and GRUB2. Users of GRUB will need to edit grub.conf and pass down the boot parameters in an analogous manner.

Tip: Remove xen-runtime from kernel-xen repository

Lately I've been messing with Xen.

In the process of using the kernel-xen repository I came across the following situation while attempting to uninstall xen-runtime:

1. # yum remove xen-runtime
Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Remove Process

Resolving Dependencies

--> Running transaction check
---> Package xen-runtime.x86_64 0:4.1.2-8.el6 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
==================================================================
  Package Arch Version Repository Size
================================================================== Removing:
  xen-runtime x86_64 4.1.2-8.el6 @kernel-xen 8.0 M

Transaction Summary
==================================================================
Remove 1 Package(s)

Installed size: 8.0 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Error in PREUN scriptlet in rpm package xen-runtime
xen-runtime-4.1.2-8.el6.x86_64 was supposed to be removed but is not!
  Verifying : xen-runtime-4.1.2-8.el6.x86_64 1/1

Failed:
  xen-runtime.x86_64 0:4.1.2-8.el6
Complete!
2. # rpm -e xen-runtime
/var/tmp/rpm-tmp.LcPd6z: line 5: /bin/systemctl: No such file or directoryerror: %preun(xen-runtime-4.1.2-8.el6.x86_64) scriptlet failed, exit status 127

To remove the little bastard the --nopreun needs to be passed down to rpm -e. By doing so the execution of the %preun scriptlet is turned off thus allowing rpm to uninstall the offending package, like so:

1. # rpm -e --nopreun xen-runtime

Done!

Now I can go back to messing with Xen ;)

HowTo: Install VirtualBox on CentOS 6.2 / SL 6.2

Oracle VM VirtualBox is an popular software package that once installed on an existing host operating system allowing for additional guest operating systems to be loaded and run, each with its own virtual environment.

In this post I'll go over the steps needed to install it on RHEL clones such as CentOS and Scientific Linux.

In summary, we start by downloading VirtualBox's YUM repository file and placing it in the appropriate directory, afterwards we install the dkms and VirtualBox-4.1 packages and conclude by adding your user to be vboxusers group.

Let's begin by fetching the repo file and placing it under /etc/yum.repos.d/:

1. $ wget http://download.virtualbox.org/virtualbox/rpm/rhel/virtualbox.repo
2. $ su
3. # mv virtualbox.repo /etc/yum.repos.d/

Afterwards, installing dkms will bring in cloog-ppl, cpp, gcc, glibc-devel, glibc-headers, kernel-devel, kernel-headers, libgomp, make, mpfr and ppl. These are needed to have VirtualBox match the kernel.

1. # yum install dkms

Then we move to installing VirtualBox itself.

For 32 bits systems use:

1. # yum install VirtualBox-4.1.i686

In the case of 64 bits systems:

1. # yum install VirtualBox-4.1.x86_64

Wrap up by adding your user to the vboxusers and rebooting the computer:

1. # usermod -a -G vboxusers ricardo
2. # reboot

Just in case, make sure to check and see if your computer BIOS has virtualization support disabled. More often that not, motherboard have it disabled.

HowTo: Install and run NetBSD on Virtualbox

NetBSD attracts me.

Maybe it's the philosophy being it... or maybe it just the old BSD daemon holding the flag on top of computer. Regardless of the reason, I've been toying with the idea of installing NetBSD on my Athon XP 3200+ nForce2 old desktop for a while now.

However the only instructions I found to convert a NetBSD 5.1.2 CD image into an USB bootable install medium involved using NetBSD. This chicken-egg dilema lead me to VirtualBox 4.1.14.

Just a few seconds after booting the install image into the VirtualBox machine it kernel panicked with the following error:

 uvm_fault(0xc0b82a00, 0, 1) -> 0xe 
 fatal page fault in supervisor mode 
 trap type 6 code 0 eip c0100d27 cs 8 eflags 3046 cr2 0 ilevel 1 
 kernel: supervisor trap page fault, code=0 
 Stopped in pid 0.19 (system) at netbsd:Xspllower+0x17: addb %al,0(%eax) 
 db{0} 

Just the luck heh?

The workaround? VBoxSDL.

VBoxSDL is a simple graphical user interface (GUI) that lacks the nice point-and-click support which VirtualBox, our main GUI, provides. VBoxSDL is currently primarily used internally for debugging VirtualBox and therefore not officially supported.

To be able to install NetBSD 5.1.2 on VirtualBox 4.1.14, assuming the virtual machine is name "NetBSD", you'll have to use VBoxSDL like this:

1. $ VBoxSDL --norawr0 --startvm NetBSD

Now you'll be able to install NetBSD without any issues.

The problem is when you, in your sweet innocence, reboot the system and VirtualBox isn't able to start the vm stating that:

 Failed to open a session for the virtual machine NetBSD. 
 The virtual machine 'NetBSD' has terminated unexpectedly during startup because of signal 11. 
 Details 
 Result Code: NS_ERROR_FAILURE (0x80004005) 
 Component: Machine 
 Interface: IMachine {5eaa9319-62fc-4b0a-843c-0cb1940f8a91} 

The workaround? Yes you've guessed... VBoxSDL:

1. $ VBoxSDL --nopatm --startvm NetBSD

Peace at last. I can finally muck around NetBSD :D

For reference:

 --[no]rawr0 Enable or disable raw ring 3 
 --[no]patm Enable or disable PATM 

Sources:
http://download.virtualbox.org/virtualbox/UserManual.pdf
http://chvnx.com/post/18341637412/netbsd-in-virtualbox
VBoxSDL --help

Tip: Using Linux to check if CPU has Intel VT-x

To verify if a Intel CPU supports the company's technology for virtualization take a look at /proc/cpuinfo and check for the vmx CPU flag.

 $ grep vmx /proc/cpuinfo 

If your interested in reading the ouput of /proc/cpuinfo in more detail:

 $ cat /proc/cpuinfo | less 

The procfs filesystem contains loads of process and system information and /proc/cpuinfo in particular contains information regarding the CPU, such as vendor, cache size, clockspeed, cores and CPU flags.

Tip: Dealing with virtualbox-ose PUEL license on Gentoo

I came across a peculiar message from portage when trying to install virtualbox-ose in Gentoo:

1. $ su
2. # eix-sync
3. # emerge --tree --ask --verbose virtualbox-ose

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!

!!! All ebuilds that could satisfy "~app-emulation/virtualbox-ose-additions-3.1.4" have been masked.
!!! One of the following masked packages is required to complete your request:
- app-emulation/virtualbox-ose-additions-3.1.4 (masked by: PUEL license(s))
A copy of the 'PUEL' license is located at '/usr/portage/licenses/PUEL'.


(dependency required by "app-emulation/virtualbox-ose-3.1.4" [ebuild])
(dependency required by "virtualbox-ose" [argument])

For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

The solution is to add the PUEL license to the /etc/portage/package.license file:

1. # echo "app-emulation/virtualbox-ose-additions PUEL" >> /etc/portage/package.license
2. # echo "app-emulation/virtualbox-ose PUEL" >> /etc/portage/package.license

Now can finally get to emerge VirtualBox.

As a side note you can also use /etc/make.conf and make use of the ACCEPT_LICENSE variable. For instances adding ACCEPT_LICENSE="*" to /etc/make.conf unmasks all licenses. Alternatively you can unmask licenses one by one.

More information can be obtained reading make.conf and portage's man pages.