AVATT: Difference between revisions

From coreboot
Jump to navigation Jump to search
No edit summary
 
(27 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page is about the "[http://code.google.com/soc/2008/coresystems/appinfo.html?csaid=ED0729A9DB64B39B All Virtual All The Time]" GSoC 2008 project.
This page is about the "[http://code.google.com/soc/2008/coresystems/appinfo.html?csaid=ED0729A9DB64B39B All Virtual All The Time]" GSoC 2008 project.
This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a Linux kernel modified to include [http://kvm.qumranet.com KVM] support, uClibc, a custom busybox image, and whatever is needed to get KVM-qemu up and running in the ROM image.
This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a custom Linux kernel modified to include [http://kvm.qumranet.com KVM] or [http://openvz.org OpenVZ] support, a small userland on top of [http://www.uclibc.org uClibc], consisting mostly from a custom [http://www.busybox.net busybox] image, and whatever is needed to get KVM or OpenVZ up and running. See this [http://panzer.utcluj.ro/~alien/coreboot/AVATT/screenshot.png screenshot] if you want to see it in action.
== Prebuilt ROM image ==


== Needed tools ==
I have prepared a pre-built image that you can use in qemu to see what's this all about without having to spend considerable time to compile it.


In order to build this, you need a modified version of [[buildrom]], that can be cloned from my git repo like this:
These are some copy/paste instructions that will fetch it and run it in qemu(yes, you should already have it installed):
git clone git://repo.or.cz/kvm-coreboot.git buildrom


In order for it to work, you need to have installed the [http://www.coreboot.org/Buildrom#Prerequisites prerequisites] of buildrom.
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/KVM/bios.bin -O /tmp/bios.bin    # This one is quite old, and may not work anymore
or
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/OpenVZ/bios.bin -O /tmp/bios.bin # This one is based on OpenVZ, but it currently lacks the vzctl utility


You need a x86 with gcc-4 Linux environment/distribution, since currently the build fails under x86-64 and when using gcc-3.x. Eventually this will get fixed in the future.
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O /tmp/vgabios-cirrus.bin
qemu-system-x86_64 -L /tmp -hda /dev/null -m 768 -serial stdio


== How to build ==
If your bandwidth allows, you can also get a [http://panzer.utcluj.ro/~alien/coreboot/AVATT/diskimage.img.bz2 disk image] usable with the KVM version. You should extract it and pass it as the -hda parameter of qemu instead the /dev/null file so that you can mount it and try to run VMs of the CD ISO images present inside it. This won't work for now, see the "Current Status" section to find out why. At [http://panzer.utcluj.ro/~alien/coreboot/AVATT this website] you can find other various data concerning this project, freely available for download.
Just type  
 
make
== Building from scratch ==
from within the buildrom directory and everything should get built.
=== Needed tools ===
 
First of all, you need a modern Linux distribution capable of running buildroot and buildrom (anything with gcc > 4.2 should do).
 
In order to build this stuff you need a build system consisting of forked versions of [[buildrom]] and [http://buildroot.uclibc.org/ buildroot] that can be cloned from my git repo using this command:
git clone http://repo.or.cz/r/avatt.git
 
You also need to have installed most of the [http://www.coreboot.org/Buildrom#Prerequisites prerequisites of buildrom], and maybe even others that may be missing from your box. I'll eventually update it or make a list of needed tools for common distros...if people report other missing tools.
 
=== How to build ===
Choose the version you want to use by using either of the "make openvz" or "make kvm" commands. '''Currently KVM is broken, and OpenVZ works but lacks the vzctl utility(whic is currently in progress).'''
Just type ''make'' and everything should get built just fine.


(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:
(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:
  make menuconfig (for buildrom)
  make buildrom-config
make kernel-config
  make buildroot-config
make uclibc-config
  make busybox-config  


== How to run ==
=== How to run the ROM image you just built ===


You should read the [[QEMU Build Tutorial]] and the [[QEMU]] pages to see how to run a BIOS image with qemu, but since this howto aims to be as short as possible, here you have all you need to get it running ASAP:
You should read the [[QEMU Build Tutorial]] and the [[QEMU]] pages to see how to run a BIOS image with qemu.
Currently the OpenVZ target can be tested by issuing "make test". The KVM branch must be tested by manually running qemu in 64bit mode.


It's quite handy to have a symlink to "deploy/emulation-qemu-x86.rom" in your $HOME with the name bios.bin, and in the same directory you should have the Cirrus VGA BIOS ROM image.
== Current status ==
Here you have cut/paste commands for this scenario:
=== The contents of the OpenVZ ROM image===
ln -s deploy/emulation-qemu-x86.rom ~/bios.bin
* Linux kernel version 2.6.24 with the OpenVZ patch applied and a custom minimalist config
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O ~/vgabios-cirrus.bin
* uClibc 0.9.30.1
* busybox 1.13.2


Then you must run qemu in 64bit mode so that we can get support for the SVM instructions:
===What works===
qemu-system-x86_64 -L ~ -hda /dev/null -m 768 -serial stdio
====OpenVZ image====


Notes:
* The kernel compiles fine on x86, not yet tested on x86-64 (feedback is welcome), although it should work just fine due to the fact that we now are using buildroot as a full toolchain, and no CFLAGS/LDFLAGS ugly hacks anymore
* The -m 768 parameter is mandatory, since otherwise the kernel will panic because I hardcoded the RAM size to 768M in coreboot. It was 128M before, and that proved to be too small. Sorry for those who have few MBs of RAM.
* There's no userland utility yet, but it should be straightforward to get it compiled
* The -serial parameter is good for those who have keyboard issues with the default SDL output of qemu, like arrow keys not working, etc. This way you can also use the terminal you used to launch qemu as a second console and here all the keys should work fine.
* It was tested and should compile on the following Linux distributions:
**Ubuntu 8.10 & 9.04 alpha
(any feedback is welcome)


== What's inside the 2MB BIOS ROM image? ==
===What doesn't work===
* Linux kernel version 2.6.22.2 with tiny patches and a custom config
====The KVM image is outdated====
* uClibc 0.9.28.3 with some patches applied (aio, fenv, long double math) and a custom config - only the .so files is included
'''As you probably already know, khe KVM image is outdated and currently unsupported'''
* zlib 1.2.3 - only the .so file included
* ncurses 5.6 - the .so file and some other small necessary files are included
* kvm-72 with some small patches - the extra BIOSes, the nbd tool and the docs are missing
* busybox 1.1.3 with a custom config - NEEDS UPDATE


== Current status ==
Last time I checked it had some of these issues:
* everything compiles fine on x86 but fails on x86-64. I have tested it on Ubuntu 8.04, Debian Lenny and Gentoo current, all on x86. with gcc<4.3.
* kvm crashes due to the poor Thread Local Store(TLS) implementation of uClibc, when ran in KVM-accelerated mode
It used to work before, but I introduced a fix for gcc-4.3.x which broke the older versions. I hope I'll get this fixed soon.
The build failed on these distributions:
* kvm crashes due to Thread Local Store(TLS) issues in uClibc.
* Gentoo hardened - failed with this error:
Internal error convert_magic 00006000 != a5a5a5a5
The convert_magic error indicates that the assembler/linker didn't initialize struct image_params at the right place.
* Debian Etch - fails to strip the kvm binary due to faulty binutils:
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value


=== TODO ===
=== TODO ===
* make the kvm userspace tool not to crash anymore. Possible solutions:
* Complete the OpenVZ image by adding the vzctl and (maybe) vzquota - work in progress - current main focus
** fix the TLS issues from the version of uClibc we currently use
* Port it on real hardware and make some cool demos - after we get full OpenVZ support
** migrate to another version of uClibc which hopefully has a fixed implementation of TLS. This would imply removing the long double math function calls from kvm or maybe even porting the uClibc-nptl branch to x86 if the linuxthreads branch proves to be too broken for us.
* Make the kvm userspace tool not to crash anymore, and keep it active. We need a fix for the TLS issues of uClibc. The uClibc developers are working on it but there's no time frame in which it should be ready.
* user-friendly tool that can create and run virtual machines.
* User-friendly tool that can create and run virtual machines in both OpenVZ and KVM (after it gets usable).
* automatically starting the virtual machines at boot.
* Automatically starting the virtual machines at boot - easy to do with OpenVZ after vzctl is done (they have some initscripts), but it should be done manually for KVM.
* get the network to work in qemu since it fails with both coreboot v2 and v3.
* Get the network to work in qemu since it fails with both coreboot v2 and v3 - this is most likely a coreboot issue, any help is welcome
* integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support to be really useful.
* (maybe) integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support, to really be useful - we're quite far away from this goal
* fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for the some extra wasted ROM space caused by 64bit code.
* <s>Fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for some extra wasted ROM space caused by the 64bit code. We can't cross-compile because we're not using a full toolchain, like buildroot does</s> - non-issue anymore, after we switched to buildroot, but it still needs some testing and feedback
* fix the kernel build with gcc-3.x
* <s>Keep the versions as up-to-date as possible but also compatible with each other</s> - non-issue anymore, after we switched to buildroot, since it can be used to update the software included in the payload
* keep the versions as up-to-date as possible but also compatible with each other


== Contact ==
== Contact ==
If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi.magherusan NO@SPAM net.utcluj.ro and I'll do it myself.
If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi NO@SPAM cs.utcluj.ro and I'll do it myself.


You can also find me on IRC most of the time. My nick is "alien".
You can also find me on IRC most of the time, and I'll eventually answer your questions when I come back.  
My nickname is "alien".




Line 72: Line 92:


Cristi
Cristi
{{PD-self}}

Latest revision as of 11:08, 23 March 2009

This page is about the "All Virtual All The Time" GSoC 2008 project. This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a custom Linux kernel modified to include KVM or OpenVZ support, a small userland on top of uClibc, consisting mostly from a custom busybox image, and whatever is needed to get KVM or OpenVZ up and running. See this screenshot if you want to see it in action.

Prebuilt ROM image

I have prepared a pre-built image that you can use in qemu to see what's this all about without having to spend considerable time to compile it.

These are some copy/paste instructions that will fetch it and run it in qemu(yes, you should already have it installed):

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/KVM/bios.bin -O /tmp/bios.bin    # This one is quite old, and may not work anymore

or

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/OpenVZ/bios.bin -O /tmp/bios.bin # This one is based on OpenVZ, but it currently lacks the vzctl utility
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O /tmp/vgabios-cirrus.bin
qemu-system-x86_64 -L /tmp -hda /dev/null -m 768 -serial stdio

If your bandwidth allows, you can also get a disk image usable with the KVM version. You should extract it and pass it as the -hda parameter of qemu instead the /dev/null file so that you can mount it and try to run VMs of the CD ISO images present inside it. This won't work for now, see the "Current Status" section to find out why. At this website you can find other various data concerning this project, freely available for download.

Building from scratch

Needed tools

First of all, you need a modern Linux distribution capable of running buildroot and buildrom (anything with gcc > 4.2 should do).

In order to build this stuff you need a build system consisting of forked versions of buildrom and buildroot that can be cloned from my git repo using this command:

git clone http://repo.or.cz/r/avatt.git

You also need to have installed most of the prerequisites of buildrom, and maybe even others that may be missing from your box. I'll eventually update it or make a list of needed tools for common distros...if people report other missing tools.

How to build

Choose the version you want to use by using either of the "make openvz" or "make kvm" commands. Currently KVM is broken, and OpenVZ works but lacks the vzctl utility(whic is currently in progress). Just type make and everything should get built just fine.

(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:

make buildrom-config
make buildroot-config

How to run the ROM image you just built

You should read the QEMU Build Tutorial and the QEMU pages to see how to run a BIOS image with qemu. Currently the OpenVZ target can be tested by issuing "make test". The KVM branch must be tested by manually running qemu in 64bit mode.

Current status

The contents of the OpenVZ ROM image

  • Linux kernel version 2.6.24 with the OpenVZ patch applied and a custom minimalist config
  • uClibc 0.9.30.1
  • busybox 1.13.2

What works

OpenVZ image

  • The kernel compiles fine on x86, not yet tested on x86-64 (feedback is welcome), although it should work just fine due to the fact that we now are using buildroot as a full toolchain, and no CFLAGS/LDFLAGS ugly hacks anymore
  • There's no userland utility yet, but it should be straightforward to get it compiled
  • It was tested and should compile on the following Linux distributions:
    • Ubuntu 8.10 & 9.04 alpha

(any feedback is welcome)

What doesn't work

The KVM image is outdated

As you probably already know, khe KVM image is outdated and currently unsupported

Last time I checked it had some of these issues:

* kvm crashes due to the poor Thread Local Store(TLS) implementation of uClibc, when ran in KVM-accelerated mode

The build failed on these distributions:

  • Gentoo hardened - failed with this error:
Internal error convert_magic 00006000 != a5a5a5a5

The convert_magic error indicates that the assembler/linker didn't initialize struct image_params at the right place.

  • Debian Etch - fails to strip the kvm binary due to faulty binutils:
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value

TODO

  • Complete the OpenVZ image by adding the vzctl and (maybe) vzquota - work in progress - current main focus
  • Port it on real hardware and make some cool demos - after we get full OpenVZ support
  • Make the kvm userspace tool not to crash anymore, and keep it active. We need a fix for the TLS issues of uClibc. The uClibc developers are working on it but there's no time frame in which it should be ready.
  • User-friendly tool that can create and run virtual machines in both OpenVZ and KVM (after it gets usable).
  • Automatically starting the virtual machines at boot - easy to do with OpenVZ after vzctl is done (they have some initscripts), but it should be done manually for KVM.
  • Get the network to work in qemu since it fails with both coreboot v2 and v3 - this is most likely a coreboot issue, any help is welcome
  • (maybe) integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support, to really be useful - we're quite far away from this goal
  • Fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for some extra wasted ROM space caused by the 64bit code. We can't cross-compile because we're not using a full toolchain, like buildroot does - non-issue anymore, after we switched to buildroot, but it still needs some testing and feedback
  • Keep the versions as up-to-date as possible but also compatible with each other - non-issue anymore, after we switched to buildroot, since it can be used to update the software included in the payload

Contact

If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi NO@SPAM cs.utcluj.ro and I'll do it myself.

You can also find me on IRC most of the time, and I'll eventually answer your questions when I come back. My nickname is "alien".


Thanks for passing by,

Cristi

Public domain I, the copyright holder of this work, hereby release it into the public domain. This applies worldwide.

In case this is not legally possible:
I grant anyone the right to use this work for any purpose, without any conditions, unless such conditions are required by law.