ESF Testbeds: Bootstrapping

NOTE: The Bootstrapping Testbed (UCSD) has Xen 3. After some experience, it is clear that current Xen 3 releases are not stable enough for our immediate purpose. We must go for the more stable Xen release 2.0.7. Please note this change if you are setting up a testbed and thinking of Xen 3. Xen 2 process is expected to be on TWiki early next week. - ASR [Sep 30 2005]

Introduction

• Xen 3 Architecture Layout.
• GT4 Workspace Service Architecture.
• ESF-core Architecture.

Xen 3 on Fedora Core 4

This document serves as a quick-and-easy guide to getting started. Attempt is being made to make the content self-contained, but it may remain in a state of flux for some time. Urgent comments or questions can be addressed to - rana@fnal.gov

Have as much memory (RAM) as you can. We have 1 GB, minimum suggested is 256 MB. Our CPU is a 2.4GHz P4.

Install Fedora Core 4 (FC4). The ISO's are available at http://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/iso/. Note that FC5 release is due in February 2006.

Configure the system and make sure networking, disk partitions, filesystem, and general system level services work.

Proceed to start Xen 3 install (as root). Please note the latest stable release of Xen is v2.0.7. Development is taking place in separate testing and unstable trees.
Current Xen 3 releases are from the unstable tree. Thus a few packages are prone to being broken.(Sep 2005)

GRUB bootloader is required.

Disable the security-enhanced mode.
Edit /etc/selinux/config or use GUI by running system-config-securitylevel. Configure SELINUX=disabled

Install the Xen 3 package. (This will get an RPM similar to xen-2-20050823.i386.rpm)
yum install xen

Install the kernel packages for the Host system and the Guest systems. (This will get 2 RPMs similar to kernel-xen0-2.6.12-1.1447_FC4.i686.rpm and kernel-xenU-2.6.12-1.1447_FC4.i686.rpm)
yum install kernel-xen0 kernel-xenU

Change the default kernel in bootloader to the Xen Host (xen0) kernel.
Edit /boot/grub/grub.conf and configure default=0

Reboot the system.
shutdown -r now

Check the kernel in use.
uname -r

Check if all the required packages have been fetched and installed.
rpm -qa | grep xen
rpm -qa | grep iproute
rpm -qa | grep curl
rpm -qa | grep zlib
rpm -qa | grep bridge-utils
rpm -qa | grep python-twisted
rpm -qa | grep sysfsutils

A few directories are not created by default (bug). We will create these.
mkdir -p /var/run/xenstored /var/lib/xenstored

The interface is not set properly to xen-br0 from eth0, if a specific order is not followed. We will restart xend and network to rectify this.
route -n
/etc/init.d/xend stop
/etc/init.d/network restart
/etc/init.d/xend start
route -n

Configure the startup information and run levels for xend.
chkconfig --level 345 xend on

List the running domains.
xm list

Create an image file (1 GB) to be used as storage space by Guest system.
dd if=/dev/zero of=/root/fedora.img bs=1M count=1 seek=1024

Format image file and create filesystem with an ext3 journal.
mke2fs -F -j /root/fedora.img

Mount the image file.
mount -o loop /root/fedora.img /mnt

Create /dev on the mounted image with a few required device nodes.
for i in console null zero ; do /sbin/MAKEDEV -d /mnt/dev -x $i ; done

Create a directory etc in /mnt.
mkdir -p /mnt/etc

Configure partitions, devices and their properties.
Create and edit /mnt/etc/fstab

Mount proc inside the image if it is not already (bug).
mkdir /mnt/proc
mount -t proc none /mnt/proc

Disable the security-enhanced mode.
Edit /mnt/etc/selinux/config. Configure SELINUX=disabled

Install the Guest system in the mounted image file.
yum -c http://home.fnal.gov/~rana/esf/yum-1.conf --installroot=/mnt -y groupinstall Base

Unmount the image.
umount /mnt/proc; umount /mnt

Create and edit /etc/xen/VM_exampleconfig.

Bring up a Guest (xenU) domain.
xm create -c VM_exampleconfig

Check the kernel in use.
uname -r

Configure networking in Guest domain.
Edit /etc/sysconfig/network
Edit /etc/sysconfig/network-scripts/ifcfg-eth0

List the running domains.
xm list

Monitor with Xen's top utility (a snapshot).
xentop

Configure and limit memory available to the Host (xen0) system.
Edit /boot/grub/grub.conf, Configure kernel /xen.gz dom0_mem=600M

Reboot the system.
shutdown -r now

The interface is not set properly to xen-br0 from eth0, if a specific order is not followed. We will restart xend and network to rectify this.
route -n
/etc/init.d/xend stop
/etc/init.d/network restart
/etc/init.d/xend start
route -n

Do more...

Globus Toolkit 4 on FC4

Globus Workspace Service on FC4

ESF Edge Services on FC4

Miscellaneous

• kernel-2.6.12-1.1454_FC4.src.rpm
• kernel-xen0-2.6.12-1.1454_FC4.i686.rpm
• kernel-xen0-devel-2.6.12-1.1454_FC4.i686.rpm
• kernel-xenU-2.6.12-1.1454_FC4.i686.rpm
• kernel-xenU-devel-2.6.12-1.1454_FC4.i686.rpm
• xen-3.0-0.20050912.fc4.i386.rpm
• xen-3.0-0.20050912.fc4.src.rpm

Major updates:

-- AbhishekSinghRana - 20 Sep 2005