Condor-ViNe test

From PRAGMA wiki
Jump to: navigation, search

Network environment

  • See UCSD implementation. Specifically, vm-container-0-13 and vm-container-0-14 are in ViNe subnet.
  • Added a network route (physical and network configuration) to on vm-container-0-14
  • Added VLAN 4001 for the new route
# rocks add host interface vm-container-0-14 vlan4001 subnet=private vlan=4001

Create virtual cluster

  • A Rocks virtual cluster with condor roll is created on fiji
# rocks add cluster 1 container-hosts=vm-container-0-13 fe-container=vm-container-0-14 fe-name=rocks-214 vlan=354
# rocks set host interface vlan rocks-214 eth1 4001
  • Set DHCP on compute node
# rocks set host interface options compute-0-0 eth0 dhcp
# rocks sync host network compute-0-0

Make compute node disk image

On compute-0-0

  • Prevent compute node to reinstall on boot
# chkconfig --del rocks-grub
# poweroff

On vm-container-0-13, gzip compute node disk image to /share/vine-disk

# gzip -c /state/partition1/kvm/disks/hosted-vm-0-13-1.vda > /share/vine-disk/vm-images/SDSC/vine_condor.img.gz

Deploy the compute node image to UFL ViNe subnet

  • Download the VM image

On UFL VMM, /share/vine-disk is NSF mounted.

# gunzip -c /share/vine-disk/vm-images/SDSC/vine_condor.img.gz > /var/lib/libvirt/images/pg/vine_condor.img
  • Modify the image

Some configuration files in the VM image need to be edited. To access the files, the filesystem in the image needs to be mounted on the VMM.

# kpartx -a /var/lib/libvirt/images/pg/vine_condor.img
# mount /dev/mapper/loop0P1 /mnt

1. Edit /mnt/etc/sysconfig/network-scripts/ifcfg-eth0 and comment out the MAC address of eth0

2. Edit /mnt/etc/sysconfig/network and change the HOSTNAME

3. Edit /mnt/etc/hosts and add hosts as needed

4. Edit /mnt/etc/udev/rules.d/70-persistent-net.rules and remove all entries

5. Create a file /mnt/etc/sysconfig/network-scripts/route-eth0 with a static route needed to join ViNe overlay.

# echo " via dev eth0" >> /mnt/etc/sysconfig/network-scripts/route-eth0

On the VMM, unmount the image.

# umount /mnt
# kpartx -d /var/lib/libvirt/images/pg/vine_condor.img

Start the VM.

# /usr/libexec/qemu-kvm -m 1024 -hda /var/lib/libvirt/images/pg/vine_condor.img -net nic,macaddr=54:52:00:1e:c5:98 -net tap,script=/root/pragma/qemu-ifup

To get condor to work between and compute-ufl-0-1

Changes made on UFL compute node

These changes can be included in the compute node disk image

  • Modify /opt/condor/etc/confor_config.local
# cd /opt/condor/etc
# diff condor_config.local condor_config.local.backup
< HOSTALLOW_WRITE =,, *.local, *.local
> HOSTALLOW_WRITE =, *.local, *.local
  • Restart condor
# service rocks-condor stop
# service rocks-condor start
  • Add the line " rocks-214.local" to /etc/hosts

Changes made on (condor master)

  • Change Condor attribute
# rocks set attr Condor_HostAllow value=","
# rocks sync condor
  • Add a static route to the whole ViNe subnet
# rocks add route netmask=
# rocks sync host network compute-0-0 rocks-214
  • Change /etc/exports and restart NFS
# cat /etc/exports
# service nfs restart