UCSD implementation

From PRAGMA wiki
Revision as of 18:33, 21 October 2015 by Wikiadmin (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Network Wiring

Fiji-vine-network.jpg

Setup ViNe server on a Rocks 6.1 system

Install ViNe software

  • Get ViNe package and install
# wget http://vine.acis.ufl.edu/vine/lib/vine2.tgz
# tar ozxf vine2.tgz -C /opt
# /opt/vine2/bin/cmds/pragma-register.sh
# ln -s /opt/vine2/bin/vine.sh /etc/init.d/vine
# chkconfig --add vine
  • Set Java path in /opt/vine2/etc/vine.vars
# cp /opt/vine2/etc/vine.vars.template /opt/vine2/etc/vine.vars
edit
# diff /opt/vine2/etc/vine.vars /opt/vine2/etc/vine.vars.template
2,3c2,3
< JAVA_HOME=/usr/java/latest
< export PATH=$JAVA_HOME/bin:.:$PATH
---
> #JAVA_HOME=/opt/soft/java
> #export PATH=$JAVA_HOME/bin:.:$PATH
  • Start ViNe service
# service vine start

Configure ViNe software

  • Install ViNe configuration software
# wget http://vine.acis.ufl.edu/vine/lib/vinecsc.tgz
# tar ozxf vinecsc.tgz -C /opt
  • Set Java Path in /opt/vinecsc/bin/csc.functions
# diff /opt/vinecsc/bin/csc.functions /opt/vinecsc/bin/csc.functions.orig
2,3c2,3
< JAVA_HOME=/usr/java/latest
< export PATH=$JAVA_HOME/bin:.:$PATH
---
> # JAVA_HOME=/opt/soft/java
> # export PATH=$JAVA_HOME/bin:.:$PATH
  • Configure ViNe
# service vine stop
# /opt/vinecsc/bin/csc.sh
> vn create 74 "PRAGMA test"
> vr update 1000000a 741
> vr subnet add 741 172.31.11.0 255.255.255.0
> vr subnet del 741 172.16.0.176 255.255.255.240
> hosts del 741
> hosts config 741 74
> vn add vr 741 74
> __quit
> vr update vineip 741 172.31.11.1
> jpacache clear
# service vine start

Configure fiji VM containers

This example is to allow VM containers to run some VMs on 10.1.1.0 subnet and other VMs on 172.31.11.0. Alternatively (maybe little simpler to setup is the dedicate VM container(s) to 172.31.11.0 subnet.

  • On fiji
# rocks add host interface vm-container-0-13 em1.354 vlan=354 subnet=vine ip=172.31.11.241
# rocks add host interface vm-container-0-14 em1.354 vlan=354 subnet=vine ip=172.31.11.240
# rocks add host route vm-container-0-13 172.16.0.0 172.31.11.1 netmask=255.240.0.0
# rocks add host route vm-container-0-14 172.16.0.0 172.31.11.1 netmask=255.240.0.0
# rocks sync host network vm-container-0-13
# rocks sync host network vm-container-0-14
  • Configure the network switch to route VLAN 354 to subnet 172.31.11.0

Setup NFS mount for ViNe project

  • On rocks-188 (nas-0-0 frontend)
# rocks add host interface nas-0-0 eth1.354 vlan=354 ip=172.31.11.11 subnet=vine
# rocks sync host network nas-0-0
  • On nas-0-0
# zfs create state/primary/disks/vine
# zfs set sharenfs=@172.31.11.0/24 state/primary/disks/vine
# zfs set sharenfs=on state/primary/disks/vine
# chmod 1777 state/primary/disks/vine
  • On fiji
# mkdir /share/vine-disk
# echo "vine-disk -nfsvers=3 172.31.11.11:/state/primary/disks/vine" >> /etc/auto.share
# rocks sync users

Testing VM migration

  • On fiji
rocks add host vm vm-container-0-14 membership="Hosted VM" disk="file:/share/vine-disk/vm-images/SDSC/vine-vm1.vda,vda,virtio" name=vine-vm1 vlan=354
  • on ViNe server (rocks-98)

Run "insert-ethers" and select "compute" then "ok"

  • On fiji
# rocks start host vm vine-vm1

Connect to vine-vm1 console via "virt-manager", observe installation of vine-vm1 as compute-0-0 or rocks-98, until complete

  • On ViNe server (rocks-98)
    • "insert-ethers" should have recongnized compute-0-0. Quit out of "insert-ethers".
    • Test "ssh compute-0-0"
  • On vm-container-0-13 and vm-container-0-14
# ln -s /share/vine-disk/vm-images /var/lib/libvirt/images/vine

Also, setup password-less ssh for root between the 2 containers

  • On vm-container-0-14

Run "virsh list" and make sure that vine-vm1 is running, then test live migration to vm-container-0-13

# virsh migrate --unsafe --live vine-vm1 qemu+ssh://172.31.11.241/system
# virsh list

The virsh list should show that the vine-vm1 is no longer running on vm-container-0-14

  • On vm-container-0-13

Run "virsh list" to verify that the vine-vm1 is running

  • * On ViNe server (rocks-98)

Test "ssh compute-0-0" again and it should work as before

  • Similarly, test live migration to UFL containers