UCSD manually deplopy AIST/NCHC VCs

From PRAGMA wiki
Jump to: navigation, search
Notes for AIST Virtual Cluster.

AIST uses OpenNebula’s “contextualization” to assign IP addresses to a generic Image.

I. Networking Assumptions

They assume two interfaces on every node: public and private.   
Standard beowulf assumes public/private only on the frontend, private on the nodes.

PRAGMA  Discussion - We should figure out the different kinds of virtual cluster topologies we want to support.

II.  building in a Rocks 6.0 (KVM environment).
1.Copy compressed images from Gfarm
2.Allocate vlan, mac addresses with rocks create cluster
a.e.g. rocks create cluster 198.202.88.82 2 fe-name=rocks-82 vlan=82
i.sample nodes: rocks-82, hosted-vm-0-0-1, hosted-vm-0-1-0
b.make sure to set the boot action to be OS and not install on all nodes
i.rocks set host boot action=os rocks-82 hosted-vm-0-0-1 hosted-vm-0-1-0
3.Unpack the VM images from gfarm
a.cat aist_frontend.img.gz | gunzip -c > /state/partition1/kvm/disks/rocks-82.hda
b.cat aist_compute.img.gz | ssh vm-container-0-0 “gunzip -c > /state/partition1/kvm/disks/hosted-vm-0-0-1.hda
c.cat aist_compute.img.gz | ssh vm-container-0-0 “gunzip -c > /state/partition1/kvm/disks/hosted-vm-0-0-1.hda
d.
III. Modify the images.  Use hosted-vm-0-1-0.hda as an example

1.Get the partition mounted

[root@rocks-75 ~]# rocks list host vm hosted-vm-0-1-0

SLICE MEM   CPUS MAC               HOST             VIRT-TYPE

0     1024  1    4a:58:ca:c0:00:06 vm-container-0-1 hvm      

[root@rocks-75 ~]# ssh vm-container-0-1

[root@vm-container-0-1 ~]# file /state/partition1/kvm/disks/hosted-vm-0-1-0.hda

/state/partition1/kvm/disks/hosted-vm-0-1-0.hda: x86 boot sector; GRand Unified Bootloader, stage1 version 0x3, boot drive 0x80, 1st sector stage2 0x144ac10, GRUB version 0.94; partition 1: ID=0x83, active, starthead 32, startsector 2048, 41940992 sectors, code offset 0x48

[root@vm-container-0-1 ~]# losetup -o 1048576 -f /state/partition1/kvm/disks/hosted-vm-0-1-0.hda


[root@vm-container-0-1 ~]# losetup -a

/dev/loop0: [0805]:105480195 (/state/partition1/kvm/disks/hosted-vm-0-1-0.hda), offset 1048576

[root@vm-container-0-1 ~]# mount /dev/loop0 /mnt

[root@vm-container-0-1 ~]# chroot /mnt

[root@vm-container-0-1 /]#
2. Modify the following files
1.[root@vm-container-0-1 /]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
NM_CONTROLLED=no
BOOTPROTO=static
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
HWADDR=4a:58:ca:c0:00:06
NETMASK=255.255.254.0
IPADDR=192.168.57.14

2.[root@vm-container-0-1 /]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
NM_CONTROLLED=no
BOOTPROTO=static
ONBOOT=no
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
HWADDR=02:00:a3:dc:39:71
NETMASK=255.255.254.0
IPADDR=163.220.57.113

[root@vm-container-0-1 /]# tail /etc/sysconfig/iptables
-A PUB -s 150.29.0.0/16 -j ACCEPT
-A PUB -s 150.82.216.0/23 -j ACCEPT
-A PUB -m state --state ESTABLISHED,RELATED -j ACCEPT

### for private network (1Gb)
#PRIV0_ACCEPT_RULES
-A PRIV0 -j ACCEPT
-A PRIV0 -m state --state ESTABLISHED,RELATED -j ACCEPT

COMMIT

3.[root@vm-container-0-1 /]# cat /etc/sysconfig/network
NETWORKING=yes
GATEWAY=192.168.57.1
NISDOMAIN=local
HOSTNAME=ascvm002

4.[root@vm-container-0-1 /]# cat /etc/udev/rules.d/70-persistent-net.rules
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="02:00:a3:dc:39:71", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="02:00:c0:a8:39:71", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

5.[root@vm-container-0-1 /]# cat /etc/resolv.conf
search hpcc.jp
nameserver 8.8.8.8
nameserver 127.0.0.1
nameserver 163.220.2.34


IV.  start the node
rocks start host vm hosted-vm-0-1-0