Configure GT4 WSGRAM on rocks 4.2.1

From PRAGMA_wiki

Jump to: navigation, search

Unless noted, all instructions below should be done by root!
This document is created by Somsak, updated by Cindy and with the help of Nadya.

Globus Container set-up

Create SSH key for Globus user (so it'll stop asking)

mkdir -p /opt/globus/.ssh
ssh-keygen -t rsa -f /opt/globus/.ssh/id_rsa
chown -R globus:globus /opt/globus/.ssh

Create container certificates

cp /etc/grid-security/hostcert.pem /etc/grid-security/containercert.pem
cp /etc/grid-security/hostkey.pem /etc/grid-security/containerkey.pem
chown globus:globus /etc/grid-security/container*.pem

Create init script to start container at /opt/globus/globus-ws.init

#! /bin/sh

# configures Globus WS container
export GLOBUS_LOCATION=/opt/globus
export JAVA_HOME=/usr/java/jdk1.5.0_07
export ANT_HOME=/opt/rocks
export GLOBUS_OPTIONS="-Xms256M -Xmx512M"
export SGE_CELL=default
export SGE_ARCH=lx26-x86
export SGE_EXECD_PORT=537
export SGE_QMASTER_PORT=536
export SGE_ROOT=/opt/gridengine

. $GLOBUS_LOCATION/etc/globus-user-env.sh

cd $GLOBUS_LOCATION
case "$1" in
    start)
        $GLOBUS_LOCATION/sbin/globus-start-container-detached -p 8443
        ;;
    stop)
        $GLOBUS_LOCATION/sbin/globus-stop-container-detached
        ;;
    *)
        echo "Usage: globus {start|stop}" >&2
        exit 1
       ;;
esac
exit 0

Create RC start up script at /etc/init.d/globus-ws

#!/bin/sh
# chkconfig: 2345 99 40
# description: configures Globus WS container
case "$1" in
  start)
    su - globus -c "/opt/globus/globus-ws.init start"
    ;;
  stop)
    su - globus -c "/opt/globus/globus-ws.init stop"
    ;;
  restart)
    $0 stop
    sleep 1
    $0 start
    ;;
  *)
    printf "Usage: $0 {start|stop|restart}\n" >&2
    exit 1
    ;;
esac
exit 0

chkconfig --add globus-ws

Check the existence of /opt/globus/etc/globus_wsrf_rft/jndi-config.xml and /opt/globus/etc/gram-service/jndi-config.xml. If they do not exist, run gpt-postinstall:

/opt/globus/sbin/gpt-postinstall

Change owner/permissions on some configuration files

chmod 755 /etc/init.d/globus-ws
chmod 755 /opt/globus/globus-ws.init
chown globus:globus /opt/globus/etc/globus_wsrf_rft/jndi-config.xml
chown globus:globus /opt/globus/etc/gram-service/jndi-config.xml
su - globus -c "touch $GLOBUS_LOCATION/var/globus-fork.log"

Try start container

service globus-ws start

Test. NOTE: Log-in as a normal user and have your globus certificates ready

counter-client -s https://your-host-name-here:8443/wsrf/services/CounterService

Enable Globus RFT

Download MySQL JDBC Driver from http://dev.mysql.com/downloads/connector/j/3.1.html. Extract mysql-connector-java-3.1.14-bin.jar from the archive and put it at $GLOBUS_LOCATION/lib

tar -zxvf mysql-connector-java-3.1.14.tar.gz mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar
cp mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar $GLOBUS_LOCATION/lib

Create database "rftDatabase" in your MySQL server

mysqladmin -u root -p create rftDatabase
Enter password:(Supply your root password here)

Create user "globus" for the database

cat > /tmp/mysql_grant_globus.sql << EOF
GRANT ALL PRIVILEGES ON rftDatabase.* TO 'globus'@'localhost' IDENTIFIED BY 'your-desired-password-here';
FLUSH PRIVILEGES;
EOF
mysql -u root -p < /tmp/mysql_grant_globus.sql
Enter password:(Supply your root password here)

Initialize database data

mysql -u root -p rftDatabase < $GLOBUS_LOCATION/share/globus_wsrf_rft/rft_schema_mysql.sql
Enter password:(Supply your root password here)

Edit $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml. Replace the following fields
- driverName "value" field with com.mysql.jdbc.Driver
- connectionString "value" field with jdbc:mysql:///rftDatabase
- userName "value" field with "globus"
- password "value" field with your database password
Make sure that your mysql server is started and working. You can try accessing your phpMyAdmin page from "Misc Admin" in your cluster main web page.
Restart globus-ws and check /opt/globus/var/container.log

service globus-ws restart

Verify rft. NOTE: Run as normal user with globus certificates ready

cp $GLOBUS_LOCATION/share/globus_wsrf_rft_test/transfer.xfr /tmp/rft.xfr
vim /tmp/rft.xfr (fix the last two lines, put your hostname instead of localhost, change the path of source and destination file)
rft -h <your hostname here> -f /tmp/rft.xfr

WSGRAM

visudo, add two lines below

globus  ALL=(ALL) NOPASSWD: /opt/globus/libexec/globus-gridmap-and-execute -g /etc/grid-security/grid-mapfile /opt/globus/libexec/globus-job-manager-script.pl *
globus  ALL=(ALL) NOPASSWD: /opt/globus/libexec/globus-gridmap-and-execute -g /etc/grid-security/grid-mapfile /opt/globus/libexec/globus-gram-local-proxy-tool *

Edit file system mapping configuration,

$GLOBUS_LOCATION/etc/gram-service/globus_gram_fs_map_config.xml. Each scheduler must have at least one mapping per scheduler

<?xml version="1.0" ?>
<fileSystemMaps targetNamespace="http://www.globus.org/namespaces/2004/10/gram/job">
<ns1:map xmlns:ns1="http://www.globus.org/namespaces/2004/10/gram/job" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:FileSystemMapType">
 <ns1:scheduler xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">Fork</ns1:scheduler>
 <ns1:ftpServer xsi:type="ns1:FtpServerType">
  <ns1:protocol xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">gsiftp</ns1:protocol>
  <ns1:host xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string"> !!!!! Put your hostname here!!!! </ns1:host>
  <ns1:port xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:unsignedShort">2811</ns1:port>
 </ns1:ftpServer>
 <ns1:mapping xsi:type="ns1:FileSystemPathMappingType">
  <ns1:jobPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:jobPath>
  <ns1:ftpPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:ftpPath>
 </ns1:mapping>
</ns1:map><ns1:map xmlns:ns1="http://www.globus.org/namespaces/2004/10/gram/job" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:FileSystemMapType">
 <ns1:scheduler xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">Multi</ns1:scheduler>
 <ns1:ftpServer xsi:type="ns1:FtpServerType">  
<ns1:protocol xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">gsiftp</ns1:protocol>
  <ns1:host xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">!!!!! Put your hostname here!!!! </ns1:host>
  <ns1:port xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:unsignedShort">2811</ns1:port>
 </ns1:ftpServer> <ns1:mapping xsi:type="ns1:FileSystemPathMappingType">
  <ns1:jobPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:jobPath>
  <ns1:ftpPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:ftpPath>
 </ns1:mapping></ns1:map></fileSystemMaps>

Check globus log files ownership and permissions. They should be

# ls -l /opt/globus/var/*.log
-rw-r--r--  1 globus globus 2087312 Jan 20 13:44 /opt/globus/var/container.log
-rw--w--w-  1 globus globus   12672 Jan 20 13:44 /opt/globus/var/globus-fork.log
-rw-r--r--  1 globus globus    1785 Jul  9  2008 /opt/globus/var/grid-info-system.log

If not, fix it.

Test wsgram NOTE: Run this as normal user with globus certificates ready

globusrun-ws -submit -s -c /bin/hostname

WSGRAM and SGE

Modify SGE to accept reporting

qconf -mconf global
  set reporting=true and joblog=true

as normal user, echo hostname | qsub once (force creation of reporting file)
Download & Install Only SEG & WSRF module from http://www.lesc.ic.ac.uk/projects/SGE-GT4.html.

wget http://www.lesc.ic.ac.uk/projects/globus_scheduler_event_generator_sge-1.1.tar.gz 
wget http://www.lesc.ic.ac.uk/projects/globus_scheduler_event_generator_sge_setup-1.1.tar.gz
wget http://www.lesc.ic.ac.uk/projects/globus_wsrf_gram_service_java_setup_sge-1.1.tar.gz
gpt-build -force globus_scheduler_event_generator_sge-1.1.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
gpt-build -force globus_scheduler_event_generator_sge_setup-1.1.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
gpt-build -force globus_wsrf_gram_service_java_setup_sge-1.1.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)

Download & Install AIST GT4-SGE job manager from http://www.apgrid.org/globus-sge/gt4.html

wget http://www.apgrid.org/globus-sge/globus_gram_job_manager_setup_sge-0.12.tar.gz
gpt-build -force globus_gram_job_manager_setup_sge-0.12.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
gpt-build -force globus_scheduler_provider_setup_sge_0_12.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)

Run gpt-postinstall once

export PATH=/opt/mpich/gnu/bin:$PATH
gpt-postinstall
cd $GLOBUS_LOCATION/setup/globus
./setup-globus-job-manager-sge --mpi-pe=mpich

Patch $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/sge.pm to make it works with SGE6.0u8.
- Copy the contents of sge.pm patch file to $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/sge.pm.diff

cd $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/
patch -p1 < sge.pm.diff

Restart globus-ws

service globus-ws stop
service globus-ws start

Test NOTE: Run as normal user again here, with globus certificates

globusrun-ws -submit -factory <put-your-hostname-here> -Ft SGE -J -S -s -c /bin/hostname

Configure GT4 WSGRAM on rocks 4.2.1

Contents

Globus Container set-up

Enable Globus RFT

WSGRAM

WSGRAM and SGE

Personal tools

Namespaces

Variants

Views

Actions

Search

Navigation

Toolbox