Configure GT4 WSGRAM on rocks 4.2.1

From PRAGMA_wiki
Jump to: navigation, search

Unless noted, all instructions below should be done by root!
This document is created by Somsak, updated by Cindy and with the help of Nadya.

Contents

Globus Container set-up

  • Create SSH key for Globus user (so it'll stop asking)
mkdir -p /opt/globus/.ssh
ssh-keygen -t rsa -f /opt/globus/.ssh/id_rsa
chown -R globus:globus /opt/globus/.ssh
  • Create container certificates
cp /etc/grid-security/hostcert.pem /etc/grid-security/containercert.pem
cp /etc/grid-security/hostkey.pem /etc/grid-security/containerkey.pem
chown globus:globus /etc/grid-security/container*.pem
  • Create init script to start container at /opt/globus/globus-ws.init
#! /bin/sh

# configures Globus WS container
export GLOBUS_LOCATION=/opt/globus
export JAVA_HOME=/usr/java/jdk1.5.0_07
export ANT_HOME=/opt/rocks
export GLOBUS_OPTIONS="-Xms256M -Xmx512M"
export SGE_CELL=default
export SGE_ARCH=lx26-x86
export SGE_EXECD_PORT=537
export SGE_QMASTER_PORT=536
export SGE_ROOT=/opt/gridengine

. $GLOBUS_LOCATION/etc/globus-user-env.sh

cd $GLOBUS_LOCATION
case "$1" in
    start)
        $GLOBUS_LOCATION/sbin/globus-start-container-detached -p 8443
        ;;
    stop)
        $GLOBUS_LOCATION/sbin/globus-stop-container-detached
        ;;
    *)
        echo "Usage: globus {start|stop}" >&2
        exit 1
       ;;
esac
exit 0
  • Create RC start up script at /etc/init.d/globus-ws
#!/bin/sh
# chkconfig: 2345 99 40
# description: configures Globus WS container
case "$1" in
  start)
    su - globus -c "/opt/globus/globus-ws.init start"
    ;;
  stop)
    su - globus -c "/opt/globus/globus-ws.init stop"
    ;;
  restart)
    $0 stop
    sleep 1
    $0 start
    ;;
  *)
    printf "Usage: $0 {start|stop|restart}\n" >&2
    exit 1
    ;;
esac
exit 0
chkconfig --add globus-ws
  • Check the existence of /opt/globus/etc/globus_wsrf_rft/jndi-config.xml and /opt/globus/etc/gram-service/jndi-config.xml. If they do not exist, run gpt-postinstall:
/opt/globus/sbin/gpt-postinstall
  • Change owner/permissions on some configuration files
chmod 755 /etc/init.d/globus-ws
chmod 755 /opt/globus/globus-ws.init
chown globus:globus /opt/globus/etc/globus_wsrf_rft/jndi-config.xml
chown globus:globus /opt/globus/etc/gram-service/jndi-config.xml
su - globus -c "touch $GLOBUS_LOCATION/var/globus-fork.log"
  • Try start container
service globus-ws start
  • Test. NOTE: Log-in as a normal user and have your globus certificates ready
counter-client -s https://your-host-name-here:8443/wsrf/services/CounterService

Enable Globus RFT

tar -zxvf mysql-connector-java-3.1.14.tar.gz mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar
cp mysql-connector-java-3.1.14/mysql-connector-java-3.1.14-bin.jar $GLOBUS_LOCATION/lib
  • Create database "rftDatabase" in your MySQL server
mysqladmin -u root -p create rftDatabase
Enter password:(Supply your root password here)
  • Create user "globus" for the database
cat > /tmp/mysql_grant_globus.sql << EOF
GRANT ALL PRIVILEGES ON rftDatabase.* TO 'globus'@'localhost' IDENTIFIED BY 'your-desired-password-here';
FLUSH PRIVILEGES;
EOF
mysql -u root -p < /tmp/mysql_grant_globus.sql
Enter password:(Supply your root password here)
  • Initialize database data
mysql -u root -p rftDatabase < $GLOBUS_LOCATION/share/globus_wsrf_rft/rft_schema_mysql.sql
Enter password:(Supply your root password here)
  • Edit $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml. Replace the following fields
    • driverName "value" field with com.mysql.jdbc.Driver
    • connectionString "value" field with jdbc:mysql:///rftDatabase
    • userName "value" field with "globus"
    • password "value" field with your database password
  • Make sure that your mysql server is started and working. You can try accessing your phpMyAdmin page from "Misc Admin" in your cluster main web page.
  • Restart globus-ws and check /opt/globus/var/container.log
service globus-ws restart
  • Verify rft. NOTE: Run as normal user with globus certificates ready
cp $GLOBUS_LOCATION/share/globus_wsrf_rft_test/transfer.xfr /tmp/rft.xfr
vim /tmp/rft.xfr (fix the last two lines, put your hostname instead of localhost, change the path of source and destination file)
rft -h <your hostname here> -f /tmp/rft.xfr

WSGRAM

  • visudo, add two lines below
globus  ALL=(ALL) NOPASSWD: /opt/globus/libexec/globus-gridmap-and-execute -g /etc/grid-security/grid-mapfile /opt/globus/libexec/globus-job-manager-script.pl *
globus  ALL=(ALL) NOPASSWD: /opt/globus/libexec/globus-gridmap-and-execute -g /etc/grid-security/grid-mapfile /opt/globus/libexec/globus-gram-local-proxy-tool *
  • Edit file system mapping configuration,

$GLOBUS_LOCATION/etc/gram-service/globus_gram_fs_map_config.xml. Each scheduler must have at least one mapping per scheduler

<?xml version="1.0" ?>
<fileSystemMaps targetNamespace="http://www.globus.org/namespaces/2004/10/gram/job">
<ns1:map xmlns:ns1="http://www.globus.org/namespaces/2004/10/gram/job" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:FileSystemMapType">
 <ns1:scheduler xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">Fork</ns1:scheduler>
 <ns1:ftpServer xsi:type="ns1:FtpServerType">
  <ns1:protocol xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">gsiftp</ns1:protocol>
  <ns1:host xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string"> !!!!! Put your hostname here!!!! </ns1:host>
  <ns1:port xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:unsignedShort">2811</ns1:port>
 </ns1:ftpServer>
 <ns1:mapping xsi:type="ns1:FileSystemPathMappingType">
  <ns1:jobPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:jobPath>
  <ns1:ftpPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:ftpPath>
 </ns1:mapping>
</ns1:map><ns1:map xmlns:ns1="http://www.globus.org/namespaces/2004/10/gram/job" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns1:FileSystemMapType">
 <ns1:scheduler xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">Multi</ns1:scheduler>
 <ns1:ftpServer xsi:type="ns1:FtpServerType">  
<ns1:protocol xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">gsiftp</ns1:protocol>
  <ns1:host xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">!!!!! Put your hostname here!!!! </ns1:host>
  <ns1:port xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:unsignedShort">2811</ns1:port>
 </ns1:ftpServer> <ns1:mapping xsi:type="ns1:FileSystemPathMappingType">
  <ns1:jobPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:jobPath>
  <ns1:ftpPath xmlns:xsd="http://www.w3.org/2001/XMLSchema" xsi:type="xsd:string">/</ns1:ftpPath>
 </ns1:mapping></ns1:map></fileSystemMaps>
  • Check globus log files ownership and permissions. They should be
# ls -l /opt/globus/var/*.log
-rw-r--r--  1 globus globus 2087312 Jan 20 13:44 /opt/globus/var/container.log
-rw--w--w-  1 globus globus   12672 Jan 20 13:44 /opt/globus/var/globus-fork.log
-rw-r--r--  1 globus globus    1785 Jul  9  2008 /opt/globus/var/grid-info-system.log

If not, fix it.

  • Test wsgram NOTE: Run this as normal user with globus certificates ready
globusrun-ws -submit -s -c /bin/hostname

WSGRAM and SGE

  • Modify SGE to accept reporting
qconf -mconf global
  set reporting=true and joblog=true
wget http://www.lesc.ic.ac.uk/projects/globus_scheduler_event_generator_sge-1.1.tar.gz 
wget http://www.lesc.ic.ac.uk/projects/globus_scheduler_event_generator_sge_setup-1.1.tar.gz
wget http://www.lesc.ic.ac.uk/projects/globus_wsrf_gram_service_java_setup_sge-1.1.tar.gz
gpt-build -force globus_scheduler_event_generator_sge-1.1.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
gpt-build -force globus_scheduler_event_generator_sge_setup-1.1.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
gpt-build -force globus_wsrf_gram_service_java_setup_sge-1.1.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
wget http://www.apgrid.org/globus-sge/globus_gram_job_manager_setup_sge-0.12.tar.gz
gpt-build -force globus_gram_job_manager_setup_sge-0.12.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
gpt-build -force globus_scheduler_provider_setup_sge_0_12.tar.gz gcc64 (NOTE: Replace gcc32 here if you're using i386)
  • Run gpt-postinstall once
export PATH=/opt/mpich/gnu/bin:$PATH
gpt-postinstall
cd $GLOBUS_LOCATION/setup/globus
./setup-globus-job-manager-sge --mpi-pe=mpich
  • Patch $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/sge.pm to make it works with SGE6.0u8.
    • Copy the contents of sge.pm patch file to $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/sge.pm.diff
cd $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/
patch -p1 < sge.pm.diff
  • Restart globus-ws
service globus-ws stop
service globus-ws start
  • Test NOTE: Run as normal user again here, with globus certificates
globusrun-ws -submit -factory <put-your-hostname-here> -Ft SGE -J -S -s -c /bin/hostname
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox