The PRAGMA Grid is established May 18, 2004
by PRAGMA Resources Working Group. This grid is composed of cluster systems and
technical expertise from PRAGMA members and friends institutions. It provides
the infrastructure and a collaborative environment for grid middleware and grid
applications to interoperate
and improve. It's
also where PRAGMA researchers can innovate and experiment
with new approaches and new solutions to make the grid easy to use. Please see
more detail information on PRAGMA Grid resources.
1.
Real science and real grid applications -
per PRAGMA charter.
2.
Applications can
submit jobs through globus gatekeeper to local job
schedulers (SGE, PBS, LSF, SQMS, etc.).
This enables resource sharing in PRAGMA grid.
3.
No absolute rules, and things may change as better technologies develop.
In case of any questions on the suitability of your application, contact Cindy Zheng.
1
Request to run
application in PRAGMA grid
Email Cindy Zheng, to introduce
·
yourself
·
your affiliation
with PRAGMA
·
description of
your application (what scientific problem it's trying to solve)
·
the
application/middleware you'd like to run
·
the person who
will be driving the application run
·
your goal
(results and schedule)
·
resources required for your application.
If you want to know what resources we have now in PRAGMA Grid, you can browse our resources info and check our
resources availabilities in real-time.
If your application is
suited to run in PRAGMA grid, Cindy will add you into the appropriate PRAGMA
mailing lists and setup an account for you at wiki.pragma-grid.net.
2
Document your
application
After you receive wiki
account info, please follow the application
document guideline and template
to document your application at http://goc.pragma-grid.net/wiki/index.php/Applications
and inform Cindy Zheng when done.
3
Request for PRAGMA
Grid access
Currently, PRAGMA grid is in
transition to VOMS based authentication mechanism - some sites are using VOMS
and some sites are not yet. You can choose to use either one or both of the two
sets of resources:
a)
To access VOMS-aware
sites, follow the steps in PRAGMA
VOMS User Guide
b)
To access
non-VOMS-aware site, follow the steps in PRAGMA User
Account Application
4
Setup and Test ssh and globus access
We will setup and test with
a SDSC cluster first, to create and verify user info tarball
before asking all sites to do the setup.
a)
Cindy will create
a tarball with the info you provided in step 2, setup
your account on a SDSC cluster and inform you the <hostname> and
<username>.
b)
You test access
i)
Test ssh access
On your local UNIX system
where ssh client client software is installed, make sure that your ssh private key is correctly installed in ~/.ssh directory. List of ~/.ssh
directory should show both public and private key files. For example:
$ ls -l ~/.ssh
…
-rw------- 1 cindy cindy 951 Mar 7 2007 id_rsa
-rw-r--r--
1 cindy cindy 234 Mar 7
2007 id_rsa.pub
…
(Make sure that you have a
backup copy of these files.)
Then run
$ ssh <username>@<hostname>.sdsc.edu
You should be prompted for a
passphrase. Enter the passphrase you
used to create the ssh key
before and this should enable you to login. On SDSC Rocks clusters, as a new
user on your first login, you will be asked to create a ssh key. This ssh
key should only be used internally within the cluster. For simplicity, hit the
return key 3 times to accept the default location for .ssh
key files and to create the passphrase with empty strings.
If you failed ssh login, then run
$ ssh –v
<username>@<hostname>.sdsc.edu
And email the output to
Cindy.
ii)
Test Globus
access
On your local UNIX system
where Globus software is installed, make sure that your Globus user certificate
and key files are correctly installed in ~/.globus
directory. List of ~/.globus directory should show
the certificate and key files. For example:
$ ls -l ~/.globus
…
-r-------- 1 cindy cindy 4998 Jan 25 2007 usercert.pem
-r-------- 1 cindy cindy 1743 Jan 25 2007 userkey.pem
(Make sure that you have a
backup copy of these files.)
Then run
$ grid-proxy-init
It will prompt you for your
“GRID pass phrase”. Enter the passphrase you used when you
requested your Globus user certificate. If successful, it will return
“Creating proxy ... Done”. Then, run
$ globusrun -a –r <hostname>.sdsc.edu
If successful, it will
return “GRAM Authentication test successful”.
If any problems, please
email all the output from the above commands to Cindy.
iii)
Test job submission
If the Globus access tests
are successful, you can test job submission by running
$ globus-job-run
<hostname>.sdsc.edu/jobmanager-sge
/bin/hostname
Depending how busy the
<hostname>.sdsc.edu is (you can check by ssh login to the host and run “qstat –f”), you may need to wait until the job
get its turn to execute. Then it should return a host name or a node name. (On
SDSC Rocks systems, you may ignore warning messages in this output, as long as
it returns a host name or a node name, the job submission is successful.)
If the job submission
failed, please email the output to Cindy.
5
Setup and test at
more sites
When all the tests in step 5
confirmed to be successful, Cindy will upload your user info tarball to the PRAGMA Grid Operation Center (GOC) web side
and email all PRAGMA grid site admins, refer them to your wiki documents for
your application requirements and ask the qualified sites to setup your
account.
As each site inform you that
your account has been setup, repeat step 5 to test your
ssh and globus access at
each site. Note that each site may use different job schedulers (PBS, SGE, …). See http://goc.pragma-grid.net/pragma-doc/computegrid.html
and http://goc.pragma-grid.net/wiki/index.php/Site_status_and_tasks
for system, software and contact info for each site.
You can also create and run script
to find out which cluster is ready or having problems.
If you have any problems
with any site, please email detail info to the site contact and cc Cindy.
6
Run and document
Don’t wait for all
sites setup your access. You can start testing your code on 1 or 2 clusters
first. If successful, expand to more sites.
Through-out the application
run, please continue to document your application status, logs, data, results
at the PRAGMA wiki site.
We hope to hear your success
and lessons learned, and to see our long-term collaborative relationship grow.