For site administrators

From PRAGMAgridWIKI

1 Notice
2 Requirements and recommendation for Gfarm v2
3 Installation and upgrade
4 Configuration for file system nodes
5 Configuration for clients
6 The following description is obsolete (for Gfarm v1)
7 Requirements and recommendation
8 Installation and upgrade
9 Upgrade from Gfarm 1.2.x
10 Configuration
11 Tips
12 Any technical questions?

Notice

Please postpone Gfarm deployment in PRAGMA grid until further notice

Requirements and recommendation for Gfarm v2

Allocating a public IP address to all your nodes is the best configuration for Gfarm. If it is not possible, however, your resouces can be still part of the Gfarm testbed and application users will benefit from Gfarm functionality. Please decide your configuration first.

When all your nodes have a public IP address
- Open TCP and UDP ports 11600 from anywhere on all your nodes
- Install the client and file-system-node packages of Gfarm to all your nodes
- Install gfarm2fs to all your nodes

When only the frontend node of the cluster has a public IP address
- Open TCP and UDP ports 11600 from anywhere on the frontend
- Install file-system-node packages of Gfarm to the frontend
- Install the client package of Gfarm to all your nodes
- Install gfarm2fs to all your nodes

[edit]

Installation and upgrade

1. Install required packages

% yum install openssl-devel postgresql-devel
% yum install fuse fuse-devel

2. Obtain the source package from sourceforge.net

% wget http://downloads.sourceforge.net/gfarm/gfarm-2.1.1.tar.gz
% wget http://downloads.sourceforge.net/gfarm/gfarm2fs-1.0.2.tar.gz

3. Install gfarm v2

% tar zxfp gfarm-2.1.1.tar.gz
% cd gfarm-2.1.1
% ./configure --prefix=/opt/gfarm_v2 --with-globus=/opt/globus
% make
% sudo make install

4. Install gfarm2fs

% tar zxfp gfarm2fs-1.0.2.tar.gz
% cd gfarm2fs-1.0.2
% ./configure --prefix=/opt/gfarm_v2 --with-gfarm=/opt/gfarm_v2
% make
% sudo make install

[edit]

Configuration for file system nodes

A Gfarm metaserver has been setup on rocks-45.sdsc.edu, at SDSC, for the PRAGMA testbed.

Email gfarm metaserver administrator (Yusuke Tanimura, cc Cindy Zheng) the hostnames of all the file system nodes on which you will install gfarm software, and a list of global user names in Gfarm file system including the corresponding real name and the GSI subject name. Then the gfarm metaserver administrator will email you a gfsd setup script and open firewall to allow your gfarm nodes to communicate with the metaserver.
Setup Globus certificates, user accounts and firewall - do batch update to install PRAGMA testbed sites certificates, user accounts and firewall settings. Please also make sure to setup your host certificate and that all of files for GSI are placed under /etc/grid-security/.
- CA's certificates, signing policy, and the valid CRL
- Host certificate and key
- grid-mapfile

Setup file system service on all your nodes or the frontend node

Create a user _gfarmfs. Execute a gfsd setup script. This script assumes that the installation directory is /opt/gfarm, the configuration directory is /state/partition2/gfarm_v2, the spool directory is /state/partition2/gfarm_v2/spool. If you would like to change the default locations, use INSTALLDIR, PREFIX, SPOOL environment variable, respectively. Note that you SHOULD NOT use a shared file system for the configuration directory and spool directory. For example, on rocks-53.sdsc.edu, /state/partition1 is a local non-shared partition and has the most space, I want to place gfarm file system there, so I'll set the gfarm configuration directory to /state/partition1/gfarm_v2. Then run:

# PREFIX=/state/partition1/gfarm_v2 sh gfsd_start.sh

[edit]

Configuration for clients

Create $PREFIX/etc/gfarm2.conf

metadb_server_host rocks-45.sdsc.edu
metadb_server_port 11601
auth enable sharedsecret *
auth enable gsi_auth *

sockopt keepalive
spool_server_cred_type host
spool_server_cred_service gfsd

[edit]

The following description is obsolete (for Gfarm v1)

[edit]

Requirements and recommendation

When all your nodes have a public IP address
- Open TCP and UDP ports 600 from anywhere on all your nodes
- Install the client and file-system-node packages of Gfarm to all your nodes
- Install the agent package of Gfarm to one of your node
- Install gfarmfs-fuse to all your nodes

When only the frontend node of the cluster has a public IP address
- Open TCP and UDP ports 600 from anywhere on the frontend
- Install the agent and file-system-node packages of Gfarm to the frontend
- Install the client package of Gfarm to all your nodes
- Install gfarmfs-fuse to all your nodes

[edit]

Installation and upgrade

Here are three kinds of installation and upgrade steps up to your cluster environment. If your system is not built with Rocks, neither CentOS nor Fedora Core, please see the third document.

For Rocks systems, see Gfarm Setup On Rocks.
For CentOS (but non-Rocks) or Fedora Core systems, see this page.
For other linux or non-linux systems, see this page.

[edit]

Upgrade from Gfarm 1.2.x

The location of the metaserver is moved from SDSC to AIST, on the new datagrid testbed of PRAGMA. Because of this, simply remove all your old Gfarm, and newly install and setup the latest Gfarm.

In addition, in order to join the new datagrid testbed, Gfarm must be upgraded to version 1.3.1 or higher.

[edit]

Configuration

A Gfarm metaserver has been setup on pine.hpcc.jp, at AIST, for the PRAGMA testbed.

Email gfarm metaserver administrator (Yusuke Tanimura, cc Cindy Zheng) the hostnames and ip addresses of all the nodes on which you will install gfarm software. Then the gfarm metaserver administrator will email you the gfarm.conf file and open firewall to allow your gfarm nodes to communicate with the metaserver.
Setup Globus certificates, user accounts and firewall - do batch update to install PRAGMA testbed sites certificates, user accounts and firewall settings. Please also make sure to setup your host certificate and that all of files for GSI are placed under /etc/grid-security/.
- CA's certificates, signing policy, and the valid CRL
- Host certificate and key
- grid-mapfile

Setup agent service on one of your nodes

Copy the gfarm.conf mailed from the administrator to /etc/gfarm.conf, and make appropriate changes in /etc/gfarm.conf to set spool directory and gfarm agent hostname. For example, at SDSC site, our agent server is rocks-45.sdsc.edu, the /state/partition1 is a local non-shared partition and has the most space, I want to place gfarm file system there, so I'll set the gfarm spool directory to /state/partition1/gfarm; and our gfarm agent server to rocks-45.sdsc.edu, so I'll change 2 lines in /etc/gfarm.conf as:

spool /state/partition/gfarm
...
agent_serverhost rocks-45.sdsc.edu
...

Then run config-agent to start gfarm_agent

# config-agent

Open 603/tcp to all your gfarm nodes within your site.

Setup file system service on all your nodes or the frontend node

Copy the gfarm.conf from the node where the the agent service is running, and make appropriate changes in /etc/gfarm.conf to set spool directory. Note that you SHOULD NOT use the gfarm.conf mailed from the administrator, and you SHOULD NOT use a shared file system for the spool directory. For example, on rocks-53.sdsc.edu, /state/partition1 is a local non-shared partition and has the most space, I want to place gfarm file system there, so I'll set the gfarm spool directory to /state/partition1/gfarm. Then run: