Building R with openmpi on Janus

Log into janus

ssh bracken@login.rc.colorado.edu

Move to a compile node

ssh janus-compile1

Create a build directory

mkdir build 
cd build 

Get R, Rmpi and Curl (this is required by some R packages so its good to have)

wget http://cran.rstudio.com/src/base/R-3/R-3.0.2.tar.gz
wget http://cran.r-project.org/src/contrib/Rmpi_0.6-3.tar.gz
wget http://curl.haxx.se/download/curl-7.35.0.tar.gz

Build Curl

tar -xzvf curl-7.35.0.tar.gz
cd curl-7.35.0
./configure --prefix=/home/$USER
make
make install
cd ..

Unpack and build R

tar -xzvf R-3.0.2.tar.gz
cd R-3.0.2
./configure --prefix=/home/$USER
make 
make install
cd ..

Now add your home directory to your path

echo "export PATH=\$PATH:/home/$USER/bin" >> ~/.my.bash_profile
. ~/.my.bash_profile

Load OpenMPI, the newest version of OpenMPI (1.7) does not work

module load openmpi/openmpi-1.6.4_gcc-4.8.1_ib
. /curc/tools/utils/dkinit

Install Rmpi

export MPIHOME=`echo $OMPIDIR`

R CMD INSTALL --configure-vars="CPPFLAGS=-I$MPIHOME/include LDFLAGS='-L$MPIHOME/lib'" --configure-args="--with-Rmpi-include=$MPIHOME/include --with-Rmpi-libpath=$MPIHOME/lib --with-Rmpi-type=OPENMPI" Rmpi_0.6-3.tar.gz

Example job script, save as job.sh

## job.sh example testing Rmpi
## you should see output that has 2 different node names 
##
#!/bin/bash
#PBS -N get_cluster_names
#PBS -l nodes=2:ppn=12,walltime=0:00:30
## Join the Output and Errors in one file.
#PBS -j oe

nodes=2
ppn=12
np=$(($nodes*$ppn))


# go to the jobs working directory
cd $PBS_O_WORKDIR

#Source the Dotkit init so you can pull in extra software via the "reuse" command:
. /curc/tools/utils/dkinit

# Get OpenMPI in our PATH.  openmpi_ipath and openmpi_ib
# can also be used if running over those interconnects. 
module load openmpi/openmpi-1.6.4_gcc-4.8.1_ib

`which mpirun` -n 1 `which R` --vanilla --slave <<EOF

library(parallel)
cl <- makeCluster($np,type="MPI")
clusterCall(cl, function() Sys.info()[c("nodename","machine")])
clusterCall(cl, runif, $np)
stopCluster(cl)
mpi.quit()
EOF

Then submit the job:

qsub -q janus-debug job.sh

When you are ready to submit an actual job then you can use Rscript instead of piping the script to R itself

`which mpirun` -n 1 `which Rscript` --vanilla /projects/bracken/ghcn/ghcn_seasonal_max.R