R programming - submitting jobs on a multiple node linux cluster using PBS

This is rather a PBS question; I usually make an R script (with Rscript path after #!) and make it gather a parameter (using commandArgs function) that controls which "part of the job" this current instance should make. Because I use multicore a lot I usually have to use only 3-4 nodes, so I just submit few jobs calling this R script with each of a possible control argument values.
On the other hand your use of pbsdsh should do its job... Then the value of PBS_TASKNUM can be used as a control parameter.

This was an answer to a related question - but it's an answer to the comment above (as well).

For most of our work we do run multiple R sessions in parallel using qsub (instead).

If it is for multiple files I normally do:

while read infile restdoqsub -v infile=$infile call_r.pbs done < list_of_infiles.txt

call_r.pbs:

...R --vanilla -f analyse_file.R $infile...

analyse_file.R:

args <- commandArgs()infile=args[5]outfile=paste(infile,".out",sep="")...

Then I combine all the output afterwards...

This problem seems very well suited for use of GNU parallel. GNU parallel has an excellent tutorial here. I'm not familiar with pbsdsh, and I'm new to HPC, but to me it looks like pbsdsh serves a similar purpose as GNU parallel. I'm also not familiar with launching R from the command line with arguments, but here is my guess at how your PBS file would look:

#!/bin/sh#PBS -l nodes=6:ppn=2#PBS -l walltime=00:05:00#PBS -l arch=x86_64...parallel -j2 --env $PBS_O_WORKDIR --sshloginfile $PBS_NODEFILE \  Rscript myscript.R {} :::: infilelist.txt

where infilelist.txt lists the data files you want to process, e.g.:

inputdata01.datinputdata02.dat...inputdata12.dat

Your myscript.R would access the command line argument to load and process the specified input file.

My main purpose with this answer is to point out the availability of GNU parallel, which came about after the original question was posted. Hopefully someone else can provide a more tangible example. Also, I am still wobbly with my usage of parallel, for example, I'm unsure of the -j2 option. (See my related question.)