How to clusterExport a function without its evaluation environment How to clusterExport a function without its evaluation environment r r

How to clusterExport a function without its evaluation environment


It's hard to guess the problem without a complete example, but I'm wondering if the error message isn't coming from clusterExport, rather than parLapply. That would happen if functionList was defined in a function rather than the global environment, since the clusterExport envir argument specifies the environment from which to export the variables.

To export variables defined in a function, from that same function, you would use:

clusterExport(cl, varlist=c("functionList", "y"), envir=environment())

I'm just guessing this might be a problem for you since I don't know how or where you defined functionList. Note that clusterExport always assigns the variables to the global environment of the cluster workers.

I'm also suspicious of the way that you are apparently setting the environment of a list: that seems to be legal, but I don't think it will change the environment of functions in that list. In fact, I suspect that exporting functions to the workers in a list may have other problems that you haven't encountered yet. I would use something like this:

mainFunction <- function(cl) {    fa <- function(x) fb(x)    fb <- function(x) fc(x)    fc <- function(x) x    y <- 7    workerFunction <- function(i) {        do.call(functionNames[[i]], list(y))    }    environment(workerFunction) <- .GlobalEnv    environment(fa) <- .GlobalEnv    environment(fb) <- .GlobalEnv    environment(fc) <- .GlobalEnv    functionNames <- c("fa", "fb", "fc")    clusterExport(cl, varlist=c("functionNames", functionNames, "y"),                  envir=environment())    parLapply(cl, seq_along(functionNames), workerFunction)}library(parallel)cl <- makeCluster(detectCores())mainFunction(cl)stopCluster(cl)

Note that I've taken liberties with your example, so I'm not sure how well this corresponds with your problem.