How to compute ROC and AUC under ROC after training using caret in R? How to compute ROC and AUC under ROC after training using caret in R? r r

How to compute ROC and AUC under ROC after training using caret in R?


A sample example for AUC:

rf_output=randomForest(x=predictor_data, y=target, importance = TRUE, ntree = 10001, proximity=TRUE, sampsize=sampsizes)library(ROCR)predictions=as.vector(rf_output$votes[,2])pred=prediction(predictions,target)perf_AUC=performance(pred,"auc") #Calculate the AUC valueAUC=perf_AUC@y.values[[1]]perf_ROC=performance(pred,"tpr","fpr") #plot the actual ROC curveplot(perf_ROC, main="ROC plot")text(0.5,0.5,paste("AUC = ",format(AUC, digits=5, scientific=FALSE)))

or using pROC and caret

library(caret)library(pROC)data(iris)iris <- iris[iris$Species == "virginica" | iris$Species == "versicolor", ]iris$Species <- factor(iris$Species)  # setosa should be removed from factorsamples <- sample(NROW(iris), NROW(iris) * .5)data.train <- iris[samples, ]data.test <- iris[-samples, ]forest.model <- train(Species ~., data.train)result.predicted.prob <- predict(forest.model, data.test, type="prob") # Predictionresult.roc <- roc(data.test$Species, result.predicted.prob$versicolor) # Draw ROC curve.plot(result.roc, print.thres="best", print.thres.best.method="closest.topleft")result.coords <- coords(result.roc, "best", best.method="closest.topleft", ret=c("threshold", "accuracy"))print(result.coords)#to get threshold and accuracy


Update 2019. This is what MLeval was written for (https://cran.r-project.org/web/packages/MLeval/index.html), it works with the Caret train output object to make ROCs, PR curves, calibration curves, and calculate metrics, such as ROC-AUC, sensitivity, specificity etc. It just uses one line to do all of this which is helpful for my analyses and may be of interest.

library(caret)library(MLeval)myTrainingControl <- trainControl(method = "cv",                                   number = 10,                                   savePredictions = TRUE,                                   classProbs = TRUE,                                   verboseIter = TRUE)randomForestFit = train(x = Sonar[,1:60],                         y = as.factor(Sonar$Class),                         method = "rf",                         trControl = myTrainingControl,                         preProcess = c("center","scale"),                         ntree = 50)##x <- evalm(randomForestFit)## get roc curve plotted in ggplot2x$roc## get AUC and other metricsx$stdres