Understanding Mongo Explain Index scanning to many results
The nscanned count is the total number of documents with a modTime greater than the time provided.
If there is good variability in the color field then you may benefit from reversing the index so that the color is the first field and modTime the second.
db.<collection>.createIndex( { color: 1, modTime : -1 } ) This will then only scan the number of documents of the right color that have a modTime greater than the time provided. If the only colors in the collection are Green, Blue, Yellow then there is no benefit. If there are at least a few additional colors you should get some benefit.
Warning: If you have queries that only use modTime then they will not be able to use the new index.
The other potential solution is to close the modTime range with a $lt (or $lte) expression. You will have to determine if that is possible for your query or if it will make a difference in the number of possible documents to evaluate.