Azure Data Factory can't access HDInsight cluster in IP restricted VNet
With Azure Data Factory V2 the Scenario is supported. It requires deployment of an Azure self-hosted integration runtime (IR) in the vnet of the HDInsight cluster. The self-hosted IR allows Data Factory service to dispatch processing requests to a compute service such as HDInsight inside a virtual network. See also the following Documentation.
ADF can access resources which otherwise can be accessed publicly. If your HDInsight cluster is in a VNet then it cannot be publicly accessed. So ADF cannot access + orchestrate it.
ADF does want to be supported in a VNet environment but that would take some time to land.
Thanks, Harish