AzureException: Unable to access container using anonymous credentials, and no credentials found for them in the configuration
EDIT:Your storage location should be wasb://container@storageacctname.blob.core.windows.net/inputdir
It sounds like your cluster is not attached to that storage account. Recreate your cluster and make sure that the account that holds the jar is attached to the cluster. Otherwise, move your jar to a storage account that is connected!
This problem has been resolved and I am putting up the answer here in case someone faces the same issue.
I made couple of silly mistakes.
1) It should be wasb://@.blob.core.windows.net. I missed 'blob' earlier.2) There should be a trailing '/'. So it should be hdfs dfs -ls wasb://data2@dsparqstorage.core.windows.net/
The correct format to use Azure Blob storage is
wasbs://<container>@<storage-account-name>.blob.core.windows.net/<directory>/Always make sure to end your wasbs with /