Is mkdir() atomic with AWS Elastic Block Storage as it is on local POXIS filesystems?
Since EBS is not a filesystem...
Amazon Elastic Block Store (Amazon EBS) provides block level storage volumes for use with EC2 instances.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html
...there is no direct answer to the question, because it isn't the correct question. EBS -- as a block device -- is not responsible for or even aware of the actual filesystem operations or its guarantees.
tl;dr: If mkdir() is atomic in the filesystem you are using, then it will still be atomic if that filesystem is on an EBS volume.
Whether using mkdir() for locking is a good practice is a different question.