Codebuild aws command not found when ran?
So it turned out I was using the wrong environment. Here is what I'm using now:
I was trying to specify my own docker image, which was ultimately not setup with any of the AWS cli utils!
Thanks to @Clare Liguori for tipping me off!