Source code being exposed by AWS Elastic Beanstalk
I ended up opening a paid case with AWS support and they confirmed it was a bug in the Python container code.
As a result of this problem, they have just released (10/25/2013) a new version of the container and any new environments will contain the fix. To fix any of your existing environments... well, you can't. You'll have to create a new environment from the ground up (don't even use saved configurations) and then switch over from the old one.
Hope this helps the next poor soul.
Update 2017-01-10: Back when I answered it wasn't possible to upgrade the container to newer versions. Since then AWS added that feature. You can even let it auto-update with the Managed Platform Updates feature.
You can also change the value of the said /static alias via the configuration console on your Elastic Beanstalk environment. Under the "Static Files" section, map the virtual path /static to point to your directory app/myapp/static/