How do Homebrew, PIP, easy_install etc. work so that I can clean up

Homebrew installs its software inside the /usr/local subdirectory on your Mac. OS X doesn't install anything there on its own; in fact, /usr/local is reserved for user-installed stuff. Since Homebrew never installs files outside /usr/local (and doesn't even have the ability to, unless you run brew using sudo - which is not recommended_) and OS X never installs files inside there, never the two shall mix.

easy_install and pip install files into system directories by default. That's why you have to run those commands with sudo to install packages with them.

I can't recommend virtualenv enough, regardless of which OS you're using. It installs a copy of Python, along with any packages or modules you want, inside a directory of your choosing. For example:

$ cd /tmp$ virtualenv foo         New python executable in foo/bin/pythonInstalling setuptools............done.Installing pip...............done.$ cd foo$ bin/pip install sqlalchemyDownloading/unpacking sqlalchemy  Downloading SQLAlchemy-0.7.7.tar.gz (2.6Mb): 2.6Mb downloaded  Running setup.py egg_info for package sqlalchemy[...]    Successfully installed sqlalchemyCleaning up...[work, work, work][decide this was a bad idea]$ cd /tmp; rm -rf foo

...and all traces of the project are now completely gone.

Use easy_install to install virtualenv into OS X itself - like you've done for those other packages - but then do all new development inside isolated directories that you can wipe clean at a moment's notice. This is pretty much the standard way of developing and deploying Python applications these days.

The advantage of using a Python installed via a package manager like Homebrew or MacPorts would be that this provides a simple way of removing the Python installation and reinstalling it. Also, you can install a more recent version than the one Mac OS X provides.