Python distributions and environments for scientific computing
Scientific computing with Python is taking a plain vanilla language and bolting on a bunch of modules, each of which implement some aspect of the functionality of MATLAB. As such the experience with Python scientific programming is a little incohesive c.f. MATLAB. However Python as a language is much cleaner. So it goes.
The basic necessary modules for scientific computing in Python are Numpy, Matplotlib, SciPy and if you are doing 3d plotting, then Mayavi/VTK. These modules all depend on Numpy.
Numpy Implements a new array type that behave similar to MATLAB arrays (i.e. fast vector calculations). It also defines a load of functions to do these calculations which are usually named the same as similar functions in MATLAB.
Matplotlib Allows for 2d plotting with very similar commands to MATLAB. Matplotlib also defines pylab, which is a module that - with a single import - brings most of the Numpy and Matplotlib functions into the global namespace. This is useful for rapid/interactive scripting where you don't want to be typing lots of namespace prefixes.
SciPy is a collection of Python modules arranged under the SciPy umbrella that are useful to scientists. Fitting routines are supplied in SciPy modules. Numpy is part of Scipy.
Spyder is a desktop IDE (based on QT) that loosely tries to emulate MATLAB IDE. It is part of the Python-XY distribution.
IPython provides an enhanced interactive Python shell which is useful for trying out code and running your scripts and interacting with the results. It can now be served to a web interface as well as the traditional console. It is also embedded in the Spyder IDE.
Distributions
Getting all these modules running on your computer can be time consuming and so there are a few distributions that package them (plus many other modules) up for you.
Python-XY, WinPython, Enthought and more recently Anaconda are all full package distributions that include all the core modules, although Enthought does not come with Spyder.
Sage is another programming environment which is served over the web or via a command line and also comes as a full package including lots of other modules. Traditionally it came as a VMWare image based on an install of Linux. Although you are writing Python in the Sage environment, it's a little different to ordinary Python programming, it kind of defines its own language and methodology based on Python.
If you are using Windows I would install WinPython. It installs everything that you need including Scipy and Spyder (which is the best replacement for MATLAB for Python IMHO) and because it is designed to be standalone it will not interfere with other installs of Python you may have on your system. If you are on OSX, Enthought is probably the best way to go - Spyder can be installed separately using e.g. MacPorts. For Linux you can install the components (Numpy, SciPy, Spyder, Matplotlib) separately.
I personally don't like the Sage way of working with Python 'hidden under the hood' but you may prefer that.
Regarding the less important part of the question:
This link may be usefull: https://www.cfa.harvard.edu/~ebresser/python/
It's the page of an astrophysicist at Harvard. It gives the point of view of someone switching from ITT-VIS IDL to python, on OS-X (but most tips also work on other operating systems).
EDIT: It seems the page was taken down. An alternative good introduction to python for a scientist/engineer is in this document (big PDF warning): http://stsdas.stsci.edu/perry/pydatatut.pdfHope this one will not be taken down!