Building lxml for Python 2.7 on Windows

I bet you're not using VS 2008 for this :)

There's def find_vcvarsall(version): function (guess what, it looks for vcvarsall.bat) in distutils with the following comment

At first it tries to find the productdir of VS 2008 in the registry. If that fails it falls back to the VS90COMNTOOLS env var.

If you're not using VS 2008 then you have neither the registry key nor suitable environment variable and that's why distutils can't find vcvarsall.bat file. It does not check if the bat file is reachable through the PATH environment variable.

The solution is to define VS90COMNTOOLS variable to point to Tools directory of Visual Studio.

That being said take a look at 11.4. distutils.msvccompiler — Microsoft Compiler section in Python's docs which states

Typically, extension modules need to be compiled with the same compiler that was used to compile Python.

Martin v. Loewis in the email titled Download Visual Studio Express 2008 now on python-list mailing list states the same

Python 2.6, 2.7, and 3.1 are all built with that release (i.e. 2008). Because of another long tradition, Python extension modules must be built with the same compiler version (more specifically, CRT version) as Python itself. So to build extension modules for any of these releases, you need to have a copy of VS 2008 or VS 2008 Express.

In the light of above statements you should use VS 2008 if you want to build lxml for Python 2.7 so although setting VS90COMNTOOLS takes care of finding vcvarsall.bat file it's not the solution.

That being said :) people do try to use older CRT with newer compiler:
Can I use Visual Studio 2010's C++ compiler with Visual Studio 2008's C++ Runtime Library?
How to Enforce C++ compiler to use specific CRT version?
VS 2008 - Link against older C runtime

I'd like to thank Kev Dwyer (for pointing out importance of version of VS which is used) and Stefan Behnel (for pointing me to distutils as a place dealing with compiler's configuration) in the thread Problem building lxml under Windows - error: Unable to find vcvarsall.bat on lxml mailing list. I'd like to also thank agronholm from freenode #distutils IRC channel for confirmation that distutils does contain code which looks for vcvarsall.bat file.

After following the recommended solution:

1. downloading VCForPython27.msi from Microsoft,
2. installing it (Win7, Python(x, y) 2.7.9 32bit),
3. entering / updating the environment variable VS90COMNTOOLS to the installation directory value (C:\Program Files (x86)\Common Files\Microsoft\Visual C++ for Python\9.0)

my problem still existed (want to build a Python extension in C).

I had to do the following 2 unbelievably dirty tweaks, before everything now is indeed working:

1. modify "msvc9compiler.py" in "C:\Python27\Lib\distutils",function find_vcvarsall, to now point to "Visual C++ forPython" instead of to "VC".
2. copy the directories founder under "C:\Program Files (x86)\CommonFiles\Microsoft\Visual C++ for Python\9.0\" to "C:\Program Files(x86)\Common Files\Microsoft\Visual C++ for Python\" (i.e. one dirlevel up).

I cannot tell who was doing something wrong here - probably I.

EDIT. Moving directories works because of the issue described in this distutils bug.

even if VS90COMNTOOLS is set, msvc9compiler isn't able to find vcvarsall.bat because it is installed in %installdir%/vcvarsall.bat and not %installdir%/VC/vcvarsall.bat

The described workaround is using the Visual C++ command prompt:

1. Enter MSVC for Python command prompt

2. SET DISTUTILS_USE_SDK=1

3. SET MSSdk=1

4. python.exe setup.py ...

Jorj McKie was almost correct: indeed installing VCForPython27.msi isn't enough, and yes there is an issue in distutils which prevent it from finding find_vcvarsall. In fact the issue is not directly in distutils, but in how VCForPython27.msi was packaged and where vcvarsall.bat is placed (the folders layout is different from the VS2008 SDK).

A simple workaround meanwhile this gets patched maybe in Python 2.7.11: use setuptools instead of distutils.

Another manual workaround if you're stuck with distutils:

1) Enter MSVC for Python command prompt2) SET DISTUTILS_USE_SDK=13) SET MSSdk=14) you can then build your C extensions: python.exe setup.py ...

Bug report and workaround by Gregory Szorc:http://bugs.python.org/issue23246

More info and a workaround for using %%cython magic inside IPython: https://github.com/cython/cython/wiki/CythonExtensionsOnWindows