How to convert a printer driver to a stand-alone console application which can generate a printer file containing the bytes to be sent to the printer?

The general problem which you formulate is difficult to solve. Mostly a printer driver consists from some well known components like Print Monitor, Print Processor etc. which are well documented in Windows Driver Kit http://msdn.microsoft.com/en-us/library/ff560885%28v=VS.85%29.aspx. Some years ago I wrote a Print Monitor. It worked many years at a customer. So I know exactly what I writing about. A Print Monitor is nothing more as a DLL with well documented functions. The same is about most other printer components. Those DLLs will be loaded and called from Spooler. If you have a modern printer driver it has no components which run in kernel mode. So one can load most of DLLs from which consist every printer driver and call corresponding function.

You are interesting for using one concert printer driver. So the first what one should do is to examine how this driver is implemented. If you find out which component do the job which you need, you will be probably able to load this DLL in your process and produce output which you need. It is possible that you post an URL where I could download this driver?

UPDATED: I though a little more about your requirements. It seems to me you can goes with the way suggested by developer of the printer driver. If the driver can print to a local port FILE, then it can print in any printer port. So you can give src of a Port Monitor Server driver from C:\WinDDK\7600.16385.1\src\print\monitors\localmon (see also http://msdn.microsoft.com/en-us/library/ff556478%28v=VS.85%29.aspx, http://msdn.microsoft.com/en-us/library/ff549405%28v=VS.85%29.aspx and http://msdn.microsoft.com/en-us/library/ff563806%28v=VS.85%29.aspx). (I is a windows 32/64 DLL, not a real driver) and makes small modification. Instead of saving results to a file you can dispatch the results to your application. It will be work with 100% without any tricks. If you will have some problem to understand localmon I can give you some tips. It is really not complex. The main changes which you have to do is to modify LcmStartDocPort LcmWritePort LcmReadPort LcmEndDocPort functions from localmon.c. Some easy thing which is distinguish Port DLL from a typical DLL, that instead of exporting all DLL's functions it export only one InitializePrintMonitor2 with pointers to all other functions.

UPDATED 2: One more tip for usage of "Local Port" monitor. If goes in printer configuration, then choose "Add Port...", select "Local Port" and click "New Port..." you can type any file name like "C:\temp\my.bin". Then all what you print through a printer will be printed in this file without any user iteration. The name can be any win32 file name (UNC names or Named pipes are also allowed). With this way you can realize some scenarios without any programming with DDK.

UPDATED 3: I looked at the printer driver from different sides and looked one more time in the API in DDK. Now I want recommend you to choose the easiest way, and the way which will be full supported from the driver manufacturer. I suggest following:

You install a printer with the driver which you need and choose as the output port a Local Port with a fixed file name (see Update 2). I named here the destination filename as C:\TEMP\Output.afp. So you receive exactly the same situation like recommend you driver manufacturer. Fixed file name is absolutely the same as FILE: port. So if you print to the printer you receive in Output.afp file in the C:\TEMP directory. To be sure the end of writing you can use ReadDirectoryChangesW or FindNextChangeNotification / FindFirstChangeNotification functions with dwNotifyFilter equal to FILE_NOTIFY_CHANGE_LAST_WRITE. Then you receive notification after last write-time of the file. It means after the end of writing and after FileClose and after the cache is sufficiently flushed. So the file Output.afp is not locked and you can really safe read the results.

For printing of simple documents you can use WritePrinter function (see http://msdn.microsoft.com/en-us/library/dd162959%28VS.85%29.aspx and remark in the documentation http://msdn.microsoft.com/en-us/library/dd145226%28VS.85%29.aspx). Writing of complex files with bitmaps, color and different fonts you have to use typical GDI API like one this in Windows (see http://msdn.microsoft.com/en-us/library/dd162865%28v=VS.85%29.aspx).

This solution looks not very spectacular like writing a printer driver component or a simulation of spooler environment for printer driver, but it will work, will safe work and will be full supported from the driver manufacturer.

(It's been 10 years since I did anything like this, but I don't think the overall concepts have changed all that much:)

What you want to do is implement a custom print processor. A print processor is the piece of code that takes the output that the printer driver generates and transports it to the output device. Print processors are implemented as regular user-mode DLLs. You should be able to find everything you need, including samples, in the Windows DDK.

A while ago we made a commercial application which captured print streams from any windows application and converted the result to XML and tiff images
We did make a prototype with the DDK, but ended up buying a SDK for the print capturing
The SDK was from BlackIce. Although it wasn´t a free SDK, the distribution of the runtimes were royalty free.

Implementation was done with Visual C (unmanaged) and VB6.The printer driver had to be installed on the server/PC that drove the printing process.
I remember that the tricky part was to control the printer settings in runtime (keep the tiffs compressed, output directory for the files, paper size:A4 or Letter and other settings that were defined in the DEVMODE print control structure).

UPDATE: (Your comment to @Oleg about MO:DCA P triggered my memory. Although it is not about a printer driver...)
For our commercial product, we also had to make a customization to convert MO:DCA (AFP) documents to tiffs and XML.
This SDK had to be able to extract both images and ascii text to enable later conversions
Conversion where then made in batch from AFP documents in one folder to XML and tiffs.
We chose to convert the AFP file after it had been printed (not during print).
The SDK is SnowBound RasterMaster and is available in different flavours (we used the Windows API with ActiveX, and I see now that it is available for Java)
So if your requirement is to convert an AFP document to someting else (extract images and extract ascii text) you could try out the software from SnowBound. Make sure you also get the Optional Feature to be able to extract ASCII text from the MO DCA documents.
This software SDK is more expensive, but it did the job.
They offer a trial version here.