how do aim bots in fps games work?

Somewhere in the game memory is the X,Y, and Z location of each player. The game needs to know this information so it knows where to render the player's model and so forth (although you can limit how much the game client can know by only sending it player information for players in view).

An aimbot can scan known memory locations for this information and read it out, giving it access to two positions--the player's and the enemies. Subtracting the two positions (as vectors) gives the vector between the two and it's simple from there to calculate the angle from the player's current look vector to the desired angle vector.

By sending input directly to the game (this is trivial) and fine-tuning with some constants you can get it to aim automatically pretty quickly. The hardest part of the process is nailing down where the positions are stored in memory and adjusting for any dynamic data structure moving players around on you (such as frustum culling).

Note that these are harder to write when address randomization is used, although not impossible.

Edit: If you're wondering how a program can access other programs memory, the typical way to do it is through DLL injection.

Edit: Since this is still getting some hits there are more ways that aimbots work that are more popular now; namely overwriting (or patching in-place) the Direct3D or OpenGL DLL and examining the functions calls to draw geometry and inserting your own geometry (for things like wall-hacks) or getting the positions of the models for an aimbot.

Interesting question - not exactly your answer but I remember in the early days of Counter-Strike people used to replace their opengl32.dll with a botched one that would render polygons as transparent so they could see through the walls.

The hacks improved and got more annoying, and people got more creative. Now Valve/Steam seems to do a good job of removing them. Just a bit of warning if you're planning on playing with this stuff, Steam does scan for 'hacks' and if any are found, they'll ban you permanently

A lot of "Aim bots" aren't aim bots at all but trigger bots. They're background processes that wait until your reticule is actually over a target and fire automatically. This can be accomplished in a number of different ways but a lot of games make it easy by displaying the name of someone whenever your target goes over them or some other piece of data in memory that a trigger bot can pin point.

This way, you play by waving the mouse at your target and as soon as you mouse over them it will trigger a shot without your having to actually fire yourself.

They still have to be able to pinpoint that sort of stuff in memory and have the same sort of issues that truer "Aim bots" do.