How can I create a window (HWND) without using CreateWindow(Ex)?
Three simple guesses:
1) Is it possible that VB apps are really calling a "DialogBox" API (e.g. DialogBoxParam, CreateDialogIndirect, etc...) underneath the hood?
2) You are running a 64-bit OS and are hooking the 64-bit user32.dll. 32-bit apps aren't getting hooked as a result. There's a 32-bit copy of user32.dll in c:\windows\syswow64
3) You aren't hooking the user32.dll that the apps are using. Many older apps may be getting some DLL redirection. From a command prompt, do "dir /s user32.dll" from the c:\windows\winsxs directory. You'll see at least one other copy of user32.dll here. Forget when this happens, but you can Bing for "winsxs" and get some pages discussion how the side by side directory solves compat issues on newer windows OS releases.
I suspect #3 is the reason for your issue.