Delphi XE2 assigning Application.MainForm.Handle to Application.Handle inside a DLL

Your solution is perfectly reasonable. I have an Excel COM add-in that does something very similar. In that code I set Application.Handle in the DLL to be the window handle of the Excel main window. That's analagous to what you are doing.

The issue is that you need to get the window ownership set correctly. You need the chain of ownership to reach all the way back to your app's main form. Forms in a DLL have no knowledge of what the main form is, and so you have to provide that knowledge.

Note that I am talking about the concept of window owner as used by Windows and not the VCL concept of owner which is totally different. In VCL terminology this is known as popup parent and you could solve your problem by explicitly setting the DLL form's popup parent to be the main form. The relevant properties are PopupMode and PopupParent. For the forms that live in the main app, the VCL will naturally make their popup parent be the main form.

However, having talked about explicitly setting popup parent, I would stress that your current solution is simpler and more convenient.

What both of these solutions do is to make sure that all auxiliary forms are owned by the main form. That means that these forms are always on top of the main form. It means that the auxiliary forms will be minimized if the main form is minimized. Read about owned windows here: Window Features.

Incidentally, if you had been using runtime packages rather than a DLL, the code in the package would be connected to the same VCL as the main form. So the packaged code would be able to see the main form and set the window owner appropriately. This is certainly one advantage of using packages. Of course, there may very well be a good reason why you need to use DLLs rather than packages.