The general steps to achieve this are as follows:

1. Fire up regedit
2. Identify the ProgID for your extension - go to HKCR\.yourextension and take note of the default value (in your case, dllfile)
3. Navigate to HKCU\Software\Classes (for user) or HKLM\Software\Classes (for all users)
4. Look for a matching key (in your case dllfile) - if it's not there, create it
5. Ensure it has a sub-key called shell
6. Add a sub-key to shell named as the command you want (refer to image below)
7. Add a sub-key to your new key called command
8. Modify the (Default) value to be the command you want to execute. %1 will give you the path of the file in context (remember to wrap it in " due to potential white-space in the path)

You seem to have done all the above, so you may be doing something wrong, as this is my result after a quick sanity test:

So, here are a few things I can think of that would make it behave non-intuitively:

• You're adding this to HKLM rather than HKCU - due to how inheritance works, I do believe adding it to HKLM would require a restart, or at best, a shell restart
• You've added this to HKCU but your dll requires elevated permissions to access
• You have some silly syntax error somewhere ;)

The sample command I used to test this was a good old boring "C:\Windows\notepad.exe" "%1"

This is based on andromeda947's answer here:

If you have admin rights you can use HKEY_CLASSES_ROOT\SystemFileAssociations\.yourextension, which is simpler as it doesn't require an intermediate ProgID.

Option 1: edit the registry manually

1. add a new key at HKEY_CLASSES_ROOT\SystemFileAssociations\.yourextension\shell\your menu entry text\command creating any keys you need to in that path (if there's not one for .yourextension add it; if there's not one for shell add it; etc)
2. set the default value for command (the last key you created) to C:\path\to\yourapp.exe "%1"

Option 2: I made a tool to do this

you can download it here. This is an example of how to register notepad.exe as a context menu item for dll files.

regwincontext.exe dll "notepad it" C:\Windows\notepad.exe