When to use SwingUtilies.invokeAndWait/invokeLater
The short answer to your question is: yes, even calling setVisible should happen on the EDT. To find out whether the current thread is the EDT, you can use the EventQueue#isDispatchThread method
Some reference links:
Edit:after reading the links I provided, it seems some of the articles on the Oracle site are outdated as in they still document you can create Swing components on another thread. There is a stackoverflow question on this which contains some nice answers and links to blogposts and articles about the 'new' policy (new as in a few years old)
Yes, if you touch a Swing object you have to do it on the EDT. In most cases you are already there, but if not, use the SwingUtilities classes. The reason for this is that the Swing classes are not multi-threaded, so you are likely to cause nasty problems if you access it on other threads. And it could be that setVisible() is doing a lot of things under the covers to make something display (like re-laying things out). Better to be safe.
Anything that is called from your
public static void main(String[] agrs) {directly (without spawning another thread or using invokeLater) is running on the main thread.
Accessing GUI objects with the main thread while they may be being accessed (simultaneously) by the EDT (which is triggered by user input) can cause threading issues. Calling invokeLater causes tasks (runnables) to run on the EDT, preventing simultaneous access by other EDT tasks ie. button presses etc.
If you can be sure the EDT is not busy (before the first window is setVisible(true)) you can access the GUI from the main thread. If you can be sure the EDT has no reference to the component you're working on (it is out of EDT's scope) ie. before it's added to any container, you can access it from main thread without the EDT accessing it simultaneously, as the EDT has no way to reach it.