How to end a multi-line command in PowerShell
And right after I asked, I figured it out... Hitting Enter twice made it happen :)
If you continue to get >> prompts after pressing enter twice, check your script for unterminated strings, i.e., unpaired quotes.