How do I rename/move items and override even if they exist (for both files, folders and links)?

I think you're looking for the UI experience of having the option to overwrite files and merge directories. These are just sophisticated error handling mechanisms to account for the same errors that you are seeing, thanks to Microsoft's thoughtful engineers.

mkdir C:\Temp\foo -ErrorAction SilentlyContinue'example' > C:\Temp\foo\bar.txtcd C:\TempNew-Item -ItemType Junction -Name bar -Target C:\Temp\fooNew-Item -ItemType SymbolicLink -Name bar2 -Target '.\foo'# Produces error: Rename-Item : Cannot create a file when that file already exists.Rename-Item -Path 'C:\Temp\bar2' -newName 'bar' -force

This makes sense. You have two distinct objects, so they cannot have the same identifier. It would be like trying to point to two different objects

# Unexpected behaviour: moves bar2 inside barMove-item -Path 'C:\Temp\bar2' -destination 'C:\Temp\bar' -force

This is not unexpected. When you specify a directory for the destination, it treats it as the target directory which the moved item should be placed inside.

# This works as per https://github.com/PowerShell/PowerShell/issues/621[IO.Directory]::Delete('C:\Temp\bar')Rename-Item -Path 'C:\Temp\bar2' -newName 'bar'

This is essentially what the thoughtful Microsoft engineers have done for you through their UI for merging folders and overwriting files.

Note that this is the same behavior for the .NET method in System.IO as well