How to remove files that are listed in the .gitignore but still on the repository?

You can remove them from the repository manually:

git rm --cached file1 file2 dir/file3

Or, if you have a lot of files:

git rm --cached `git ls-files -i -c --exclude-from=.gitignore`

But this doesn't seem to work in Git Bash on Windows. It produces an error message. The following works better:

git ls-files -i -c --exclude-from=.gitignore | xargs git rm --cached  

In PowerShell on Windows this works even better (handles spaces in path and filenames):

git ls-files -i -c --exclude-from=.gitignore | %{git rm --cached $_}

Regarding rewriting the whole history without these files, I highly doubt there's an automatic way to do it.
And we all know that rewriting the history is bad, don't we? :)

An easier way that works regardless of the OS is to do

git rm -r --cached .git add .git commit -m "Drop files from .gitignore"

You basically remove and re-add all files, but git add will ignore the ones in .gitignore.

Using the --cached option will keep files in your filesystem, so you won't be removing files from your disk.

Note:Some pointed out in the comments that you will lose the history of all your files. I tested this with git 2.27.0 on MacOS and it is not the case. If you want to check what is happening, check your git diff HEAD~1 before you push your commit.

As the files in .gitignore are not being tracked, you can use the git clean command to recursively remove files that are not under version control.

Use git clean -xdn to perform a dry run and see what will be removed.
Then use git clean -xdf to execute it.

Basically, git clean -h or man git-clean(in unix) will give you help.

Be aware that this command will also remove new files that are not in the staging area.

Hope it helps.