The workaround given by amiuhle works fine for me:
git push --no-verify
It prevents the pre-push hook of git lfs from trying to upload the files. So link-files are committed, but the actual binaries are missing.