Are there any free/open source build systems to which you can volunteer your own computer as a build agent?
Jenkins can do it with slaves. See this document. You can group slaves by giving them (multiple) labels and "attach" builds to labels. Jenkins slave agents can run under users with limited permissions. An agent can be started and terminated by slave owner - thus they can be sure that there are no security issues.
One potential issue is that a slave can be brought down during a build.
I've only used it with Windows, but TeamCity might do this.