After two days of trials and failed attempts, I finally got our build server up and running.
Building using MSBuild but without Visual Studio installed proved difficult, but not impossible. However, it was apparently more convenient to just install Visual Studio to get things working.
At this time, the build server will only build our main solution. However, it is possible to extend it to build anything at a later time, if the need arises.