To install "RIA Services" in an IS 7.x environment:
1) Download the "RiaServices.msi" file from the Microsoft download page for "WCF RIA Services Release Candidate 2 for Silverlight 4 and Visual Studio 2010".
2) Since you are not expected to have Visual Studio .NET 2010 on the server, it suffices that you only install the assemblies into the GAC. This is done by installing the msi file with these command line swithches:
msiexec /i RiaServices.msi SERVER=TRUE