I get so many folks contacting me asking what do I recommend when storing massive amounts of files in SharePoint. Using SP as a document management tool. As a developer I recommend creating an application that will do this for them. However I know the answer is “We have to use SharePoint” and that’s very true because of all of the built in features.
For this reason, I would like to recommend this approach to everyone:
- Have a staging document library that implements all the possible content types.
- Create many sites with its own content database, design this structure with the idea in mind that you will want to evenly split up your files
- Create a workflow that moves those files from a staging document library to one of the document libraries in another content database
- Design a new search screen and configure search to enable searching these libraries with the different content types as a drop down. To do this, you may need to modify the XSLT
- Create several front end application servers (web apps) that will do nothing but crawl the site, leaving the other front end(s) usable and ready for user interaction.
Now, determine the size requirements for your content DB for now, and the length of your retention schedule in years but no more than 7 years. Add up the length of all of your files, and ensure that you multiply that number by 10 if you have versioning turned on.
Because SharePoint 2010’s recommend content database size should be < 200 GB you might want to consider remote storage by using SQL’s FILESTREAM provider. You will have to get it working and it’s a small chore. Please reference this link to get started: http://technet.microsoft.com/en-us/library/ee748631.aspx
Build an application that will COPY (not move) your files from the old system to the new system. Make sure you throttle the files. Also make sure that you keep a detailed log about every file you COPY over.
It is also a great idea to keep the old system in place and for the life of the retention just in case you find any immediate issues in the future. Maybe create a new DNS name for the legacy file server, but move the DNS to the new system if needed.
That’s my two cents.
If When I do something like this, I will blog about it!