Exploring SharePoint Migration

Migrating data is a tough role in any organization. Getting all the data from source to destination is a challenging task, especially with Microsoft SharePoint, mapped drives, and other storage resources. Users will complain about data being missing, even if it is irrelevant. In some cases, they will be irritated or even combative, because they are afraid of change. This overview will give you valuable insights to help you master these challenges to deliver a painless migration.

Getting ready: know your content

When contemplating a SharePoint migration, the first piece of advice is to plan and make sure you understand your users’ data, some of their processes, and the content they are using. You will also need to find out what data is relevant and what data may not be relevant before moving to the new environment. You will find this to be a daunting task when dealing with mapped drives because people do not know what the latest versions of files are in most cases; for example, some may be named FinalFinal. To make sure you’re using the correct files, you must work with the customer to clean up any irrelevant files before going forward. You have lots of choices when it comes to storage locations due to SharePoint and OneDrive being available. Don’t go back to mapped drives if you can avoid it, but if you do, clean up those files before moving them.

When looking at content, you must be prepared to classify and reclassify files as needed and tag them within the new environment correctly. The use of managed metadata and/or columns to describe the content is key for search results. Securing the data based on those classifications comes into play so that once they’ve been migrated to a document library, they can be secured. Files within the libraries can also be secured individually if needed. Breaking inheritance is also a concern at this point when you’re talking to your customer because you do not want to do this a lot when you’re setting up content. So, be mindful of how you set up new libraries in the new environment.

Here is a list of areas to pay attention to before migration:

  • File size

We must also remember that even with mapped drives, we can also migrate to OneDrive. Security and various options for folder organization will help the customer manage those files. It’s not like SharePoint, but if a customer wants to implement this type of storage location, be ready to answer questions. This is contingent on you being in a hybrid environment where you already have Microsoft 365 set up.

If you’re working with MS SharePoint, you will also need to look at versioning settings, column types, and any updated aspects that are moving from one version of SharePoint to another. Help them understand that moving to this new location will give them more control over the data from a security aspect; waiting for an admin to update security for new and existing users within mapped drives is also possible.

This sometimes gets them excited about moving and helps them relax, which will make your users feel as though everything is going to be OK. Show them that you have investigated everything and build confidence in the migration by having meetings about this migration. You can even suggest the use of retention libraries for holding documents online either with OneDrive or SharePoint so that the customer understands there are choices when it comes to how old documents can be handled. Processes can be put in place for handling new files that need to be moved to a separate location for safekeeping as well.

As we mentioned previously, this is not an easy task. You are in the hot seat to figure out how to get 100% of your identified content from one version to the other. We will talk about some of the areas to focus on and which migration method will work for your migration scenario. We have also identified some migration tools within the method to give you a breakdown of my experiences with those tools.

We will also talk about creating schedules for your departments and testing with users as we also want to break down the use of preliminary migrations and final migration testing. If you have not, and we will say this again, create new dev and test environments to make sure you can provide areas within the environment where users can test their content and have new ways of accessing it.

As part of our environment, we should have dev, test, and production environments, which gives us a complete, functional SharePoint environment. This means we have a place we can test out migrations and have the user test them too. The last thing you want to do is a test in your test and production environments; always do this during the development phase. If you have these environments set up, then we can move on and find out what we need to do in order to start moving content from our old farm to the new one. Make sure all your farms have been configured correctly and are the same so that the results of the tests will be the same across the board. Configuration management is important!

During migrations, you also want to make sure you put source content in read-only mode when doing final migrations so that nothing is changed during this process. There are many things you need to make sure are in place before we start this migration. Take full backups of content before the migration, as well as once you have got the content over to the new farm. This backup should be taken once you have settled all issues and confirmed that all the errors have been fixed with the customer. This helps create a baseline that acts as a safeguard from the migration of both the source and target farms.

Know your customers

Identify all the departments that are being migrated and create a list of departments and users (if possible). If you have departments that are ready to move and eager to take advantage of new technology, then work with those customers first. These departments are the ones that will be successful and will give you a chance to build a good story going forward. Make sure to interview all departments before picking and choosing a schedule as well.

The following are some example questions you should ask when performing department interviews:

  • What are you currently using SharePoint for?

It is crucial that the migration is successful. When you fail in front of IT staff, it is OK; you can recover since it’s a small subset of the project. However, when you get to the migration stage, this affects the company because everyone is watching. This also brings a spotlight on why some users do not like SharePoint. Do not give them any ammunition to criticize SharePoint because it can cause issues and it’s something you will have to live with for a while. Believe us, success is the goal here.

Some of the things you can do to be more in tune with your customers or department include understanding what they do. Knowing what they do and how they use SharePoint can go a long way in supporting the department during the migration process. If possible and if you have time, find out who used to work in those departments that might have data that is still valid in the source content. Knowing some of those user accounts that may not be in Active Directory now will help to troubleshoot errors going forward. Some tools that are used when migrating lists and libraries do not work well when versioning items that could have a user account specified in a people and group column because, when the item is migrated, it will fail. Use the tool mapping component to map users and groups if needed. By doing this, accounts that are not there can be masked with the admin account or another account so that the item does not fail during the migration.

Find out what your customer or department needs from you. Whether it is hand-holding or more information for them to feel comfortable, make sure you provide it. There might be multiple questions you wish to ask, including:

  • Are you migrating a file share or SharePoint site?

Reduce the likelihood of any scheduling hold-ups by interviewing all departments before you finalize your schedule. If there is a person on the staff that would like to help or be that go-to person, find that out now so that they can be your liaison with the rest of the team.

Find out the immediate or future goals of the department. This is crucial so that you can get an understanding of where they would like to go with SharePoint. This also gives you the opportunity to discuss how SharePoint can help them manage data more efficiently. Having a demo ready for some of the new features within SharePoint can show you are prepared and that these features are working as they should in the new version. This gives the business analyst some information on what could be done to help them in the future. This is the time to start being proactive and not reactive with your customers. Start fresh and make this transition useful for everyone.

Migration — choosing your weapon of choice

When choosing how we migrate, we must look at the content and versions of SharePoint we are working with. Some questions that you should be asking yourself are as follows:

  • What method should I use to get this task completed?

There are a few ways to migrate content in SharePoint:

  • Content database migration

Let’s take a look at these in more detail.

Content database migration

This is the out-of-the-box task of moving content from one farm to another. This supports migration from version to version and cross-web applications for on-premises environments. There are technical points that need to be addressed as part of both migration paths when using this method. This provides a better capture of the site from a bigger standpoint, but it does not give you some of the capabilities you may need in order to clean up content beforehand. This is best used when custom solutions are not involved, and the content is mostly out of the box.

Using a migration tool

This depends on whether you’re using an external application to move content from one SharePoint site to another site. These tools can be used for either on-premises or cloud migrations. This also means we can migrate from one version of a SharePoint farm to another SharePoint farm. Using a migration tool provides us with flexibility for scheduling, picking specific content, using mapping capabilities to map columns in lists, and updating permissions on items that may be held by users who no longer work at the company, to name a few. Governance tools also come into play with tools you select, as many companies provide those to help you understand your environment before migration.

One thing we do to speed up the migration process is to create as many server-named sites as possible to extend our web applications. This will give you many URL points of entry for content migrations so that you can use separate servers to push the content through, instead of using a DNS name where all traffic will hit one or two servers. The server resources are then separate from one another and you can have different admins pointing different tool installations to different servers. You will see no bottlenecks pushing content through as long as your SQL server can keep up with the influx of data.

Tools also allow you to make choices for other Microsoft cloud services and third-party solutions such as Google, Dropbox, and OneDrive. If you have a tool that can connect to those types of services and you want to migrate content from those repositories, this is the way to do it. A migration tool provides such integrations and can be helpful in moving these documents over to a new platform. It will also leave behind database errors found in the content, such as errors with solutions that were left behind from other versions, plus other content that may have faced errors due to database issues.

You can also use the tool in combination with content database migration and migrate your databases first, before using the tool on the weekend to push incremental changes to the content. This cuts down how long it will take you to move content and save you money since with most tools you must pay per GB. Unfortunately, you cannot use this method when moving to the cloud.

Important note

Remember: your logging files will grow while you’re completing this process, and large amounts of logging data will be created on your SQL and SharePoint servers. Make sure that all your logging services for IIS and SharePoint are pointed to the Data or Log drive on your machine and that it has a lot of space during this process!

PowerShell migration

PowerShell migration is another way we can migrate content. It is more tedious and requires you to be skilled in PowerShell. We do not recommend this method for newbies or admins who do not have this skill set already. The chances of errors occurring are high, and you do not want that when you are on a schedule. It is not the time to practice PowerShell at this point.

Tip

Some examples of PowerShell being used as a migration tool can be found on Microsoft’s website at https://docs.microsoft.com/en-us/sharepointmigration/overview-spmt-ps-cmdlets.

Manual migrations

As an example, let’s say we have a folder on a server that is being used as a shared folder. This folder contains documents that need to be migrated to a SharePoint library. We can take these files from the shared folder and copy them directly into the library using the Windows Explorer functionality within the SharePoint library. This is an easy way to migrate documents from desktops, but the limitation is that you can only copy up to 100 documents at a time. You also do not have the opportunity to add column information or metadata to these documents as they are being uploaded.

Important Note

Beware! Do not make the mistake of having a live site on a destination farm where you have not finished configuring your services. Make sure you have completed all configurations before you put any new ‘live’ sites in your new production environment. If you need to work with some of the departments and have sites that are live while migrating, ensure your dev and test environments are set up so that you can test any migrated content before it’s put into production as a live site. You can then make configuration changes there and not in production to test the farm.

There are other ways we can use manual processes to move data as well. Microsoft Office tools such as Microsoft Access and Microsoft Excel allow us to move content from one list to another or one farm to another. All lists are basically Excel and Access databases.

So, capturing columns and pulling data back into another list is fairly straightforward using PowerShell and even just Access itself. Look around and see what you can find. In some instances, this may work when you are just moving small items or concentrating on one document library and don’t want to purchase something, or you don’t want users bothering the admin when it comes to moving the list through Central Administration.

The Central Administration tool, which can be used to export lists and libraries, can be seen in the following screenshot:

Figure 1 - Exporting a site or list

As you can see, there are many ways to migrate content from SharePoint. Using the tools that work for your situation is the name of the game. Do not just take someone else’s word for it. Some tools work better for some people than others. It’s all about what you want to achieve, not how expensive or popular a solution is. Even within SharePoint, we can pull content from top-level areas and import them. In some cases, this would not work, while in others, it could be the best solution to use. Just make sure it works for you and your requirements.

Now that we have talked about the different ways we can perform migrations, next we will learn about the types of migrations that can be achieved.

Types of migration

In this section, we will cover different types of migration. This will provide you with a use case to follow once you start the migration process. The types of migration that can be achieved are as follows:

  • Version to version

Version to version: A version-to-version migration is when you migrate from a SharePoint 2013 farm to a SharePoint 2019 farm. However, as part of that migration, you need to validate your content through the in-between version of SharePoint — in this case, SharePoint 2016. This also needs to be a content database migration method.

Server to server: Server-to-server migrations can be easy or may become very complex, depending on the way the content was structured in SharePoint. Server-to-server migrations could be when you migrate from one cloud to another cloud where you have the same version of SharePoint, or it could be when you need to upgrade the server platform from Windows Server 2012 R2 to Windows 2016 Server. This could be for licensing or supportability reasons, or just to keep your servers up to date with the latest and greatest server technology. If you are migrating from SharePoint 2007 to SharePoint 2019 using the content database migration method, this could be considered a server-to-server migration as well because when lifting content from 2007, you have to take it through the different versions of SharePoint to get the content to 2019.

When moving through 2010, 2013, and 2016, each server would have to be set up and configured to support the migration. There is also a step in the 2010 migration that will upgrade your workflows and InfoPath forms during the process, and there’s a tool to migrate you to 2019 from that point. You would have to manually migrate the business data catalog (BCS in 2019) from 2007 to 2010, for which you can refer to https://docs.microsoft.com/en-us/previous-versions/office/developer/sharepoint-2010/ff817559(v=office.14). After that, the services database could be moved through this process as well to keep those services intact as you move through the migration. You will also need to migrate classic users to claims, which occurs during the migration from 2007 to 2010. For more information on migrating from classic-mode to claims-based authentication in SharePoint 2013, refer to https://docs.microsoft.com/en-us/sharepoint/upgrade-and-update/migrate-from-classic-mode-to-claims-based-authentication-in-sharepoint-2013.

Service to service: These migrations can be complex. We were recently on a migration from Rackspace to AWS and it was challenging due to the size of the content databases that we had to work with. Getting such databases transferred to a new service is tough, especially when you have small maintenance windows. The one thing to watch out for is the size, but also what type of web application configuration you choose.

Say, for instance, you chose Path-Based Web Applications as your standard application. Now, in terms of migration, all content needs to be migrated all at once, regardless of how many databases you have in that web application. If you do not move all the content databases at once, your users will have no way of getting to the content because there is no way to split the URL so that it talks to two separate farms.

However, you can migrate Host Named Site Collections one at a time or all at once. This is because the Host Name is the key to getting to the content, and it is associated with the site collection and not the web application. This is great when it comes to these types of situations, where content can be very hard to move to another service or farm. Plan accordingly so that you do not get caught in a situation like this, especially when you’re managing the sizes of your content databases.

Cross-web applications (internal farm): These migrations also happen during the production use of a SharePoint farm. You need to understand how or why you need to use any other migration methods mentioned at the beginning of this section. The approach you should use for list and library migrations would be to use a tool or use the area within Central Administration to extract the site, list, or library and then move the content from one site to another. Some tweaks may be involved here, but this is something you should explore, depending on your requirements.

Migration to the cloud: This can only be achieved using a tool or a very time-consuming manual method. Of course, a tool is preferred in this scenario due to the amount of time you would have to spend going through files and uploading them to SharePoint Online. The manual process would be to download all files to a local system and then upload them back into the new farm libraries. This would mean reapplying all permissions, as well as losing version history. We can see why a tool is very important when migrating to the cloud. Most tools charge you per GB, so please budget for this!

We remember migrating from DocuShare a few years ago and the only tool available was the command line. During that process, you could export all the files, but a lot of information was lost. At the time of writing, there are lots of command-line tools, such as JSOM, that can help define these files and permissions. They are exported in a flat-file structure to help pull that information into SharePoint and apply some metadata from the old system, which in some cases is not SharePoint.

You can also migrate many different types of information using tools to SharePoint or OneDrive. Some tools give you different capabilities with different applications. Some connect to Google Docs, Dropbox, and even some other applications. We have seen some tools, such as Saketa, offer Microsoft Teams migrations as part of their toolset.

Microsoft also has a free tool that can be used to perform migrations from an on-premises environment to the cloud. We have only used this tool once and it works as it should. However, at the time, it didn’t have a lot of bells and whistles compared to what other tools offer. We believe they have made updates to this tool, so if you are looking for one, try it and see if it will work for your project.

Important note

Microsoft has its own migration tool that only migrates to the cloud. You can find more information at https://docs.microsoft.com/en-us/sharepointmigration/introducing-the-sharepoint-migration-tool.

File shares: File shares are always forgotten, but the return on investment in migrating these files to SharePoint is huge. When we leave these files shared on servers, you are requiring these files to be hosted by one or many servers with extended storage space. These servers cost you money to run and always require disk space upgrades. Instead, pull these files into SharePoint and use the databases for sharing and managing these files.

OneDrive can also be used to hold personal files and get rid of MySites and the storage comes for free with your plan. The SharePoint 2019 Server must be set up to use a hybrid connection to the cloud.

The users win as well because they gain functionality when using SharePoint, and your help desk will love you for it. With this, you have the benefit of versioning and much more control over your files. You can also place the files into segregated site collections/document libraries so that there will be no slips in data security, unlike file shares. They will also not be calling the help desk to add users to certain folders, which will cut down administration use as well. Users can also use files for workflow processes, versioning, and other collaborative means. They can include these files in links throughout sites in order to share information using the latest version of the file via Manage Copies.

Migration tools

Most tools run on any OS platform and provide a lot of options when it comes to migrating using mappings, PowerShell, and scheduling. Our experience with ShareGate has been great, but it does not offer an array of solutions as it seems to concentrate on the migration and governance space. As far as the tool goes, it is great and provides everything you need to prepare for a smooth migration process. We have not had many issues using the product, and we would say that it is first class!

ShareGate has been the tool of choice for our migrations. We really like both ShareGate and Saketa. Unlike other tools in this space, ShareGate gives you a great desktop and many functionalities you will not see in other tools. The tool is inexpensive and runs on any Windows platform. Again, pricing is tiered, so you can find a comfortable spot that fits your budget based on your data requirements.

Important note

If you are performing an on-premises migration, make sure to try out content database migrations first. This will save you money in the long run. You can then use ShareGate to push the deltas needed to finalize the updates to the user’s content when performing on-premises migrations.

Some of the features of ShareGate are as follows:

  • Auto-generates PowerShell scripts for all types of migrations with copy options

The following is a list of some of the tools that can be used for SharePoint migration:

Figure 2 - Tools that can be used for SharePoint migration

AvePoint also has a new migration tool called Fly that does not require any server resources. This really changed the game for the company’s migration offering from our perspective. It can be installed free of charge from the AvePoints enterprise application interface and is similar to portable applications such as ShareGate. The product is great! We have tested it, but not with a lot of data. Please check this tool out for your next migration and tell them we sent you!

Metalogix has been a key player in SharePoint since we can remember. We wrote up some of this information a while back with the first release of the product, and you definitely need to try this product. Although we had a bad experience with it, we have heard good things about the tool now and the features are very impressive. The downside to it is that is not very user-friendly, so there is a learning curve. Also this tool, despite its improvement, does not support PowerShell integration. We just wanted to update you on this tool to make it clear that you should try it.

Microsoft also has a free migration tool that allows you to migrate to Microsoft 365 from SharePoint On-Premises platforms, including 2016, 2013 including Foundation, and 2010 including Foundation. It also includes hooks for connecting to the network and local file shares. To find out more about this tool, please go to https://docs.microsoft.com/en-us/sharepointmigration/introducing-the-sharepoint-migration-tool.

If you are looking for a planning and assessment tool, Microsoft provides the SharePoint Migration Assessment Tool (SMAT), a command-line tool that scans SharePoint farms to identify issues before you migrate. This will give you a report so that you can go back and fix any issues before migration. This tool can also be found in the preceding link.

Managed metadata: This is a more complex migration. Some of the migration tools that are available allow you to migrate metadata using the methods provided by the tool. The issue with managed metadata is that there is a GUID for each term set, and the term within the service will not change if you migrate the data. So, if you are doing a service and content database migration, this method of migrating the databases works because there is no GUID change. If you cannot get these methods working and you must do a new installation, you will have to recreate the service and the terms within it.

When performing a content database migration, these details for the managed metadata can be brought over by backing up and restoring the managed metadata service database. When you’re creating your managed metadata service, you will use that restored backup to create the new service on the new farm. If you are using a content type hub, update that link using the following PowerShell script:

Set-SPMetadataServiceApplication -Identity "Managed metadata
Service Application" -HubURI "http://sitename/contenttypehub"

If this is not the method you would like to use to restore the service database, you can use the export-import method. When you migrate the service in this way, you will have to create the service first and then import the information using Excel. You can do that using the following PowerShell script:

Add-PSSnapin Microsoft.SharePoint.Powershell $metadataApp= Get-
SpServiceApplication | ? {$_.TypeName -eq "Managed metadata
Service"} $mmsAppId = $metadataApp.Id $mmsproxy = Get-SPServiceApplicationProxy | ?{$_.TypeName -eq "Managed metadata
Service Connection"} Export-SPMetadataWebServicePartitionData
-Identity $mmsAppId -ServiceProxy $mmsproxy -Path "C:\MMD.cab"

Important note

Make sure you do not have any duplicates in your file before moving forward. If you have duplicates, the process will error on import.

The script for importing is as follows:

Add-PSSnapin Microsoft.SharePoint.Powershell $metadataApp=
Get-SpServiceApplication | ? {$_.TypeName -eq "Managed
metadata Service"} $mmsAppId = $metadataApp.Id $mmsproxy =
Get-SPServiceApplicationProxy | ?{$_.TypeName -eq "Managed
metadata Service Connection"} Import-SPMetadataWebServicePartitionData -Identity $mmsAppId -ServiceProxy $mmsproxy -Path "//
SharedPath/MMD.cab"

Remember that if you were using managed metadata previously in your farm, then you need to migrate this service database and create a service application first before migrating any content. The content, when migrated, will find out whether this data is available so that it can populate the data in the fields that use metadata columns. In some cases, it may give you the functionality to migrate this metadata, depending on the tool you are using. When performing content database migrations, you will have to move the metadata first.

You also need to check that there is a Content Type Hub in your past farm. If so, you will want to make sure you configure that in the new SharePoint 2019 farm. You can do this by setting up a new site collection solely for this purpose and adding the URL to the Content Type Hub form field when editing your Managed Metadata Service application. You can also update the Content Type Hub using PowerShell, as shown in the preceding script, to create the service application. If you have already created it without it, use this script:

Set-SPMetadataServiceApplication -Identity <MMSAppName> -HubURI
<ContentTypeHub>

As you can see, there is a sequence you must follow when performing migrations, and scheduling is a big part of this process. Next, we will learn how and when to schedule migrations.

Scheduling migrations

At this point, everyone knows they are moving to the new SharePoint environment. Everyone is on edge to do this because, at the end of the day, most are afraid of change. So, how do we schedule migrations? Our take on this is that you start with the departments that are most eager to jump in. The reason you want to do this is that you will get the most help with the process from resources within the department. This creates a team effort that you might not get from another department, which could defeat the migration before it starts.

If you don’t have any eager departments, which we doubt you will, take one department that you have some camaraderie with or one of the hardest you will have to move due to complexity, and test their migration first. You can at least test migration with this first department and have the users test the migrated content and compare it between the source and the destination. You should be able to give them an error report from any migration type and work with them on how you will mitigate the errors when performing the final migration of the content.

In the Types of migration section, we discussed that migration tools have some scheduling automation capabilities that you can use to configure a migration and create a scheduled task when a particular content migration should run. You can also migrate using PowerShell by scheduling a command to execute on the server at a certain time, but most tools have this integrated with the product.

When running scheduled migrations, you do not have to be present. You could migrate several departments at a time if the content size is reasonable. This also depends on what your time frame is because you want to make sure your users have access to the content you’re migrating, maybe the next morning or after the weekend on Monday morning.

You will need to support the migration from time to time to monitor the progress of the migrations you are scheduling. Believe us, there are always errors. To help with such errors, you may need to assign tasks to people in your department to help monitor their progress. If you have tested your migrations and are pretty sure you have done the testing well, or if you know there is not much complexity in the site you’re migrating, then you may be able to let the scheduled migration run on its own.

If you were testing and used the most difficult customer based on the complexity of the content, then this would have given you a realistic look at the tool and any errors that come up, based on this content. Always take this on with all resources on the ground. If it is just you, get support from Microsoft, a third-party company, or from the company that owns the tools you purchased. This can be done successfully if you plan out the migration properly and identify any errors and mitigate them prior to a test migration.

Again, this may take a couple of test migrations, but the mitigation of errors that surface is the key to success and how quickly you can turn them around. Make sure you always take account of the time you may have to spend on the backend recreating content in a site. Again, if the solution is not available in the next version of the farm, you have to make sure you recreate it using a new tool and communicate the difficulty and/or discontinuation of the product to the customer.

Important note

If you have a couple or even a few web applications, NEVER EVER migrate one web application if the services on the new farm are not stable. If you are still in configuration mode and making updates to services, DO NOT migrate anyone to that unstable farm. Once you have a stable service platform and you have tested those services, you can migrate your first set of user content.

Concluding remarks

Success is the best way to build confidence in the company as you undertake migrations. Choose the right department to work with and test your migration. Review your errors and figure out how to fix them before the final migrations. This may take another test migration, but do not let the first migration fail. Do your best to be successful and have the department using the new environment brag about how easy it was to migrate so that others will be at ease and want to move immediately.

To find out how to bring on-premise and cloud collaboration features to life with Microsoft’s SharePoint Server, check out the book Implementing Microsoft SharePoint 2019 by Lewin Wanzer and Angel Wood.

We help developers build better software | Email customercare@packtpub.com for support | Twitter support 9-5 Mon-Fri

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store