I'm currently working on a data migration tool for more than 10k users to transfer their data from Sharepoint(on-premise) to a Nextcloud. The average quota for each user is 3GB, but some users have over 100GB of data. To ensure a smooth data transfer, I've designed an algorithm that fetches the list of files and folders for each directory and uploads them to the new platform, exploring other subdirectories in the same way. In the test environment, I'm able to transfer around 1GB of data with ease, and the algorithm can handle any problems during the transfer, such as network errors or host downtime. However, transferring data with multiple user permissions, such as read-only or full control, adds complexity to the process. Now I'm a bit confused about whether I should transfer all the data first and then fetch the file shares along with the permissions or do it after each directory transfer. What do you think? Is there a simpler yet effective way to do this? I'd love to hear your opinions! TC: 🥜 #tech #microsoft #migration #faang
You will probably get better answers in Nexcloud forums. If there's a way to export all files from Sharepoint, you can just copy them into Nextcloud's directory with all users' directories. Then you can run Nexcloud file scan command for Nextcloud to find all the files. This way you don't have to transfer one file at a time.