Want to eliminate platform migration headaches? Reduce cost, effort, and lost time? You’re in the right place. I want to share some tips to help you simplify your migration activities.
I have been performing file-based migrations for well over 20 years, primarily with EMC Technology. The majority of migrations have been SMB-based including server-to-server, server-to-NAS, and NAS-to-NAS migrations.
Some of the tools that I have leveraged over the years include:
- ROBOCOPY: This Microsoft utility, originally part of the Windows NT 4.0 Resource Kit, has been around since 1997.
- EMCOPY: This EMC utility is used primarily to copy to a Celerra based file system; however, the other tools in the suite comprised of EMCopy, Sharedup, and LGDUP have been used on several occasions, even in conjunction with Robocopy.
- RSYNC: This UNIX utility is used for NFS migrations.
As technology advanced, storage vendors provided a way to perform these migrations using internal tools. Some of the tools were developed for disaster recovery while others were developed explicitly for data migration. Two examples include:
- VNX Replicator: This tool can replicate from Celerra/VNX to another Celerra/VNX array.
- Isi_vol_copy: The isi_vol_copy utility from Isilon uses NDMP streams to copy data from a NetApp array to the Isilon array. Eventually EMC added isi_vol_copy_vnx that allowed NDMP- based copy from VNX to the Isilon.
Considerations: One of the major considerations that we face when performing file-based migrations is which tool is the best for the job. The answer to this question is not easy because it may take multiple tools to complete the job. Another consideration is the ability to streamline the data migration process.
Among the better known file migration tools that can address these considerations: Datadobi DobiMiner. For the last five years, I have recognized Datadobi for their CAS-to-NAS migration methodologies and their NAS-to-NAS migration tool, which is always evolving. DobiMiner helps streamline file-based data migrations, simplifies the entire end-to-end process, and tackles all of the following additional challenges:
Challenge 1: Bandwidth Throttling
Customers may need to throttle the bandwidth that is consumed by the migration. This may limit the amount of concurrent sessions or the actual time required to stop the migration if necessary.
DobiMiner can schedule bandwidth based on hours of operation per file server. This feature limits bandwidth during normal business hours so that migrations continue to run with reduced bandwidth and then automatically increases it during the next window. Imagine the flexibility to change schedules based on file server and managing multiple bandwidth schedules over the duration of the migration phase. It also includes file server scans to refresh the data.
Challenge 2: Detailed Reporting
Customers often ask to see a concise daily status report on all migration jobs. This includes successes, failures and the details surrounding both cases. Other tools do have log files associated with them, but it will take some tweaking to get an exact report. The ability to email a report, if supported, will typically require additional configuration effort.
DobiMiner generates reports on-demand. In addition to the source and target destination, the reports show the number of files, directories, symbolic links and errors. In the event of an error, the reports allow you to drill down to analyze the cause. Migration reports can be emailed daily, weekly, or monthly.
Challenge 3: Job Scheduling
Job scheduling needs to be done from the Windows scheduler at each server that is performing the migration job. The job schedules can overlap if multiple proxy servers are used for the migrations.
DobiMiner can schedule steady state incremental copies. Each individual migration can be scheduled to run based on the migration effort. For example, the migration can start with daily incremental copies and then be changed to hourly as the migration cutover window arrives.
Challenge 4: Estimating Switchover Time
This manual process includes reviewing the logs from each incremental run to estimate the time it takes to perform the incremental copy.
DobiMiner allows you to create migration windows based on the cutover time. DobiMiner determines if the final migration will fit within the specified time window. A dry run can also be performed to simulate the final copy.
Challenge 5: Creation of Migration Jobs
Job creation can be scripted, but needs to be imported on each server that will be performing the jobs, and then scheduled per server. For Windows, a batch file needs to be created, then scheduled with the Windows scheduler.
DobiMiner can perform a bulk import of not only the file servers, but also the migration pairs from a template. After the template is created, there is an import function that will create the migration jobs at the DobiMiner software level.
Challenge 6: Incremental Copies
All the tools mentioned have a way to perform incremental copies. Some need to perform a file compare and could take hours to find the one file that needs to be copied. Others will not synchronize the directories as per NDMP-based protocol.
DobiMiner performs a scan of the directory faster than the other tools mentioned. The scans are performed on a scheduled basis and information is stored in DobiMiner.
Because it scans and performs incremental copies faster than the other tools mentioned in this article, it helps speed up the migration saving you precious time.
Challenge 7: Mix of SMB and NFS Clients
If a customer wants to copy NFS files as well as SMB files, then two tools will be required, such as RoboCopy and Rsync.
DobiMiner migrates both NFS and SMB files in a single pane-of-glass. It supports the migration of SMB, NFS, and even mixed-mode protocols.
Conclusion:
It’s no surprise that DobiMiner has been named as the tool of choice for file-based migrations on EMC Unity and other platforms. In my experience, DobiMiner has been a welcome addition to the file migration process and continues to impress me. DobiMiner addresses all of the current challenges presented in this article and offers technological advantages over other legacy tools.