I had an interesting discussion today with a large hospital client that is currently upgrading to Documentum 6.5. Like most of our Documentum customers, the client has chosen to refresh hardware, database, and disaster recovery tools as well as upgrade the operating system (from Solaris 9 to Solaris 10). Initially the client was targeting a database clone approach with an upgrade in place for roughly 1.4 million documents.
As referenced in a previous blog post, the client found the clone approach challenging to say the least – issues they encountered included:
- People – Client needed 5 people to “figure it out” including a DBA, Unix Admin, Storage Admin, Documentum Administrator and Project Manager.
- Downtime – Team estimated that “hopefully” the Documentum system would only be down for 2-3 days.
- Effort – The client thought the 5 person team would take at least 2-3 weeks to “figure it all out” to be able to complete the migration accurately during the 2-3 days of downtime.
While talking to TSG about OpenMigrate, the client decided to try a 2 day proof of concept to use OpenMigrate for a migration approach rather than the clone and in-place upgrade. The proof of concept was successful in handling 90% of the client’s needs. The client IT resources were able to successfully address 100% on their own after the POC with minimal phone support from TSG. Benefits of the migration approach included:
- Reduced Downtime – because the migration approach allowed for “chunks” of archive documents to be moved without bringing the current system down, downtime was reduced from 2-3 days to 2-3 hours.
- Reduced Risk – with less downtime plus the ability to check the integrity of the new system with the archive documents the risk of failure was substantially reduced. Also, the fallback approach was to leverage the old hardware and migrate any newly scanned content. With the clone approach, fallback was much more complex and might involve rescanning content.
- Reduced Development Environment Size – With the clone approach, the client had previously maintained a development environment that was the same size as production. By using OpenMigrate going forward, the client was able to leverage a significantly smaller development environment and only migrate subsets of the production docbase.
Hello,
I used open migrate to migrate 1 million documents and it took me a month or so to migrate documents . I created chunks od data to migrate. Migration was smooth and effective. No issues. But now I have a task at hand and have to migrate over 2 TB of data . I have estimated around 7 months to do it using oprn migrate. Client does not want to go beyond two months. So that made me think an alternative to open migrate that is more faster like cloning the database, copying the file stores and installing contnet server using same config, aek.key etc .
I’m glad to hear you were able to achieve success with OpenMigrate! We’ve found several cases where folks have used OM without our assistance, but yours is by far the largest migration we’ve heard of.
In our experience though, OpenMigrate itself adds negligible overhead in large migrations. If there’s slow disk, slow network, slow database or slow TBOs/Lifecycle code, any migration product would have the same performance limitations, right? In fact, the multithreading offered by OM can often help offset some infrastructure drawbacks.
In one of our most recent migration, File System/Database to Documentum running on a very old HP-UX machine, our average performance has been 10 docs per second, or just under 1M documents per day. Assuming an average doc size of 150K, that would be (roughly) just under 2 months of constant ingestion for 2+ TB, assuming a similar throughput. It looks like your throughput was less than ours, leading to the 7 month estimate. Or were there other factors driving out the end date?
But you raise a good point: With a very large migration like this, the approach you lay out, a basic database clone, is often the preferable option. Not every system migration calls for an actual “migration”; sometimes a clone is the lowest-risk, fastest way to go, especially if you want to simply move everything. TSG has performed a number of these types of migrations. I’m not familiar with any product that performs this type of move; instead it’s a fairly manual and technical process, relying on a DBA to do most of the “heavy lifting”.
A colleague and I have looked at what it would take to modify OM to do something similar to what you suggest, but possibly even better: migrate a 1-byte file for each document, then “swap in” the appropriate content files via straight filesystem copy. This way we could migrate a slice of the repository, do reorganization and cleanup, etc., very quickly; but instead of streaming content into Documentum (slow), we’d “poke” the content in there post-save. We assume a filesystem-level transfer would be faster than a stream.
Yes that’s a good idea to save 1 byte files as place holders. This will allow us to migrate chunks of data without worrying about the data size . swap-in will still consume time if it happens on every post-save of a document even if its a file copy; though it will be far better than streaming content. I was thinking of a event based post save i.e. asynchronous. When OM migrates the document it posts an even that has the source file location and the target file location. Then Have a different process pick up that event and migrate the files using simple copy. But if somthing goes wrong with file copy we should have a mechanism to report it back.