Next-Level Web Hosting Migration Tools at Work

Takeaway: From time to time, a web development agency—any agency for that matter—finds itself faced with a huge ask for which the resources, the tools, and the know-how appear to exceed the realm of what’s realistic. We migrated UC Riverside's websites from their current hosting platform using tools we had to build just to keep up with the demand and the deadline.

READ THE UC RIVERSIDE CASE STUDY

It’s tempting to decline and return to the safe zone of the known, but stepping up to help a client who is in a jam, and committing to meeting a deadline based on determination and a drive to innovate can serve as an opportunity to discover what the team is made of.

Once a successful outcome is achieved, agency capabilities and offerings are catapulted to the next level. 

That was the case for Promet Source, when in 2022, our partner Pantheon approached us with an urgent request to migrate 525 University of California, Riverside sites from an existing hosting relationship. 

Our solution?

Build a net new application that uses terminus commands to perform web hosting migration tasks.

Expediting migration via automation

Steps that we automated for purposes of expediting migration processes included: 

1. Backup of the sites before migration;
2. Receipt of an alert when the site archives were backed up and ready for the next step;
3. Extraction of needed data from the archives, while minimizing required space on the hard drive. Some of the archived sites were eight to nine gigabytes in size;
4. Spin up of a new site on Pantheon hosting;
5. Push files and the database to the new site;
6. Push all environments (dev, test, and life) to the new site;
7. Run cleanup on the new Pantheon site;
8. Uninstall modules;
9. Update configuration;
10. Deploy code updates; and
11. Run automated visual regression tests so that we knew whether we were successful or if anything failed along the way.

How our team performed the web hosting migration

1. We created a PHP application that runs on an Apache web server, which enabled us to initiate backups site by site.
2. When backups were ready, the system informed the PMT endpoint with the associated URL, and saved the files on a server from which the files were readied for migration.

The new tool allowed for the creation of multiple functions within the UI that functioned as a bulk deployment tool that utilized terminus commands as if they were API endpoints for managing the many tasks associated with the migration of a site.

Leveraging the functionality provided by the terminus command line tool, in conjunction with Pantheon custom upstreams, enabled full automation of the site migration onto the Pantheon hosting platform. 

Automated hosting migration for up to 20 sites

We added group IDs to the site’s database so that we could target specific groups, which enabled us to run migrations for 15-20 sites at the same time.

This process also enabled efficient sequencing. As we received a backup on one group, we immediately initiated deployment on the next. 

We set it up so that the process could run itself, and was executed by Cron jobs.

Monitoring and troubleshooting using a high-functioning interface

Results were monitored from a front-end UI, which refreshed every 60 seconds and reported on multiple states.

Visual regression testing during deployment allowed for pixel-by-pixel screenshot comparisons as sites were migrated.

This provided immediate insight and flagged failures that required attention—ensuring that issues were not lost or overlooked within the sheer volume of sites being migrated simultaneously.

We were able to initiate and monitor the status of every migration from a single interface and dashboard.

The interface provided a color-coded, comprehensive list of sites to indicate current states in all environments, with red indicating the detection of an error. 

Color bars on the dashboard functioned as clickable links from which we were able to access the sites in order to individually address error indicators. 

Visual Regression Testing proved key to the success of the project

VRT enabled us to move forward at a rapid clip, with the confidence that errors at any point in the process would be detected quickly and addressed immediately without ever becoming buried in the flow and requiring complex and time-consuming troubleshooting.

The system allowed for approximately 15 to 20 site migrations an hour, depending on size.

When sites were between 400 and 500 megabytes, the process ran quickly. Sites that exceeded four gigabytes slowed things down.

The ability to effect site migrations in bulk—sometimes up to 25 at a time—allowed for unprecedented levels of efficiency and accuracy.

The process drove home the power of our Pantheon partnership and resulted in a tool that is uniquely positioned to fuel successful migrations in the future.

READ THE UC RIVERSIDE CASE STUDY