Mastering Migrations–Faster, Easier, and Safer Ways to Move Your Site From A to B

Migrations can be tricky! We all know that feeling of frustration (or utter dread) when we see one fail, but we also know the sense of relief when a migration succeeds. With all the complexities and nuances of migrations, how can we drive up our success rate and win back our time to focus on the work we really want to be doing?

Whether you need to copy an established project to your local machine or deploy a handful of incremental updates to production, follow along as we walk through how you can speed up, simplify, and de-risk your migrations.

Speakers:

• Kevin Hoffman, Senior Product Manager at WP Engine
• Austin Wendt, Senior Product Manager at WP Engine

Session Slides:

Transcript: 

AUSTIN WENDT: Welcome, everyone, and thank you for joining. We’re excited to have you. And welcome to the DE{CODE} Conference.

My name is Austin Wendt, I’m a senior product manager here at WP Engine, working on building our Local product. And my colleague, Kevin, and I, who you will meet here in a minute, are excited to talk to you today about building smarter– particularly in terms of mastering your migrations. So we’ll cover faster, easier, and safer ways to move your site from point A to point B so you feel confident in those development workflows, whether you’re bringing a site down to a local, safe development environment, or you’re ready to push that site live over time.

I’ll cover a quick agenda before we dive in. So what we’ll cover today is we’ll introduce the three types of migrations that we like to think of here at WP Engine as we talk about moving code. We’ll define our ideal migration workflow, and kind of walk you through that over the course of this presentation, of different ways to move your code. We’ll cover exporting your existing site, and bringing that down– importing into a local development environment.

We’ll talk about performing a first-time deployment– so when you’re taking your site live for the first time, what that looks like, and a couple of ways to accomplish that And then synchronizing those two environments over time. So let’s dive right into it.

The three types of migrations that we think of– there’s three main options that a user could be attempting to accomplish. The first is remote-to-local. So if you’ve got a site hosted somewhere on the web already, and you want to bring it in to a Local, maybe lowercase l, local environment, it’s useful when you’re maybe beginning work on a client’s existing site. So you’ve either inherited a new client or a client’s asking you to make changes and bringing that down somewhere safe, so that you can troubleshoot that in a low-risk environment.

Also very useful when you’re just trying to pull down the latest database changes, so that you can make sure that your remote environment and your production– or excuse me, development environment– are as closely matched as possible. The second is Local to remote. So when you’re going back from your personal machine up to the hosted server somewhere– so you’re either deploying a full site for the first time, or you’ve made some code changes and you’re pushing those changes up, call up for themes, or for plugins, whatever that might be that you’d like to see reflected live on your site.

And the second– excuse me, the third is remote-to-remote. We won’t dive too much into this one today, but it is something that is possible with the tools you’ll learn about. You’d use this typically when you’re either switching hosting providers– so moving from host A to host B, or potentially when you’re moving between dev, staging, and production environments, wherever your site may be hosted.

So with that, I will hand it off to Kevin to introduce himself and get us started on what that ideal migration flow looks like. Kevin, take it away.

KEVIN HOFFMAN: Hey, thanks, Austin. So my name is Kevin Hoffman, I’m the product manager of WP Migrate. I want to kick things off today by starting with a game plan of the type of migrations we’re going to jump into. So any time you’re going from a remote environment into your local machine, and back up to a remote host, that can be a daunting task. But we want you to leave this presentation with a game plan of solutions so that you can make these migrations yourself with confidence.

And to start, we want to get the existing site out of the old host. So that’s going to include a full site export using WP Migrate. We’re then going to move into Local, where we can do our local development changes, and then deploy that site back up to our new host.

So to get things started, I’m going to move over into a full site export flow using WP Migrate. You might be asking yourself, why do we use a full site export in this situation? Why not push or pull directly between two environments? Well, there are a few reasons for that.

To get started, I’m going to use the Pro version of WP Migrate, but you can also follow along with WP Migrate Lite, which is the free version of our plugin on the WordPress plugin directory.

The four main reasons why we’re using a full site export in this situation is firstly because it’s a one-way migration. We want to get off of the remote host, and we have no plans of going back. There’s also no existing local install for us to move the site into. If there was, we could use maybe a push migration, or a pull migration to get the site down into the Local machine. But because there’s no existing install, a drag and drop import to local makes the most sense.

And finally, by doing a full site export, we also get a free backup. The entire site will be encapsulated in one bundled zip file, which is a great backup before you make any future changes.

So to get things started, let’s jump in to WP Migrate and see how this works.

So when you first open WP Migrate, you’ll have six actions in front of you. Because we want to get the site out of the remote host as quickly as possible, we’re going to choose the export action. Opening the export profile gives us the ability to configure the database options, as well as media, themes, plugins, and WordPress core files.

Let’s go ahead and start with the database configuration. Now, if I wanted to, I could exclude certain tables, or post types from this migration. But for now, I want to go with the default configuration and just get the entire site out of the remote host. I do want to mention standard find and replace fields, like the URL, or the path of the local WordPress install, on the site that we’re exporting.

Now, if you were doing a manual migration, you might want to move these values over and edit them to match the destination. However, because we’re using Local, it’s smart enough to take care of this find and replace for us, so we don’t actually have to fill out these optional fields. We can leave them blank and move on.

Next up is the custom find and replace. This is the ability to search for any strings in my WordPress database, or in my content across my site. For example, maybe I have an old company name, and I want to replace it with my new company name, I can do that through these custom find and replace fields. And I can add additional rows, as needed.

So that takes care of the database. Let’s move into our media uploads. Now, because I’m moving the whole site, I want to select Export all Media Uploads. But I do want to exclude some files like logs, backups, and caches, which could bloat the export.

As we move on into theme files, I want to include all of my themes. Not this time, just the active themes, because I’m only concerned with the themes that are actively affecting the live site.

Similarly, with plugins– I only want to export my active plugins. And with WordPress core files, I do want to go ahead and include those, because I want to make sure that my WordPress core matches the exact version of the site that I’m exporting from.

With the profile fully configured, I can now kick off the export, which is going to quickly step through my database tables, media uploads, themes, plugins, and WordPress core files.

At this point, the database and all the files within the site are being bundled into one convenient zip file. So in just 18 seconds, the entire site was zipped up.

I am now ready to move into Local. And before I do that, I want to take a quick peek inside the zip file to see what’s there. You can see that I have a files directory. This includes all the WordPress files, including my WP content, plugins, themes, and uploads. And I also have the database dump.

And one other file, which is really important and unique to WP Migrate– the WP Migrate export JSON file includes key information about the site that was exported, such as the PHP version, and MySQL version, so that when Local takes care of the import, it can match that remote environment as closely as possible.

So with that, you’re ready to import into Local. And I’ll send it back over to Austin.

AUSTIN WENDT: Awesome, thanks, Kevin. Yeah, I’m excited to cover, like Kevin mentioned, how we can get that zip file imported into Local and ready to start building. But first, I want to make sure to introduce what Local is. If you’re not familiar, Local is the number one WordPress development tool, built by the humans here at WP Engine, and we’re very excited to share and offer with the community for free.

So it’s a free dev tool. If you haven’t heard of it, please check it out, localWP.com, we’d love to have you using the product. But today, we’re going to be using Local to facilitate this workflow.

And why Local? Similar to any environment that is specific to your machine, it’s very low risk. And as Kevin said, what Local’s going to try to do when you import that export from WP Migrate is we’re going to closely mimic the production environment. So as close as we can, the WordPress version, the PHP version, the database, your local machine should mimic what’s going on in production, so that if you’re troubleshooting or trying to see what’s going wrong, Local should be able to tell you, and be as close as you can to what is going on, on your hosted environment.

Another key benefit of doing this with Local is the workflow that Kevin just mentioned is host agnostic. So regardless of where you’re hosting, whether that be with Flywheel or WP Engine, you’re going to be able to export that site and drop into local very quickly and easily.

So I will kick it over to a demo and show you what this looks like inside the local UI.

Awesome, so I’ve already done a WP Migrate, and I’ve saved that zip to my desktop. And when I go to create a site in Local, there’ll be this new drag zone that indicates you can drag and drop zip files here. What’s also nice about Local is I can do this from any screen within the UI. So if I drag and drop that zip file over Local, it’s going to suggest the site name for me from that WP migrate export JSON file that Kevin mentioned.

It’s preselected my PHP, my web server, my database. And then, I click Create, and Local takes care of the rest. So Local is actively unzipping that zip file, importing all of those WordPress files, and getting that site set up on my machine in a state that is as close to production as we can get it.

While this is spinning up, it’s going to ask for permission to update my host file, which I’ll enter my password and allow it to do. But then, Local starts adding WordPress and you’re good to go.

While this finishes, something I’ll highlight real quick is you can see on the left-hand side– the ability to group your sites is new in Local in the last couple of weeks. So I’ll drag and drop Garrett’s Grocery up into my DE{CODE} demo section– which is a nice way that I would encourage you to check out to organize your sites, maybe group them by client, or by version, connected to WP Engine or not, whatever works best for you. So give that a try.

But Local is finishing up here, it’s changing that site domain. And what that’s going to do is going to configure it on my machine so that it’s available, as you can see here, at mysite.local. If I click, Open Site, here’s Garrett’s Grocery. So I’ve effectively gone from my hosted environment, and drag and dropped it into Local, and have it running on my machine in less than two minutes, which is awesome.

So with this example, what we’ve shown is being able to take it from your old host, regardless of wherever that might be on the internet, and with a combination of WP Migrate full-site export, getting that into Local and mimicking your production environment in less than a couple of minutes.

Now, the question is, once I’ve got it in Local, I’m ready to start making changes. How do I take it back out and get it live on the internet again? For taking your site from Local and getting it back up to your host, we’ll be using Local Connect to deploy to WP Engine, or to Flywheel. Both from a full site migration as well as partial migrations.

But why would you want to do a full site deployment? The first-time deployment of an entire site to your host is a good example here. So maybe the site doesn’t exist at all yet, or maybe it’s just a templated site up on the host. If you wish to include entire themes or plugin changes, or maybe you’re just ready to completely overwrite the current site that’s on the host today. So maybe it already has content, but what’s up there now just isn’t productive or conducive anymore, and you’re ready to just wipe it, you would use a full site deployment.

So using Local, that’s fairly easy to accomplish. And I will show you a demo of what that looks like. So I’ve got Garrett’s Grocery here, and I’ve made a series of changes to the website that I am ready to push up. Now, Local has the concept of Local Connect, as I mentioned– there’s this cloud icon on the left-hand side, for Connect. There’s also a Connect to Host in the bottom right, that will allow me to connect either WP Engine or to Flywheel.

For today, I’ll accomplish this by going to the Connect tab, and clicking Connect to a Platform. I’ll log into my WP Engine account, which I’ve spared you watching me log in. What you can see happen is Local Connect pulls in all of the sites that I have access to on WP Engine. Now, what I’ll do is I’ll go back to Garrett’s Grocery in my overview. In the bottom right, I’ll select Connect to WP Engine.

Local’s going to check that that site is compatible with WP Engine’s infrastructure. So using up-to-date WordPress and PHP, and then I can click Push.

Push is going to allow me to choose the sight that I want to overwrite on WP Engine. It will allow me to choose the environment. So I’ll choose the Austin Wendt site, and I’ll choose Production. And what you’ll see happening on the right-hand side of the screen is Local is determining the file list.

What that means is local is essentially running a diff between what is on my machine and what exists up on the server, and providing that to me so that I can really see and understand the changes that I’m about to make. So because this is a full site deployment, you can see nothing’s happening on my local environment, but I’m going to overwrite everything that’s on the production, as you can see with those red X’s on the right-hand side.

So then I click, Push to WP Engine, and Local starts to take care of the rest. This entire video is about four minutes– I’ll spare you watching that with me, as I sit here. What’s happening is Local is packing up those files. It begins to upload those files to WP Engine. And starting to analyze, like I said, the differences between what’s on my machine, and what’s on the WP Engine server.

This same workflow would apply to Flywheel, as well, if you host there. We’ll follow the same flow entering that file difference between your machine and the server.

So now, Local begins to pack up the database. It’s pushing that to WP Engine, as well. So it’s dropping all of the existing tables that exist on the remote server and replacing those with what’s coming from my machine.

As part of that database transition, it’s going to look at the site domain and perform a search and replace for me, as you can see now. So that all of my links and URLs that are stored in my database are going to be updated, along with table prefixes, to work correctly in the production environment.

So it’ll update those table prefixes for me. And just like that, my site has been pushed to WP Engine.

So starting this again, Garrett’s Grocery is still on my machine. But also, if I go to the Connect tab, I can see that Austin Wendt site that I pushed to on the right, it says it’s connected to Garrett’s Grocery. And if I were to click that name of the site, Austin Wendt, it’ll open in a browser for me to show me that new content live on the internet.

So now that we understand how to use Local to accomplish a full site deployment, I’d like to cover how we can use Local to also synchronize environments using a feature we know as MagicSync.

So MagicSync is another word for incremental migrations. So moving just small bits and pieces of code between your local environment and the remote server. And why would you want to do this?

So maybe you don’t want to replace the entire site. You’ve only made smaller partial changes to an existing site that you’re ready to push live. What’s also nice about Local– a Local will allow you to, as I mentioned, that diff feature, pick and choose the files that you want to include, or even exclude. So a big common use case here is maybe I’ve done a lot of things on my machine, but I want to exclude pushing and pulling media, because that’s a very heavy and intensive portion of my site. I can just unselect media.

So I we’ll dive into a demo here of what MagicSync looks like. So again, here I’ve got Garrett’s Grocery– say I’ve made another smaller set of changes this time, and I’m ready to see that reflected live on WP Engine. Same workflow here– in the bottom right of my screen, I go back to push to WP Engine. It’s already preselected the Austin Wendt site for me, and the environment, remembering from the last time I did it.

And this time, it’ll be shorter– it’s determining, again, that differential between what’s on my machine and what’s on WP Engine servers. So it’s going to come back here, and there’s a smaller set of changes that it detects have been made to the site. I can unselect all of the file changes that I want. I can select just my WP content folder.

Or maybe in this case, let’s say I only want to push my database. So I can check the database box and hit Push. So now what’s happening is the same workflow we witnessed before, except Local’s actually not pushing any files up to WP Engine. It’s only replacing the database changes that I’ve made on my machine with the database that was currently on the WP Engine server.

So similar workflow here– we’ll actually watch this one process through, because it doesn’t take quite as long. Because the differential is smaller. So we’ll upload the database to WP Engine. Local will, again, go forward for me and do that search and replace. So it’ll detect if table prefixes have changed, URLs that were different on my machine needing to be reflected on the remote host.

It will make those updates for me. And in just under roughly about a minute, the site changes that I have made on my machine will be pushed up to WP Engine, and ready to be consumed by whether it’s colleagues and peers wanting to just review the work that I’ve done, maybe I’ve pushed to a development environment or if it’s live on the web in production, and ready for my clients or customers– or just consumers to view on the web.

So just like that, the site has been pushed to WP Engine and if I went back out to the browser, you would see the site updated and reflected there. So now that we understand how to use Local to accomplish incremental migrations, I’d like to hand it back over to Kevin to show you another way of accomplishing this using the WP Migrate tool.

KEVIN HOFFMAN: Hey, thanks, Austin. I appreciate you running us through the Local to WP Engine workflow but we know that you don’t always have control over your hosting provider. So the next workflow is going to show you how to migrate between any two WordPress environments. In this case, going from Local up to any other web host.

To do that, we’re going to use a concept called pushing and pulling, using WP Migrate. Now, why would you do a push or a pull? Now in contrast to a full site export, this is a two-way migration. That means both sites already exist, and it requires a little bit more upfront investment for a longer term payoff.

So once you have this setup complete, you’ll be ready to go to handle your incremental migrations over time, and keep two environments in sync on an ongoing basis.

So let’s see what that looks like. So let’s say your site is ready to be deployed to your remote host. You have a number of posts, and a number of images in your media library. We’re going to take this content and move it into a new site that currently has zero posts, and no images in the media library.

The different approach we’re going to take here is to use a push migration. And the first thing it asked me for is connection info from the remote site. So I can switch over to the remote site, and in my settings tab, copy the connection info directly to my clipboard. I also want to enable push migration, so that I can accept these push requests from the local site.

By pasting that info into the connection info box, I’m now connected to the remote site, and I’m ready to configure my database options. The big difference you’ll notice here, compared to our export workflow, is that both of the Find and Replace sides of the URL and path are completely filled out for us. And that’s because WP Migrate is on both sites, and has access to that information, and can take care of it for us without us having to enter anything in order to kick off the migration.

I’m not going to do a custom find and replace, but I am going to include all of my media uploads from the library, as well as all of my themes and my plugins. Now, one unique feature you’ll notice here when selecting my plugins is that it shows me what the state of that plugin is on the remote site. Now, in this case, there are no plugins there, so all of these plugins will be added for the first time, and the current version number is indicated when you hover over that icon.

I’m going to go ahead and save this profile for future use, and I’m going to name it Push Full Site. So any time I need to push a full site to that remote location, I can simply revisit this profile and run it.

When I run the profile, you’ll see it once again step through tables, media uploads, themes, plugins and you’ll get some information about the size of the requests as the migration is going on.

Once the migration completes, you can go ahead and close the modal, and your two environments are now synchronized.

At this point, you may want to revisit your profile screen to check out how that saved profile is available for you to click back into, if you ever need to run it again.

So that’s a full site deployment with a save profile in WP Migrate. But you might be wondering, what about deploying incremental changes? So like Austin showed you, by using MagicSync in Local, this is another way to do it with WP Migrate. So I’m going to create another push profile, enter the same connection info, but this time, when I select my media uploads, I’m only going to push new and updated media uploads.

This means, the first time the migration runs, it’s going to include everything. But every migration thereafter, it’s only going to include the media files that have changed.

This is an excellent workflow any time you’re pushing content and media files without having to worry about themes or plugins. So I’m going to now save this profile, and I’ll name it Push Content and Media.

So this leaves me now with two migration profiles I can use for two different purposes. They’re saved on my profile screen, and they’re available any time I need to jump back into them. I could even set up a pull profile to then pull the production data down into this local site, and keep the two environments in sync in both directions.

So that concludes our workflows using local and WP Migrate to move from remote to local, and back to remote.

So as you can see, now our game plan is complete, we have solutions for moving out of the remote site using a full site export from WP Migrate, drag and drop importing that into local, and then pushing up to WP Engine or Flywheel, or any other host. So this is just the tip of the iceberg when it comes to migration solutions and what’s possible when you use WP Migrate and Local together.

So we hope that gives you a game plan the next time you need to run your own migrations. Look forward to hearing from you on our Twitter accounts for WP Migrate and Local, and we hope you enjoy the rest of DE{CODE]. Thanks for joining us.