Develop, test and deploy your WordPress website

Just as many others have, we’ve made the choice of WordPress for our main website streamdata.io.

We first tried to use Jekyll, as it was lighter and easier to integrate into a software development process. But we reverted to a WordPress based website to allow non-technical members of the team to edit the website content, and also to benefit of the huge WordPress ecosystem, providing lots of themes, plugins and widgets.

WordPress is a powerful tool to create nice, easy-to-edit websites. A static part of the site (for instance theme and plugins) is stored as files, and the dynamic part (pages, posts, user management) are stored in the database. Once our WordPress website is live, you might start wondering how to develop, test and deploy changes. Here are a few questions that we’ve asked ourselves.

• How to back up my website and make sure I’ll be able to restore it if ever my server would crash?
• How to make changes on my website, working in a sandbox isolated from the production environment?
• How do I smoothly report those changes in the production environment after testing?
• How can I version the modifications made on my website in order to be able to revert back to a previous revision if something goes wrong?

We can find a lot of blog posts or support on the web, but none were that satisfying to us, so here is what we decided to set up.

1- Version WordPress files with GitHub

A WordPress website has a large part of file-based content, so let’s apply the standard version control that we apply for every other technical project!

First, we have created a private repository on GitHub (whereas this contains no secret; we don’t think the content of our website is worth sharing with the community). As our website was already up and running at that time, we created an empty repo, and used git remote add origin command on the website file system to commit and push the current state.
We have used gitignore.io to create the appropriate .gitignore file. Let’s see the result:

# Created by https://www.gitignore.io

### WordPress ###
*.log
.htaccess
sitemap.xml
sitemap.xml.gz
wp-config.php
wp-content/advanced-cache.php
wp-content/backup-db/
wp-content/backups/
wp-content/blogs.dir/
wp-content/cache/
wp-content/upgrade/
wp-content/uploads/
wp-content/wp-cache-config.php

First of all, we can see that wp-config is excluded from version control. We agree with that as we try as much as possible to not keep under version control files containing passwords (even in a private repo), and the database connection info will much likely be different between a development environment and a production environment.

Then, we can see that some directories that may contain temporary data are also excluded (advanced-cache, cache, backup, upgrade,…).

Finally, we can see that the uploads directory is also excluded from source control. There may be very good reasons for that, as explained in Steve Grunwell’s blog:

1. As soon as a client starts uploading media on production, your repo will be out of date
2. By default, WordPress generates multiple copies of every uploaded image (the thumbnail, medium, large). If all of those images are in your repo, everyone who clones it will be downloading every version of every image that’s been uploaded. That means more bandwidth, larger repos, and no real benefit.
3. Remember version control! == site backup. Just like anything else, repos can be deleted, corrupted or generally messed up.

Although all the reasons mentioned here are exact, we decided not to apply the advice. In our case, we need to be able to set up an exact copy of the production site on a local development environment and make sure that the resources can be loaded in order to render the website exactly as the production version. Moreover, if our images are not versioned in Git, nor stored in the database that we will backup, then how do we guarantee that we’ll be able to restore everything if ever we had to restore from a backup? For all those reasons, we then decided to remove wp-content/uploads/ from our gitignore file.

2 – Create database backup

In order to restore the full live website on a development environment, you not only need the file structure, but you also need to copy the content of the database to get all pages, articles, and all the plugins data. But dumping the database on the production server, and restoring it on the local environment is not enough: you need to perform some search and replace to change the values that are hardcoded in the DB, such as the website URL and the WordPress installation directory.

To make the backup simpler, we use WP Migrate DB plugin. It allows you to perform an SQL dump, and the search and replace it in one single step.

To make the backup procedure even simpler, you can save the migration profile. In our case, we have defined two profiles: one to migrate DB from production to a local environment for development, and the other one to create a backup of the database that could be restored in production in case of failure.

In the development backup profile, we took advantage of the search and replace feature to remove all Google Analytics tracking code from the local copy, so we stop reporting what happens on the local website to Google Analytics.

3 – Prepare a local environment for development

Once the backup of the database has been done, you are ready to copy the website on your local development environment. First of all, you need to install all the prerequisites for WordPress on your machine. You can find the official WordPress list of requirements on the WordPress website, or follow one of the tutorials that can be found on the web, like this site for instance.

If you are working on Mac, we advise you to install MAMP and to configure the ports to standard values (80 for Apache, and 3306 for MySQL).

After installing the prerequisites, clone the Git repository to get the file structure. Note that this directory path must match the replace string used in WP Migrate DB. To simplify the process, you can also decide to install it on the same root path as on the production server. Then, change permissions on the WordPress directory to give full access to www-data user and group.

Then, create a wp-config.php file in the WordPress directory containing your local database access details.

Finally, import the database dump that has been performed on the production website. To do this import, you can either use phpMyAdmin, or type the following command line:

mysql -uroot -p wordpress < wordpress-migrate.sql

That’s it! You’re all set to perform any modifications on your local copy of the website, without fearing to break anything on your production environment!

4 – Report changes to production

Once you have performed all the required modifications on your local copy of the website, and test that everything is ok, we need to report those changes to the production website. Most of the tutorials or articles we have found on the web explained the first three steps that we just went through. But none of them went through how to report the changes made locally to the production.

First of all, you will need to make sure that no modifications were applied to the production environment while you were working locally. To do so, simply run a git pull and do a new extract of your production DB to synchronize your local environment, and run new tests on your local copy to make sure that everything is ok.

Then, you can start pushing changes back. As the files are versioned with Git, applying the modifications is straight forward: use git commit and git push on your local environment, and finally, git pull back on the production environment.

The tricky part is about database modifications. We did not want to use WP Migrate DB in this case, because we found it too risky to overwrite the whole content of the production database with a development copy.

So, what we do, is we dump the database on the local development copy, using WP Migrate DB, but without specifying any substitution. You can then compare the exported SQL file with the file used to import the content of the DB with a diff tool, like Meld for instance if you are working on Ubuntu. If you find any modifications, you can report them on the production environment, either by doing the same modification via the WordPress dashboard on the production website or, if you feel lucky, apply updates to big changes to the website, you could on the database via the command line (this is not what we recommend!).

Note that if you have performed enormous changes to the website, you could test applying changes to a staging environment before really pushing to production. A staging environment can be set up by following the same procedure as creating a development environment.

5 – Tag versions and backup

After releasing the changes to production, we perform a snapshot of the result for backup purposes. We do this by adding a tag on our Git repository and create a database backup, in order to be able to revert to the current state later in the future in case of a problem. For database backup, we use WP Migrate DB without any string substitution. The resulting SQL file name will be renamed with an indication of the version tag on the Git repo and will be saved to an external backup server.

Conclusion

Most of the articles we’ve found on the web were about how to duplicate a running website on a development or staging environment, but none of them resolved the matter of how to validate the changes made on a local environment to the production environment. We have managed to describe a development process for our WordPress website and push back to production.

Following this process, you can even introduce an intermediary integration environment, to test apply the changes on a server before pushing to production.  Unfortunately, a quite important part of this process is still manual. In the next step, we will work on how we could automate pushing changes back to production, and automatically run tag and database backup on a regular basis, to make sure that we’ll be able to revert back to an as-recent-as-possible version of our live website in case of a crash. For doing so, we have already identified some tools that could be helpful, such as WP CLI.