Tuesday, August 10, 2010

Migrating System Settings with Drush Migrate and Drake

Exporting Pathauto settings

As I mentioned in the last post one of my biggest headaches with Drupal is moving changes from one server to another. I specifically chose pathauto alias settings as an example because the form can be rather large and repetitive to fill out. I'll skip the part about how repeating your steps on this form gets boring, because it was covered in the last post. The focus of this post is to talk about different ways Drake can solve this problem.

Breaking down the Pathauto settings form

The pathauto alias settings form is defined in pathauto.admin.inc. The part that we're most interested in is the very last line of the form definition.

return system_settings_form($form);

This form, like many other core and contrib module defined settings forms, is utilizing a system settings form function to handle its processing. Calling this function on your form adds a submit handler which loops over each of the keys in your form and calls a variable_set on the submitted value. So now, when you click "Save Configuration" the values you entered in get placed in your site's variables table.

Write a drake script to simulate a form submit

One of the things you can do with drush out of the box is set variables by command line. Very simply you could manually log onto each of your servers and run a series of "drush vset foo_var_name bar_var_val" commands. But why do that when you could write a drake script?

$drush drake-file
$drush vim 432412431.drake
vset pathauto_max_length 100

Now all you have to do is execute drush drake on each of your servers and your variable change is executed. Your changes are sequentially executed, can be stored in source control management, and can be handled automatically if your build process was built around drake.

Taking it to the Next Level

It didn't take me very long to be unsatisfied with writing my own drake scripts to set these variables. For starters you have to either dig through the code or look at the form with Firebug to get all the variable names right. You're also probably repeating your work since you'd make your pathauto changes by hand in the admin panel, test them out, and then write a drake script. The other issue that came up has to do with array data. Currently, drush's vset doesn't work well with array data. There is some discussion about how to handle a user inputting array data. So what we're left with is: getting too familiar with form element names, repeating yourself, and only working with simple key/value pairs. Time to give up?

Enter the module side of drush_migrate. In drush_migrate_form_alter I scan the form's submit handlers for the presence of the system settings form submit handler. At this point I can assume that the main goal of this form is to set each of its keys to the site's variables table, which means you can generate a script file automatically that accomplishes the same goal. The form_alter function adds an extra "Export to Drake" button that creates a drake file and writes a series of vset's for your form's keys.

This gets rid of problem 1 and 2, because you are no longer writing form keys by hand, and you are also only making your configuration changes once. The problem with array data still exists. For safe exporting, when the module creates the drake script it serializes each value, regardless of if it is an array or not. When drake processes your scripts, it attempts to unserialize the value. When processing arguments, if the value is serialized we unserialize it. If the value can not be unserialized (i.e. the command name, hand written arguments), it is kept as is. We can now export array data from system setting forms easily, because each line is getting serialized by the drush_migrate module. Exporting your pathauto settings is now as easy as clicking a button. Importing them is as easy as typing a one line command.

Closing

This initial functionality to drush_migrate solves a very small portion of the problems that arise in a multi environment Drupal team. However, utilizing a little Drupal knowledge and the pre existing vset drush command I was able to find a solution that I was happy with. My next goal is to tackle migrating roles and permissions.

Sunday, August 8, 2010

Managing your Drupal Environments

Developing in multiple environments

Imagine you are a developer on a team that is made up of 3 developers, 2 UX resources, and a hands on PM. Each team member has a local installation of the Drupal site, as well as there being a QA environment, a content staging environment, and the production environment.

Your hands on PM gives you a task: Configure URL path rules for our content types using the pathauto module. It starts out as being pretty easy, you define your rules of what words and punctuation to ignore. You give path patterns for you content types. Then you run the bulk update on your local machine and check to make sure everything looks ok. Done, right?

Well, not exactly. You need to somehow get the same configuration settings onto QA and staging, and then eventually production. Also the other members of your development team need to have those rules on their instances. How are you going to accomplish this? There are 2 general approaches that I've seen used to solve this problem.

Approach 1: DRY - DO Repeat Yourself

One of the ways I've seen this problem tackled is to designate one environment as an integration or HEAD environment. This is a setup that may or may not be the content staging environment, but is mostly treated as a shared dev instance. Once you have completed your pathauto dev task locally, you check the module code into a source control and then update the integration env. You then log onto the admin of the integration environment and repeat your configuration steps. The other developers on your team are now responsible for pulling a db snapshot from the integration environment to their machines. At this point you might choose to pull specific tables of data from integration to QA and production.

What's wrong with this approach?

For starters you're repeating yourself. It could take you N times longer to complete a task (where N is the number of environments you maintain). You'll have to repeat your steps on each environment.

You're also running a risk of making a mistake. The most common mistake I've made with this approach is either forgetting to update an environment, or updating one incorrectly. Pathauto is a great example because you could be updating several rules for several content types.

Approach 2: Document and follow

The second approach is to keep documentation of each configuration change you make. Each developer would give step by step configuration instructions for someone to follow in a future deployment. When it's time to roll out new changes, someone would then have to follow each instruction line by line.

What's wrong with this approach?

It's not going to take much to describe what's wrong here. First of all developers aren't very good at writing directions. Second, people aren't very good at following directions. There's no guarantee that your directions are going to be followed correctly, if even at all. More than once I've been in a situation where a defect is reported that was a direct result of documented migration instructions not being followed.

Enter drush_migrate

I've started a project to tackle this problem using drush to create a sort of framework for executing migration scripts, inspired by RoR's rake. The behind Drake (drush rake) is to write your changes into scripts that can then be executed on each environment. Each script is a collection of drush commands that would let you do things like: download modules, enable modules, set variables.

Why drush_migrate

When you use drush_migrate, your drake scripts can be put into a source code management. Also, while developers might not be good at giving instructions to other humans we are very skilled at giving instructions to machines. Likewise, machines are better at following instructions than people.

I've started coding drush_migrate in github, and will be creating a drupal.org project in the very near future. I'll be posting more about how to utilize drush_drake very soon.