Deployment And Change Management - What Helps Now

Having defined the problem in my last post, we can now look at some of the options currently available for dealing with it. None of them is a complete answer, but they can all help out with some of the pieces. As a reminder, I'm just concerned with the problem of deploying changes to live sites for the moment. This is not a complete list by any means, but it is some stuff that I've found useful. Have you got another tactic you use to manage these types of changes? Let me know in the comments. If you're interested in this topic you should also subscribe to the Build Systems & Change Management group on groups.drupal.org.

  • Content Copy - If you're using CCK you're probably already familiar with Content Copy, an included module which creates a scripted CCK Content Type (and its associated fields) which you can then copy and paste into another site to be imported. It works really well, barring a couple bugs here and there (it likes to add an extra line to nodereference fields which you have to remove by hand.) One can also take these exports and use drupal_execute() to install them with a script like so:
    include_once ('./'. drupal_get_path('module', 'node' ) . '/content_types.inc');
    include_once ('./'. drupal_get_path('module', 'content') . '/content_admin.inc');
    $export = file_get_contents('your_file.txt');
    $values = array('type_name' => '', 'macro' => $export, 'op' => 'Submit');
    drupal_execute('content_copy_import_form', $values);

    So you can actually write a PHP script to do an import of your CCK content types, push it to your production server, and run it to get your new content types in place. This is pretty cool, although it takes a little work in advance to set up. I've found it's easiest to put the export scripts from content copy into a separate file and read them in, because there are various issues with needing to escape quotes or $strings that cause irritance when putting them inline. More on this here.

  • Views Export - Views offers the ability to export and import view definitions the same way content copy does. The definitions can also be dropped into code as described by Adrian in his comment to my last post, however I believe this removes the ability to maintain your view through the UI which may or may not meet your needs. I'm not a fan of this because we have a lot of less technical staff who still know enough to do things through the admin like create content types and views, but don't necessarily want to be diving into the code to make changes.
  • Journal - Journal adds a text box to every form on a Drupal site for users to enter comments on their changes. These comments are then saved to a custom log. On any form using system_settings_form (most forms under /admin), this field is required. This can be really useful while developing for keeping notes on your changes so you don't forget about things later when you're trying to put together a deployment plan later.

And here are a some things that are more experimental or not-quite-there but good from a "food for thought" perspective.

  • Autopilot - I wasn't even going to bring this up, since I really only wanted to talk about technology that actually exists. However Workhabit just got a Drupalcon proposal in under the wire, in which they claim we will finally be seeing a new release of Autopilot. Ever since their first announcement, Autopilot has looked like it is the product which will solve everyone's problems in this realm. Unfortunately, in the 9 months since that announcement, there's been little to see other than an alpha version they have disavowed, and a series of posts on their blog claiming the new release is just around the corner. Honestly it has been a little frustrating. I seriously hope the product is ready soon. If it is even half as great as their initial posting made it sound it will render many of these discussions moot. EDIT: It is now over 5 months since I posted this, and there is still no sign of AutoPilot. At this point I find it difficult to consider it anything but vaporware.
  • CoCKtail - This is pretty new, and the code is experimental, but it looks interesting. CoCKtail is a UML-like language for describing CCK types developed by Raincity Studios. It aims to address all sorts of configuration and deployment issues related to CCK content types, and while I'm not sure I buy into it, its very interesting and I look forward to see where it goes. You're best going to their website for all the details, there are a series of blog posts where they lay out everything the want to do with CoCKtail (Part 1, Part 2, Part 3) as well as the comments on those posts. Raincity has also proposed a session on CoCKtail at Drupalcon.
  • MySQL Dump Hacks - In theory it would be easy in a lot of cases to just mysqldump the appropriate tables, strip out the rows you want, and import these rows into the other database. Unfortunately this runs up against a huge obstacle - there's no way to guarantee your primary keys will be unique between the two databases. I read a posting somewhere a while ago, possibly on the change management group although if so I can't find it now, that posed a novel workaround for this - he hacked db_next_id() on his dev server such that it only produced odd IDs, and on his production server such that it only produced even IDs. Thus he could move records around from one to the other without risk of collision. I had actually toyed with a similar idea at one point - setting up my dev environment with two database connections (dev and live), and hacking db_next_id() to always pull IDs from live using db_set_active(). All your keys are then generated from one source. Please note, I am not actually advocating doing either of these things, they're just interesting ideas.

Next up, ideas! Progress! Maybe even some proof-of-concept code if I'm not too scared.

Comments

It broke on the simplest of fields, and I spent more time messing about with it than I was working on my install profile.

I also recently had had enough of working around cck's deficiencies for my project, and started rewriting all my content types into standard drupal node types. Which in turn has made the code a _lot_ cleaner, and has given me sufficient control that I didn't have to do anything "hackish" to make it all work together (such as fields that should not have any user interface elements, as they are exclusively backend). It also made about 9 dependencies just go away.

And the upgrade path is also now completely clear. And everything is cleanly versioned. CCK was incredibly useful for prototyping however, but in the end it was just saving me from implementing 1 CREATE TABLE statement , 5 sql statements (insert, update, select, delete, delete revision) and 2 elements per field (input and display). Even views integration isn't that much extra work. It's basically ~ 100 lines of boiler plate code with a few details slotted in (see node_example.module).

In my experience, CCK is less work for admins, but more work for developers. Guess who's time is more valuable.

Also. you can override module provided views, but then you lose the upgrade path and versioning.

I can see your point, I think it just depends on what you're looking for.

I have found CCK 100% invaluable in getting our sites up quickly, especially when I was just starting out (a mere nine months ago.) However, the biggest value has been in the fact that there are now whole swaths of our site that don't need a developer at all. We're about to launch a big project with 8 or 9 new content types, views, site-wide redesign, the works. I did nothing on it! Three months of development with no developer! A (admittedly pretty technical) designer, some business owners and artists, it is just amazing to me this is possible and frankly I embrace it. My experience has been the polar opposite of yours - it has saved me mountains of time.

Honestly, I'm happy not to have to reinvent the wheel, and leave the upgrade paths and security fixes to their immensely capable team. If the downside of that is I have to hand-edit a content copy export every three months, so be it. Well that an the UI for multi-input fields is pretty awkward. I expect CCK 6 to be a big improvement though.

Using Views export to maintain your views is perfectly compatible with the UI. You can still make 'on-the-fly' UI tweaks by overriding the view and tweaking the database. Then, your higher level site maintainers are on the hook to re-export that form and put it back into code and push it out. Once it's pushed, remove it from the database.

One other thought would be using macro module to record the forms you submit, such as setting up a view, a CCK type, etc. I have not tried this out yet, but the Lullabots have written the process up: http://www.lullabot.com/articles/moving-cck-field-changes-from-dev-to-live .

I have tried this method several times, making an export just before creating a complete new database to work with to apply changes and check results.
Maybe my sites are not very very complex regarding database schemes, but proceeding this way I have never faced a problem with my primary keys. I know, I have no guarantees, but it has worked fine, and I have saved lots of time.

http://drupal-la.blip.tv/#1059992
Starts with Part 10, continues with parts 11 and 12.

As far as I'm concerned, until AutoPilot gets released, it doesn't exist.

Thanks for the heads up Stephen. Has a firm release been officially announced?

Add new comment

I wrote two chapters of this book - Drupal 7 Module Development and I co-wrote it with Matt Butcher, Larry Garfield, Matt Farina, Ken Rickard, and John Wilkins. Go buy a copy!
I am the owner of the configuration management initiative for Drupal 8. You can follow this work at the dashboard on groups.drupal.org.
I work at Lullabot!. If you don't know who Lullabot is then you haven't been around in the Drupal world long have you? Come check us out!