drupalgardens

Well, Sprint 39 just finished up at Acquia Engineering and we've got a number of new exciting features and bug fixes releasing to Drupal Gardens in the next week.  One of these is Mollom - the best spam protection service on the internet - for all Drupal Gardens site owners.  For those not in the know, Mollom protects your site from spam by analyzing the contents of comments or other content that is contributed by users and presenting Captcha challenges if the content is suspect.

What this means for Drupal Gardens beta testers:

You'll get setup with the best spam protection available from Mollom.  No cost, nothing for you to do.  Just works.

What this means for Developers who want to provide Mollom to their customers:

Most people aren't aware, but Mollom offers an extensive provisioning API for Mollom partners to create accounts on behalf of their customers.  Gábor Hojtsy and I got to work on this feature and Gábor wrote the mollomapi module which does the provisioning. Here's a sample of how the API works:

$mollom_keys = mollomapi_site_add('whitehouse.gov', 'obama@whitehouse.gov');
if (!empty($mollom_keys) && is_array($mollom_keys)) {
  $public_key = $mollom_keys['public_key'];
  $private_key = $mollom_keys['private_key'];
}

When we (Acquia) started planning our hosted Drupal service (Drupal Gardens) a year ago we had to take the call of developing in Drupal 7 or Drupal 6.  I don't think this was ever really in doubt, but the decision to try to build a product on Drupal 7 core at that stage was certainly a risky one. Why? there were almost no contributed modules or themes and the architecture and APIs were changing daily. Acquia's business is squarely Drupal.  Drupal support, Drupal promotion, Drupal migrations, Drupal polish, Drupal architecture, Drupal Drupal Drupal (you know the song).  So even though this was a risk, we knew that the benefits to Drupal and Acquia would be too great to back down from it.

There is a saying open source software: 

Q: When will it be done?

A: When it is done.

One of the things the Drupal community prides itself on is a dogged dedication to quality irrespective of commercial or social pressures.  This means that we have to be prepared to stay up to date with new developments and adapt our product to the movement of the many thousands of developers who drive Drupal forward. 

In other words, everyone on the Gardens engineering team has to be aware of what's going on with core.  We have to keep up with every commit and know which ones are likely to land in the future.  Many of our engineers also figure heavily into the list of core patch writers and reviewers.  

The upside is that we all get to work on core, improving Drupal while testing the core APIs by building the first product on Drupal 7.  The community gets a massive influx of hours by Acquia engineers AND a ton of free beta testing as we have thousands of Gardens sites using Drupal 7 and finding bugs / suggesting improvements. The feedback thus far  from gardens testers on the usability improvements in Drupal 7 has been extremely positive!

Recently, we decided to switch from basing Drupal Gardens releases off of Drupal 7 Alpha releases to running off of "HEAD".  What is HEAD and what is the difference?

• Alpha releases are getting released every month or so.  They mark a point in time which Angie Byron (webchick) and Dries have decided that the product is "stable enough" to declare an alpha release.   It doesn't mean a whole lot, but is basically a chance to mark the time, look at our achievements and encourage people to try it out and get involved outside of the core developer community.
• HEAD means the latest and greatest code that gets added to Drupal core.  Every day Dries and webchick commit about 8-12 new pieces of code.  So if you grab HEAD today, you have the very last thing that was added, if you grab it tomorrow, you'll get the changes that went in tomorrow.

Running from an Alpha vs HEAD is like the difference between playing Jenga on a sleeping elephant to playing Jenga on a cocaine addled elephant riding a skateboard being jabbed in the ass with a hot poker.  In other words, it's a wild ride.

To facilitate that process, here's what we do:

(Warning: It's about to get nerdy up in here, so if you don't care about such things, leave now with the elephant image)

• Every sprint (3 weeks) we create a new branch for the core upgrade.  We do this to keep trunk from being broken while we at least get it installing.
• Next, we go through the entire CVS log from the last time we updated until now.  I use Git for core development and GitX for reviewing the backlog.  It's a pretty sweet process, see the pic (click for larger):
  
  
  
  
  
  
  
   
• While going through the backlog we:
  • Identify each change which will affect databases and require a head2head update function (more on that below)
  • Identify each change which changes APIs that we use in our custom modules or install profiles
  • Identify each change that will affect our documentation and screenshots
• Then we repeat the process for every contributed module we have in our code base.  We check their changelogs, update their code and if it hasn't been updated (which is often the case), we contribute patches to the queue or commit if we have rights.
• Nathanial Catchpole (Catch), a member of the Clarity team started the head2head module recently.  This module is an effort for D7 developers to collaborate on update functions.  Since Drupal 7 is not released yet, there is no upgrade path supported.  This means that when code changes, it can totally break your site or destroy data (remember the elephant).  Since we've got thousands of people using Drupal 7 on real sites (like this one) in Drupal Gardens, we can't do that.  So people who contribute to head2head add those update functions which would be present in all official Drupal releases (and will be supported when the first Drupal 7 beta comes out).  We love this idea and have already made a few commits there!
• We then update all of our custom modules and make sure that we can install DrupalGardens.
• Then comes the fun part.  To ensure that we don't break our beta-tester's sites, we actually take dozens of example sites and upgrade them using our automated build process.  We run automated testing and manual testing to ensure that there was no data loss and the upgrade smoothly completed.

We live on the bleeding edge, so you don't have to.

I'm proud to be a member of our team because even though this is a really difficult task and we expose ourselves to a lot of risk, it all of this leads to big improvements in Drupal 7 core and contrib. For instance, here's a sample of some of the modules we have either written or contributed to as a part of Gardens: Follow, Mailing List, Comment Notify, Media, Simpleviews, Pathauto, Rotating Banner, Mollom, Styles, Typekit, Wysiwyg, Addthis.  This version of Drupal Core may be one of the most widely tested versions by the time it releases and thanks to the efforts of the #D7CX crew will have more than enough contributed modules!

If anyone is interested on how to keep up2date with Drupal 7 and/or how to get involved in Drupal 7 core development, please feel free to get in touch.