Continuous Deployment to CPAN

Recently I was working on a refactor of one of my CPAN modules which, among other things, involved changing its name from Test::A8N to the specific Test::Story.  Doing so made me think about the process I usually go through when I consider releasing a CPAN module.

First, let me explain something about myself: I don’t like tedious or repetitive tasks.  I hate having to do the same thing over and over again, partly because I don’t want to waste my time, but mostly because inevitably one of the following will happen:

  1. I’ll forget a crucial step, and will screw something up;
  2. I’ll forget how to do it, and in my efforts to re-learn it I’ll screw something up;
  3. I won’t care enough to go through the effort, so something will get screwed up.

I expect you’re noticing a trend here.  Really the only reason programmers come into this profession in the first place I suspect is because we’re just so bad at doing things the normal way, we have to automate everything we’ll either do poorly, lazily, or forget to do all together.

For those of us who are programmers, many times we’re so lazy that we won’t want to do the same thing within our programs more than once, so we abstract functionality into reusable modules.  By that token the Perl community must be some of the most inventively lazy group of people, because CPAN is full of useful tidbits like that.  Getting modules to CPAN requires a contributor to actually, you know, submit their project.  And this is, in itself, a somewhat manual process.

I do all of my development in a version control repository, and I write a decent amount of unit tests to prove my functionality works.  So once I come to the decision that a set of new features is worthy of a new release, this is the process I go through:

  1. Run “perl Makefile.PL && make && make test” to verify everything runs okay;
  2. Run “make dist” to create the distribution tarball;
  3. Noticing the version number is wrong, I go in and change it in the main Perl module;
  4. After running “make dist” again, I realize I forgot to change the README;
  5. Potentially after another “make dist“, I’ll remember I’m supposed to update the Changes file to indicate what I’ve added;
  6. It’s at this point I realize that I forgot to add new documentation to cover this new feature;
  7. I run “make test” again, this time with TEST_POD=1 set to ensure my documentation checks are run;
  8. I’ll then try to remember what the command is for uploading a new CPAN module, which involves searching on CPAN for something related to “Upload”;
  9. Finding “cpan-upload“, I’ll have to look at its documentation to figure out how to use it;
  10. I run “cpan-upload“, only to realize I forgot to set my PAUSE credentials in ~/.pause.

At this point, if I’m lucky, the upload will succeed.  This process isn’t meant to be a negative reflection on CPAN, but rather on my own forgetfulness and need to automate.

Read on to find out how I managed to automate this part of my life as well.

Continue reading “Continuous Deployment to CPAN”

Building iPhone apps with Hudson, Part 2

I’ve already posted before on how to set up Hudson to compile and build iPhone applications, but I just had a “OMG I ❤ Hudson!” moment just now, and felt I had to share it.

I do most of my mobile development literally while I’m mobile; on the train during my morning commute, from coffee shops on the weekend, or in front of the TV in the evenings when I’m winding down for the night. Because of this, I don’t have any consistent time when I’m making checkins, nor do I have the time to create builds for my beta users.

One of the iPhone apps I’m building is for a client, and because my time is limited, I want to streamline communication as much as possible, especially since I have a day job that I really like and uses 8 of my precious 24 hours per day. I don’t want to have to tell them every time new features are ready to be tested. Additionally, if I have to manually compile and send a beta build to them whenever I complete a new feature, I’ll never have enough time to actually get any work done. Add to that the fact that many mail servers will block iPhone applications for one reason or another (file size limits, misfiring antivirus filters, or any number of reasons).

So the solution I’ve come up with is an extension of my iPhone application build scripts combined with some nice plugins for Hudson. All of the above steps are completely automated away and handled for me just by checking my code into git.

  1. The first thing I did was updated my build scripts to generate an “.ipa” archive for Distribution builds containing an iTunesArtwork file, and a “.zip” archive for Release builds.
  2. My build script sets the version number of my application to the $BUILD_NUMBER of my Hudson job, so that the version is guaranteed to be unique.
  3. I added the SCP plugin to Hudson and configured a location on my web server where Hudson could upload completed builds to the website. This enables my clients to access the page to download new releases.
  4. In order to inform my clients that a new build is available, I’ve used the customizable email support in Hudson to send an email to a custom list of email addresses when a successful build is completed. In this email I include a link to the uploaded IPA so they can easily download the beta archive.
  5. So that I don’t have to go through the trouble of describing what changes I made, I use the $CHANGES_SINCE_LAST_SUCCESS variable in my custom email template that lists all the commit messages I’d submitted to git that were included in this build.
  6. For good measure, I SCP to the website a copy of the “.mobileprovision” certificate file that was used to build the application, so the beta tester can add it to iTunes if the contents have changed.

The result of all this work is that I can work wherever I am, continually committing changes into git. When I’m ready to cut a new build, I submit my changes to github where a webhook script will notify Hudson that changes have been made. Hudson checks out my changes, builds them on my Mac Mini at home, SCP’s the resulting archive up to my web server, and notifies my beta testers that a new archive is available and what changes they should keep an eye out for.

For a copy of the build scripts and configuration I use inside my projects, take a look here:

This assumes there’s a file in the build user’s home directory called “.build_password” that contains the password used to unlock your keychain. Run “chmod 600” on that file in order to make it hidden to anyone else on your system.  And make sure you check in the “.mobileprovision” files your XCode project file references.  If you change your certificates, you’ll need to copy the updated mobileprovision files from your “Library/MobileDevice/Provisioning Profiles” directory.

So you can see why I’m excited about this.  I hope you are too.

How to automate your iPhone app builds with Hudson

As any iPhone application developer who’s released at least a single app to the App Store will tell you, releasing your app is a terrible pain in the…well, it’s not a fun experience.  After your second or third app you start to get the hang of things, but there’s still pain and suffering involved.  Managing certificates, getting settings configured properly, and iterating between development, AdHoc beta builds, and the final App Store release builds, all make the process seem tediously manual and prone to human error.

In professional software development shops, you would use a Continuous Integration server to monitor your source control repository, check out changes as they’re submitted, compile, test and package up builds, and notify developers of the build’s health via emails and a web-based “Dashboard”.  I missed having this while developing my PhoneGap-based iPhone applications, so I decided to once and for all bring good development practices to my iPhone work.

Why do I need to configure automated builds anyway?

I get this a lot from people when I’m trying to convince them of the need for automated builds.  I personally find it hard to imagine people getting by without them in a single-developer project, let alone when multiple developers contribute to a project.

Monitoring the health of an application

Lets face it, we’re human, and we make mistakes.  It’s alright to break code from time to time, but what really sucks is when you find out far too late.  Did your recent changes accidentally eliminate your Entitlements.plist file, thus breaking distribution or release builds?  Do you have a file or library you forgot to check in, meaning when you delete the project from your working directory all those changes will just vanish?

Instead of having to remember to check each of those things manually (which, lets face it, you’ll forget at least half of the things you’re supposed to do inevitably), why not have an automated system tell you every time you make a change?  And if you’re in a multi-developer project, you’ll be able to see who broke the build and what change specifically broke it.

Always be ready for distributing your application

Many times in the natural course of development you’ll break code.  You’ve gotta break something in order to improve it.  But sometimes someone (your wife, a client, a beta tester) will want to try out your application before you have an opportunity to finish off your recent changes.  Instead of spending ages back-tracking your work to get your application to compile, why not rely on your automated build system to keep archives of previously successful builds?

Release what you test

Since you want to test an application before you release it to the App Store, you’ll probably create an Ad-Hoc distribution build to give to friends, family, or official beta testers before you bundle your application up to send to the App Store.  Maybe your testers will find bugs, maybe they won’t.  But at the end of the day that compiled app bundle you just created isn’t actually what you submit to Apple.  You need to compile a completely different app bundle with very different files stored in a Zip file, and if you’re not careful you could potentially be releasing something different than what you tested.

Why not have your automated build system create both your Ad-Hoc distribution build as well as an App Store release build every time?  That way you’re not only always ready to release something to the App Store, but you can be guaranteed that you’re submitting to Apple the exact code that your testers evaluated.

More benefits than I can list

If you’re really serious about best practices, you’ll probably want to write unit tests for your code and have those run after your code has been compiled, but before your build is packaged and archived.  Just because your code compiles doesn’t mean that it will behave correctly.  And lets face it, if you have a lot of tests, you’ll never wait for all of them to run throughout the course of your work.  So by running your tests as a prerequisite to a build succeeding, you’re guaranteed that you’ve got a safety net.

There’s plenty of other best practices that having an automated build system can help with, so what I’m discussing here will just cover the tip of the iceberg.  If I’ve convinced you that automating your builds, read on.

Continue reading “How to automate your iPhone app builds with Hudson”

A Git Workflow for Agile Teams

I’ve been using git for all my new software development and have been converting my old Subversion (and gasp CVS) repositories over. Throughout this process I keep learning more and more about git, and my love story with it continues every day. Thanks to my friend Chris, he sent along this blog posting that really describes how you can do hardcore agile software development in teams with git. This really is helping me solidify in my head what it is I want to do, but have just never had the tools or know-how to put into practice.

At Hashrocket we use git both internally and in our Agile mentoring and training. Git gives us the flexibility to design a version control workflow that meets the needs of either a fully Agile team or a team that is transitioning towards an Agile process.

via ReinH | A Git Workflow for Agile Teams.

New PhoneGap feature: ActionSheet support

A while back David Roe had sent me a patch for an implementation of the ActionSheet control for PhoneGap. He was using it in an application of his, and probably due to my plea for patches from the PhoneGap community at large, he submitted this little tid-bit to me. Since I’ve been making some pretty big strides toward implementing a unified callback and event dispatch mechanism within PhoneGap, I refactored a bit of it, and created a generic Dialog class for PhoneGap.

Currently only button dialogs are supported, but in the future I’d like to create a variety of dialogs that PhoneGap application developers can use; date pickers, scroll picker widgets, and so forth. The API is as simple as I could make it, while still keeping in mind that other platforms may not have a native control to accomplish these features.

Basically I was thinking “What would I want to put in an action sheet”, and I came up with:

  1. A list of buttons, maybe with different colours
  2. A date / time picker, maybe with a different start date
  3. A scroll-wheely value selector
  4. ….

Here’s the desired API I came up with for a very complex usage:

{ label: “First button”, type: “default”, onClick: function() { … } },
{ label: “Second button’, type: “cancel”, onClick: function() { … } }

which would result in the following GAP command:

gap://Dialog.openButtonDialog/First button/Second button?type_0=default&type_1=cancel&onclick_0=…

and so on. With the event callback system I built, the onclick handlers will be given their own callback ID if they’ve been specified. Additionally, the following syntax is supported as well, for more simple uses:

“First button”,
“Second button”,
{ onClick: function(index, label) { … } }

so you just give it a list of buttons, all shown as a default type, and the function would be called no matter what button was pressed.

This second form is necessary as you’d probably use it for the other types of action sheets:

new Date(‘2009-02-14’),
{ onClick: function(newDate) { … } }
‘Value 1’,

‘Value 2’,
‘Value 3’,
{ onClick: function(value) { … } }

Those most recent examples aren’t implemented yet, but it shows how this particular class can be extended in the future. And they’re all potentially supported on other platforms as well.

Does all this sound reasonable? Please leave comments if you have any thoughts on the matter.

Check out the documentation of the Objective-C Dialog implementation, and the JavaScript Dialog interface to it. If you want to play with this before I push it into the official branch, you can try it in my experimental UIControls branch.

PhoneGap UIControls ready to go

I’ve merged the results of my UIControls branch on github into my master branch. I think my little experiment went well, and I’d love to get feedback from people on how this new API works for you. I still have some great plans for it, but before I get ahead of myself, let me cover what I’ve done:

• Refactored the command call API to allow for a richer set of arguments to PhoneGap commands
• Moved some commands around to more appropriate classes (e.g. Alert and Vibrate both moved to Notifications)
• Reorganized the XCode project so commands are clearly separated from PhoneGap infrastructure
• Renamed the Settings.plist PhoneGap config file to PhoneGap.plist, and created a new Settings.plist file that contains custom application-specific configuration.
• Made all PhoneGap commands inherit from a common base-class that auto-loads its own Dictionary of configuration from the main PhoneGap.plist file.
• Added UIControls command class that exposes tab bars and toolbars to JavaScript.
• Updated the demo to show off tabs and toolbars

All that looks like a big change, but almost all of it was infrastructure changes that were necessary to get UIControls to work. Previously, all PhoneGap commands were class method calls, meaning it was very difficult to maintain state between command calls. Now that commands are actually called on an instance of a given command class, it’s easier to maintain state. So when a tab bar is created, multiple calls can be made, each to construct different aspects of the UI. Without all this, the following example would have been much more complicated:

uicontrols.createTabBarItem(“toprated”, “Top Rated”, “tabButton:TopRated”);
uicontrols.createTabBarItem(“recents”, “Recents”, “tabButton:Recents”);
uicontrols.createTabBarItem(“more”, “More”, “tabButton:More”);
uicontrols.showTabBarItems(“toprated”, “recents”, “history”, “more”);

The changes to the PhoneGap configuration were necessary because now, when a PhoneGapCommand subclass is constructed, it will look in the PhoneGap.plist configuration file to see if there’s anything pertaining to it. So if there’s a key in the dictionary with the same name as the class being constructed, it will use that as a local configuration dictionary influencing just that one class. That way, if you don’t use a feature of PhoneGap, or you have to configure a lot of options for a single type of command, these options won’t be cluttered alongside the standard global PhoneGap settings.

I created Settings.plist because of experiences I had in an application I’ve been creating in order to test my new UIControls branch changes. I found that I wanted to set compile-time options (for instance a “lite_mode” boolean) that influenced the way my app runs, without having to change HTML or JavaScript code every time. So instead what I have is a configuration plist file that is used exclusively in JavaScript for my application. I’ve added the excellent SB-JSON framework in to the PhoneGap project, and use that to pass these settings into the JavaScript application at start-up time. So all you have to do is read Settings.lite_mode, for instance, in order to read properties set in your plist file.

Oh, and finally, I’ve moved the JavaScript documentation to javascript/docs, since they were practically impossible for new users to find in the past. I’ve been working on creating DoxyGen docs of the PhoneGap code for use in XCode’s documentation browser, but I haven’t gotten far enough there to actually check anything in besides comments. I found instructions on how to generate XCode docsets from DoxyGen, but I haven’t gotten it working just yet.

So please, try out my PhoneGap updates and let me know what you think of it. I think the integration of the JSON framework will make things like Contacts much simpler to implement.

Before I go though, I want to give a little “wish list” of features I’m planning on adding in the near future. I’ll get to them whenever I can, since my own app development takes priority of course.

• Toolbar buttons
• Tabbar show/hide animations
• File support (read / write local files)
• Camera and photo library support (POST to a server, save to a local file, etc)
• Native “Flip / Slide” transitions (no more having to mimic them in CSS)
• 3rd party API integration, like AdMob and Medialets (I already have both of these done, but I’m in discussions with both companies to determine if the terms of their license allows me to redistribute it)

Managing sites with Git and intelligent post-update hooks

I’ve recently begun drinking the koolaid of Git, and damn it’s tasty! The things I can do with git that I couldn’t have done before (or would have been difficult to do) makes me excited about it. In fact, the one feature that I thought was a drawback — the no-one-true-server nature of it — is actually its strongest selling point.

See, the way I’ve taken to doing my development now is I create two “remote” repositories. First is “origin” which points to a repository managed by Gitosis. Second is a “live” repository that points at a working directory on my production server. That working directory is where my live site actually runs in.

On its own this is handy. As I develop new features, I push my changes to origin. Once my code is ready, and after I run “make test” to verify my site passes all its unit tests, I push to my live repo. At this point I ssh to the server, restart my server processes, and in theory all should be well.

The need for automation

I discovered quickly that in practice this was fraught with error. After a fairly large refactor, I found that the code that worked perfectly well on my development laptop fell over on production. Old or missing libraries, dependancy problems, ownership permissions on files, you name it. My site was down for 2 hours while I tried to resolve these issues. I added more unit tests to my code, but still this wouldn’t have caught these problems.

At this point I decided that a more automated approach was needed. I used a friend’s post-update hook as a template, which simply merged in changes and restarted nginx following a push to live. To this I added a long one-liner, and added the relevant commands to /etc/sudoers with the “nopasswd” option. In the end, the function looks like this:

bounce_fcgi() {
  echo $PWD >&2
  [ -f Makefile ] && make clean >&2
  perl Makefile.PL >&2 && make test >&2 &&
    sudo /etc/init.d/nginx test 2&>/dev/null &&
    sudo /etc/init.d/nginx reload &&
    sudo /etc/init.d/webapp restart >&2

Essentially, after my new changes are merged in, it:

  1. Creates and runs my Makefile
  2. Runs all my unit tests
  3. Tests my new nginx server configuration
  4. Reloads my new nginx configuration
  5. Restarts my FastCGI web application

If any of those steps fails, the full process halts. All the >&2 arguments ensure that the output of these commands are echoed to my local console from the remote server. So when I type “git push live”, all the test output is displayed to me inline. If an error occurs, I can immediately fix it in my local environment and push out a new change without having to log in to my remote server once.

My web application is written in Catalyst, and I use Test::WWW::Mechanize::Catalyst extensively, so I not only unit test my back-end classes, but I test the URLs users actually interact with. It even goes so far as creating and destroying test accounts within my database, so every feature of the website is tested, right down to validating the contents of my robots.txt and sitemap.xml files.

Next Steps

There are still some holes that need to be filled here.

  • If running my tests fail for some reason, I would like to roll-back the remote working directory to the previous version and restart my services, that way the site continues to function under a known-good state.
  • I would like to use WWW::Google::SiteMap::Ping to notify Google, and perhaps other search engines that support XML Sitemaps, that the contents of my site have changed and a reindexing is needed.
  • My site is localized, so I would like to regenerate my PO translation files, and if any strings have changed or are out of date, automatically send an email to my translators with the new POT template file attached.
  • Run my HTML through a spelling checker, to verify I don’t put any typos in any of my pages.
  • Since I try to use caching as much as possible, when a web page’s content has changed, I would like to automatically connect to my Memcached servers to purge the relevant pages from its cache so new versions are immediately available.

A fringe benefit is makes it fun and exciting to write unit tests! Whenever I find an area that isn’t covered, I sit down and crank out more tests to validate new features. I never want to be caught with my pants down on a deployment. Each time it catches something I missed, it makes the whole thing worthwhile.

Does anyone have any thoughts on how I can improve this? Is anyone doing something similar that I can learn from? I’m very excited about my new deployment system, and wish I’d had this ages ago. If there’s anything you want to know more about, please leave a comment.

Reblog this post [with Zemanta]