Logging with CocoaLumberjack and TestFlight

Consider the following situation that happens far too often in mobile app development: You’ve just released an app that works perfectly for you, and you’ve tested it extensively. You’re proud of your accomplishments and submit the app to the world, only to have several emails sent to you from users who have nothing but difficulties in running the app. You send a bug fix release to the App Store, but since you’re still unable to reproduce the problem you’re at the whim of luck and end-user feedback. You can hope your users know how to send you a crash report, but what if the app isn’t actually crashing? Wouldn’t it be great to be able to access your app’s log information from that client to be able to troubleshoot the problems?
Continue reading “Logging with CocoaLumberjack and TestFlight”

Building iOS apps for Over-The-Air AdHoc distribution

I’ve written about building iOS applications with Hudson Jenkins, but until recently there hasn’t been a convenient way of getting those applications to your testers. Of course the most important part of your build output will be the app bundle you send to Apple’s iTunes Connect web interface, but throughout your development cycle you’ll want to test your app.  Sure you could build and deploy a debug build straight to your own personal device, but you get the most benefit from having other people beta test your app.

With recent releases of Xcode and the iOS SDK, Apple improved their AdHoc distribution support with two main enhancements:

  1. Mobile provisioning files can now be embedded in the App’s IPA itself, meaning you don’t have to maintain and update separate .mobileprovision files separately;
  2. A specially-formated manifest Plist file can be created that, when linked to properly, allows test devices to install new versions of your AdHoc app without needing to plug into a computer to sync the app using iTunes.

These improvements are huge, but require some changes to your build scripts and your Continuous Integration environment.  I’d like to show you how to do this in your own installations, and show you some options for how to distribute your apps to your testers.

Continue reading “Building iOS apps for Over-The-Air AdHoc distribution”

Last minute talk on automated Perl builds using Hudson tonight

My friend Scott McWhirter, who heads up the Vancouver Perl Monger’s group, asked me yesterday to give a last-minute talk on anything in particular at tonight’s Vancouver.pm meeting. He wasn’t exactly begging, but I know he’s short on speakers this month, and he wanted something interesting to show.

So I decided I’d talk about building and testing Perl-based projects using Hudson. I’ve been planning on writing a blog post on the subject for the past month, but haven’t found the time to finish off the post properly. So if you’re interested in the topic, and you don’t want to wait for me to get around to writing about it online, please feel free to drop by tonight!

Update: The talk went well! Until I have time to put a more comprehensive post up on the topic, you can always read the slides from tonight’s talk.

How to automate your iPhone app builds with Hudson

As any iPhone application developer who’s released at least a single app to the App Store will tell you, releasing your app is a terrible pain in the…well, it’s not a fun experience.  After your second or third app you start to get the hang of things, but there’s still pain and suffering involved.  Managing certificates, getting settings configured properly, and iterating between development, AdHoc beta builds, and the final App Store release builds, all make the process seem tediously manual and prone to human error.

In professional software development shops, you would use a Continuous Integration server to monitor your source control repository, check out changes as they’re submitted, compile, test and package up builds, and notify developers of the build’s health via emails and a web-based “Dashboard”.  I missed having this while developing my PhoneGap-based iPhone applications, so I decided to once and for all bring good development practices to my iPhone work.

Why do I need to configure automated builds anyway?

I get this a lot from people when I’m trying to convince them of the need for automated builds.  I personally find it hard to imagine people getting by without them in a single-developer project, let alone when multiple developers contribute to a project.

Monitoring the health of an application

Lets face it, we’re human, and we make mistakes.  It’s alright to break code from time to time, but what really sucks is when you find out far too late.  Did your recent changes accidentally eliminate your Entitlements.plist file, thus breaking distribution or release builds?  Do you have a file or library you forgot to check in, meaning when you delete the project from your working directory all those changes will just vanish?

Instead of having to remember to check each of those things manually (which, lets face it, you’ll forget at least half of the things you’re supposed to do inevitably), why not have an automated system tell you every time you make a change?  And if you’re in a multi-developer project, you’ll be able to see who broke the build and what change specifically broke it.

Always be ready for distributing your application

Many times in the natural course of development you’ll break code.  You’ve gotta break something in order to improve it.  But sometimes someone (your wife, a client, a beta tester) will want to try out your application before you have an opportunity to finish off your recent changes.  Instead of spending ages back-tracking your work to get your application to compile, why not rely on your automated build system to keep archives of previously successful builds?

Release what you test

Since you want to test an application before you release it to the App Store, you’ll probably create an Ad-Hoc distribution build to give to friends, family, or official beta testers before you bundle your application up to send to the App Store.  Maybe your testers will find bugs, maybe they won’t.  But at the end of the day that compiled app bundle you just created isn’t actually what you submit to Apple.  You need to compile a completely different app bundle with very different files stored in a Zip file, and if you’re not careful you could potentially be releasing something different than what you tested.

Why not have your automated build system create both your Ad-Hoc distribution build as well as an App Store release build every time?  That way you’re not only always ready to release something to the App Store, but you can be guaranteed that you’re submitting to Apple the exact code that your testers evaluated.

More benefits than I can list

If you’re really serious about best practices, you’ll probably want to write unit tests for your code and have those run after your code has been compiled, but before your build is packaged and archived.  Just because your code compiles doesn’t mean that it will behave correctly.  And lets face it, if you have a lot of tests, you’ll never wait for all of them to run throughout the course of your work.  So by running your tests as a prerequisite to a build succeeding, you’re guaranteed that you’ve got a safety net.

There’s plenty of other best practices that having an automated build system can help with, so what I’m discussing here will just cover the tip of the iceberg.  If I’ve convinced you that automating your builds, read on.

Continue reading “How to automate your iPhone app builds with Hudson”

Managing sites with Git and intelligent post-update hooks

I’ve recently begun drinking the koolaid of Git, and damn it’s tasty! The things I can do with git that I couldn’t have done before (or would have been difficult to do) makes me excited about it. In fact, the one feature that I thought was a drawback — the no-one-true-server nature of it — is actually its strongest selling point.

See, the way I’ve taken to doing my development now is I create two “remote” repositories. First is “origin” which points to a repository managed by Gitosis. Second is a “live” repository that points at a working directory on my production server. That working directory is where my live site actually runs in.

On its own this is handy. As I develop new features, I push my changes to origin. Once my code is ready, and after I run “make test” to verify my site passes all its unit tests, I push to my live repo. At this point I ssh to the server, restart my server processes, and in theory all should be well.

The need for automation

I discovered quickly that in practice this was fraught with error. After a fairly large refactor, I found that the code that worked perfectly well on my development laptop fell over on production. Old or missing libraries, dependancy problems, ownership permissions on files, you name it. My site was down for 2 hours while I tried to resolve these issues. I added more unit tests to my code, but still this wouldn’t have caught these problems.

At this point I decided that a more automated approach was needed. I used a friend’s post-update hook as a template, which simply merged in changes and restarted nginx following a push to live. To this I added a long one-liner, and added the relevant commands to /etc/sudoers with the “nopasswd” option. In the end, the function looks like this:

bounce_fcgi() {
  echo $PWD >&2
  [ -f Makefile ] && make clean >&2
  perl Makefile.PL >&2 && make test >&2 &&
    sudo /etc/init.d/nginx test 2&>/dev/null &&
    sudo /etc/init.d/nginx reload &&
    sudo /etc/init.d/webapp restart >&2

Essentially, after my new changes are merged in, it:

  1. Creates and runs my Makefile
  2. Runs all my unit tests
  3. Tests my new nginx server configuration
  4. Reloads my new nginx configuration
  5. Restarts my FastCGI web application

If any of those steps fails, the full process halts. All the >&2 arguments ensure that the output of these commands are echoed to my local console from the remote server. So when I type “git push live”, all the test output is displayed to me inline. If an error occurs, I can immediately fix it in my local environment and push out a new change without having to log in to my remote server once.

My web application is written in Catalyst, and I use Test::WWW::Mechanize::Catalyst extensively, so I not only unit test my back-end classes, but I test the URLs users actually interact with. It even goes so far as creating and destroying test accounts within my database, so every feature of the website is tested, right down to validating the contents of my robots.txt and sitemap.xml files.

Next Steps

There are still some holes that need to be filled here.

  • If running my tests fail for some reason, I would like to roll-back the remote working directory to the previous version and restart my services, that way the site continues to function under a known-good state.
  • I would like to use WWW::Google::SiteMap::Ping to notify Google, and perhaps other search engines that support XML Sitemaps, that the contents of my site have changed and a reindexing is needed.
  • My site is localized, so I would like to regenerate my PO translation files, and if any strings have changed or are out of date, automatically send an email to my translators with the new POT template file attached.
  • Run my HTML through a spelling checker, to verify I don’t put any typos in any of my pages.
  • Since I try to use caching as much as possible, when a web page’s content has changed, I would like to automatically connect to my Memcached servers to purge the relevant pages from its cache so new versions are immediately available.

A fringe benefit is makes it fun and exciting to write unit tests! Whenever I find an area that isn’t covered, I sit down and crank out more tests to validate new features. I never want to be caught with my pants down on a deployment. Each time it catches something I missed, it makes the whole thing worthwhile.

Does anyone have any thoughts on how I can improve this? Is anyone doing something similar that I can learn from? I’m very excited about my new deployment system, and wish I’d had this ages ago. If there’s anything you want to know more about, please leave a comment.

Harder, Better, Faster, Stronger

On the train ride to work today I was listening to “Daft Punk” on my iPhone while reading my blogs, my usual morning activity. And as my mind wandered, the song “Harder, Better, Faster, Stronger” came on; it’s one of my favourites, as I’m an avid electronic music fan. But since I was reading Ajaxian and several other software industry business blogs at the time, the song seemed strangely appropriate. Everyone is trying to find ways of working harder, developing better software, releasing to market faster, and having a much stronger market position (honestly, this is what was running through my head). This made me think more about what I do to fulfill those 4 business goals, so well voiced by Daft Punk.

“Work it harder”

Personally, I like having a life. I have a wife, hobbies, Open Source projects I like to contribute to. And of course I have my day job where I do more than I’m asked to do, always trying to come up with creative side projects to help make more sales, improve the product, or visualize our usage data in a way to help us build a better product.

So given the cliché that we all have 24 hours in a day, and you have sleep, meals, and commute time involved, how do you optimize your daily work habits to maximize that time?

The way we do this at work is through Extreme Programming or Agile Software Development, both are complementary techniques to eliminate waste from the traditional development cycle. Planning documents, requirements analysis, task assignment and many other wasteful planning processes are typically eliminated.

Coming up with a comprehensive specification before the project is even started is dreadfully wasteful. They typically become a hindrance because no developer can possibly foresee what problems they’ll encounter part-way through. These development practices allow a team as a whole to be flexible, to adapt to changes mid-way through the development cycle, and develop the important bits first. If you haven’t heard about or tried Extreme Programming, I highly recommend you go read up on it. Extreme Programming Explained is a great introduction for people used to more conventional development practices.

The important thing I’ve found is that these practices can even be adapted to single-person teams. You have to alter a few things in order to get it to work for a solo team, but adhering to the principles of “Build the simplest thing that works end-to-end first, then refactor later” is perhaps the most important practice I can suggest. For myself, it’s hard to get lost in developing “Teh Bestest Softwares Evar!” Sometimes you have to take a step back, and question yourself as to what is truly important to your project? Go build that, and don’t waste your time with unnecessary frills.

“Make it better”

Cutting out what isn’t necessary, or building the simplest possible thing doesn’t mean eliminating best practices. Quite the contrary, Extreme Programming pushes very heavily on test-driven development. The ultimate goal is to write your tests before you write a single piece of code. It doesn’t have to be a difficult test, but when you first run it, the test should fail. Once you get a failing test, you write the corresponding code for your project to make the test pass. Then you move on to the next task, rinse and repeat. One of the books on our bookshelf here at work is Test Driven Development, and it’s a great book on learning how using tests to drive your development can really help. I actually need to re-read it again, since my test-writing skills are getting a bit rusty.

By stepping through your project one bit at a time, writing tests as you go, you ensure that any regression in your code is caught by your test coverage. If you’re developing applications that a user will interact with, either through a GUI or a web interface, then automated tests are needed to be able to programatically exercise your interface.

We use Selenium within a series of Windows VMWare instances. Our test runner machine spawns VM instances on our VMWare server, as well as installs fresh copies of our software on our test servers. It then runs our tests using a test harness I developed called Test::A8N to be able to execute our tests, remote-controlling the browsers from within Windows.

Along with all these tests, you need visibility into how your tests are performing. There’s no point in writing all those tests, if you never run them. So we have a nightly process that runs all of our tests in parallel across several servers (so that they can complete in a single night), and then renders an HTML “Dashboard” showing which tests passed and failed, summed up as Red, Yellow or Green. A wide-screen LCD display is easily visible by the entire team, and shows us at a glance just how we’re doing. If you see large patches of red, then you know you’ve got some bug fixing to get to today.

There are other philosophies behind both Extreme and Agile programming, but I won’t get into them here. I highly suggest you read one of these books, as they’re really eye-opening into how you can develop better software without the traditional pain and bureaucracy found in more traditional methods.

“Do it faster”

With all this attention to detail and automated tests, it’s important that your development is still faster than the traditional ways of writing code. Sometimes you have to throw out what isn’t working, and this includes your preciously-written code. If something that served your purposes a year, 6 months, or even 2 weeks ago isn’t serving you anymore, then it’s time for it to go!

Refactoring is the process of taking something that worked before, and altering or culling what doesn’t fit anymore, and writing fresh code to replace it. Many times this means throwing entire chunks of code away. It’s important to not be married to your software. It’s just work, you’ve got plenty more where that came from. Just write what is needed, and nothing more. Don’t over-engineer, since that future “Maybe” feature you’re coding for most likely won’t be needed or used anyway.

Case in point: here at work, a former employee had written a UI framework that all our UI pages were built upon. It handled navigation, Ajax code execution, and so forth. It served its purposes very well at its time, but as features were added it became more and more brittle. Finally, shortly before I was hired, they added localization support, and it broke the camel’s back. Developing new pages was tedious, time consuming, and was littered with unused features that no one knew how or even if they worked.

After a short time of dealing with this, I cut it all out and replaced it with something smaller, lighter, and custom-fit to what it was we needed to do. It cleaned up our URIs so they were easy to understand, sped up development significantly, and gave us a stable platform to continue our subsequent refactoring on top of. All of this, and we eliminated approximately 3,000 lines of code. We’ve settled on the MooTools JavaScript framework for our code, and extend it liberally to build our client-side Ajax UI.

The framework is still holding up a year later, though I know it will have to go through some growing pains later. It’s currently Apache2/mod_perl based, and we’ll eventually be ripping this out and replacing it with a lighter FastCGI wrapper. But the important thing to take from this is that we weren’t so tied to our code that we weren’t afraid to throw it away when it started to smell.

“Makes us stronger”

The end result of all of this is to improve everyone’s lives. The developers have more fun at work, the project gets completed faster, more value is delivered to the customer in a timely fashion, the shareholders make more money, and the business improves.

Therefore I like to find areas in which I can help the company. Partly because there are many more interesting problems than just what I work with in my day job. So I write tools to help the salespeople sell more software. I write tools to help the product managers identify what our customers need and use. I’m currently working with our documentation team to find new ways to represent our documentation to make it more visible to our customers when they need help.

Ultimately, if you think outside the box and thing about your project from your customer’s perspective, it opens up a much bigger world for you to find your niche. Forget about what your job title is or what you’re being tasked to do. What problems do your customers have? If you had to use the software you’re developing on a day-to-day basis, would you enjoy yourself? Or would you pick up the phone and try to find a new vendor?

There’s a great rant over on Seth Godin’s blog titled “Just doing my job.” Don’t hide behind your job title as an excuse to justify your company’s actions, and don’t use it as an excuse to be lazy. Expand your skills, help your customers solve real problems, and don’t write bad software.

“More then ever after, our work is never over”

Most importantly than anything, never blindly accept that what you’re doing now is the best there can be. Newer technologies are developed, better strategies for developing or planning software can be invented, and people become complacent. Always read, always learn. Attend conferences, teach others what you know so you can understand your craft better.

So it’s very fitting that the last phrase in Daft Punk’s “Harder, Better, Faster, Stronger” is this: “Our work is never over”.