The AIR Install Badge is a very handy little flash application for delivering AIR applications to your users via the web. The badge allows your users to download and install both your application and the Adobe AIR runtime. Additionally, the install badge will automatically prompt users to upgrade if a previously installed version is detected. At Gorilla Logic, we use the AIR Install Badge on the FlexMonkey download page (free registration required).

Alas, flash is opaque to analytics. We have no idea what our users are doing inside the AIR Install Badge application. Are they installing? Or upgrading? No problem, we just need to write some code…

The Code

Using flash’s ExternalInterface, we can manually push the data out of flash and into javascript. Once we have the data in javascript, we have total control. One option is to use google analytics to store our badge data. In the case of FlexMonkey, we send the badge data along with the user’s credentials to our CRM platform, SalesForce.com.

Step 1: First, open AIRInstallBadge.as and add this to the top:

import flash.external.ExternalInterface;

Step 2: Next, add the ExternalInterface call to the top of the handleActinClick() function in AIRInstallBadge.as:

protected function handleActionClick(evt:MouseEvent):void {
    if (action == 'install' || action == 'upgrade') {
        //send data to js
        ExternalInterface.call('badgeJS',action);
    }
    ...
}

Since I only care about the install or upgrade actions, I’ll only send those out to javascript. Re-compile the badge and deploy.

Step 3: Last, add the badgeJS() javascript callback to the page containing the badge and do whatever you want with the incoming badge data:

function badgeJS(action) {
    //do metrics here...
    alert('badge action=' + action);
}

Conclusion

With an hour of effort, and a very small amount of code, we’ve managed to get the useful metrics of installs and upgrades out of the AIR Install Badge and into our analytics engine of choice. A job well done.

When we launched the new and improved Gorilla Logic website, we decided to bring all our open source projects together under one roof. In order to migrate all things FlexMonkey back to our website, we need to get our forum data migrated out of Google Groups. Alas, Google doesn’t provide any way to export data from Google Groups. The only way to preserve the amazing contributions from the FlexMonkey community was to scrape Google Groups. So that’s just what we did.

With a very minimal amount of PHP, I was able to walk the entire FlexMonkey Google Group, scrap all the topics (aka threads) and all the posts inside each thread. The first step was to build a generic scraper class that grabs an html page (using cURL) and parses out all unique outbound links.

Here’s the code for the Scraper class:

class Scraper {
    private $url = '';
    public $html = '';
    public $links = array();
 
    public function __construct($url) {
        $this->url = $url;
    }
 
    public function run() {
        $this->html = '';
        $this->links = array();
 
        //scrape url & store html
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $this->url);
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $this->html = curl_exec($ch);
        curl_close($ch);
 
         //parse html for all links
        $matches = array();
        preg_match_all('#<a.*?href\s*=\s*"(.*?)".*?>(.*?)</a>#i', $this->html, $matches);
 
        if ($matches !== false && count($matches) == 3) {
            for ($i = 0; $i < count($matches[1]); $i++) {
                $href = $matches[1][$i];
                $val = $matches[2][$i];
 
                //unique links
                if (!array_key_exists($href, $this->links)) {
                    $this->links[$href] = $val;
                }
            }
        }
    }
}

In the run() method, cURL is used to grab the html. Next, a regular expression is used to match all outbound links. The links are are stored in a hash, while making sure they point to unique urls.

Built on top of the generic Scraper class is a specialized Google Groups scraper class, aptly named GoogleGroupsScraper. For a given Google Group, the url of the main page (containing a list of most recent topics) is:

http://groups.google.com/group/[GROUP]/topics

And the url of a single topic (aka thread) is:

http://groups.google.com/group/[GROUP]/browse_thread/thread/[THEAD_ID]#

Where [GROUP] is the name of the Google Group, and [THREAD_ID] is some alphanumeric id. Most importantly, at the bottom of the main page is an Older » link that points to the next page of topics. The GoogleGroupsScraper exploits this to spider the entire group, recording topic title and topic url as it walks each page.

Next, each individual topic page is scraped by the GoogleGroupsTopicScraper class and parsed into a list of posts with author name, date, timestamp, etc. The topic scraper uses various regular expressions to extract and massage the html to extract the different parts of each post. In particular, the post body needs a lots of work to strip out any Google Groups specific links and code.

Lastly, the topics and their posts are assembled into an XML document with a nice big CDATA block around the post body to preserve the html content.

Here’s some sample output from the scraper:

<?xml version="1.0" encoding="UTF-8"?>
<scrape group="flexmonkey">
  <topic>
    <title>FlexMonkey User Group is now located at www.gorillalogic.com/flexmonkey!</title>
    <link>http://groups.google.com/group/flexmonkey/browse_thread/thread/fe9ed66bf56db88e#</link>
    <posts>
      <post idx="0">
        <author>Stu</author>
        <email>stu.st...@gorillalogic.com</email>
        <date>February 10, 2010 21:17:52 UTC</date>
        <timestamp>1265836672</timestamp>
        <body>
<![CDATA[
<p>People of FlexMonkey, <p>We have migrated the FlexMonkey discussion forum to <a href="http://www.gorillalogic.com/flexmonkey">http://www.gorillalogic.com/flexmonkey</a>. Please note that you will need to re-subscribe to the new forum to continue receiving FlexMonkey discussion messages. <p>-Stu <br>
]]>
        </body>
      </post>
    </posts>
  </topic>
  <topic>
    <title>Record button clicks based on Ids instead of names?</title>
    <link>http://groups.google.com/group/flexmonkey/browse_thread/thread/4f079b1959374f53#</link>
    <posts>
      <post idx="0">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 9, 2010 23:44:44 UTC</date>
        <timestamp>1265759084</timestamp>
        <body>...</body>
      </post>
      <post idx="1">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 10, 2010 00:05:44 UTC</date>
        <timestamp>1265760344</timestamp>
        <body>...</body>
      </post>
      <post idx="2">
        <author>Gokuldas K Pillai</author>
        <email>gokul...@gmail.com</email>
        <date>February 10, 2010 00:16:34 UTC</date>
        <timestamp>1265760994</timestamp>
        <body>...</body>
      </post>
      <post idx="3">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 10, 2010 01:18:42 UTC</date>
        <timestamp>1265764722</timestamp>
        <body>...</body>
      </post>
...

Finally, there is a very simple PHP driver for the scraper that runs the scraping process:

require_once('GoogleGroupsScraper.class.php');
 
$scraper = new GoogleGroupsScraper('[GROUP]');
$scraper->run();
 
print $scraper->getXML();

And you run it as usual:

php scrape.php > output.xml

Just enter the name of the Google Group you wish to scrap, and away you go. Here are a couple of notes to help you along:

1. [GROUP] is the group name as it appears in the url, so no spaces, etc.
2. It’s not fast, so be patient, or modify the scraper code to generate some intermediate output.
3. Via a browser, Google Group displays 30 topics per page, but via PHP & cURL you only get 10. Probably some Cookie or User Agent magic going on.
4. Not much error handling. The error handling that exists isn’t very good. It will break.
5. Good luck!

Please download the code and use it however you wish. Hopefully, putting the code online and writing this post will save someone else some time when migrating data off Google Groups.

Files

• GoogleGroupsScraper.tgz

We’re pretty big on testing at Gorilla Logic, and in the world of Flex that usually means using FlexMonkey to test the UI and using FlexUnit to test the code. Alas, it is a huge pain in the ass to correctly test the many async objects and services inherent in any Flex app with FlexUnit. Enter Fluint, an superior Flex unit tesing framework by the cool guys at digital primates (no relation). Fluint is the heir apparent to take over the unit testing crown from the venerable FlexUnit. So let’s take Fluint and its enhanced async testing support for a spin.

Service Layer

First, assume we have a nice service layer in Flex that talks asynchronously to our backend. Just something simple to start:

public class MyService {
    public function getSomething(result:Function, fault:Function):AsyncToken {
        //call the backend
        var token:AsyncToken = backend.getSomething();
 
        //wire the callbacks to the result
        token.addResponder(new AsyncResponder(result, fault, token));
 
        return token;
    }
}

In this example, our service only has one method, getSomething() that takes two callback functions. It simply calls the backend method, wires up the callbacks (which get called when the backend method returns a result), and returns the token. It is absolutely critical that our callback-powered service method return the AsyncToken. The reason for this will become apparent.

We might use our service like this:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application
        xmlns:mx="http://www.adobe.com/2006/mxml"
        creationComplete="complete()">
 
    <mx:Script>
    <![CDATA[
        import com.saturnboy.services.MyService;
        private var service:MyService;
 
        private function complete():void {
            service = new MyService();
            service.getSomething(resultHandler, faultHandler);
        }
 
        private function resultHandler(result:Object, token:Object=null):void {
            lbl.text = result.result.name;
        }
 
        public function faultHandler(error:Object, token:Object=null):void {
            lbl.text = 'fault';
        }
    ]]>
    </mx:Script>
 
    <mx:Label id="lbl" text="initial" />
</mx:Application>

We make a call our service, and then use the callbacks to alter the UI however we want depending on the result. In common usage, the fact that our service returns an AsyncToken is worthless, it might as well return void. So, why did I say this is critical? Throw Fluint testing into the mix and it’s “Show em what’s behind door number 2, Johnny!”

Fluint Testing

Fluint provides two different async wrapper methods: asyncHandler and asyncResponder. The first allows a test to be wired to an async method by events, the second allows a test to be wired to an async method by a responder. Since the service method we’re trying to test doesn’t throw any events, we’ll need to use the latter. So inside a Fluint test case, we have our test method:

public function testGetSomething():void {
    //call service with dummy callback
    var token:AsyncToken = service.getSomething(dummyResult, dummyFault);
 
    //create async test responder
    var responder:IResponder = asyncResponder(
            new TestResponder(testHandler, faultHandler), 1000, token);
 
    //wire test responder as 2nd callback
    token.addResponder(responder);
}
private function testHandler(result:Object, passThroughData:Object):void {
    assertEquals('something', result.result.name);
}

The trick is to wire a second callback via Fluint’s asyncResponder helper that actually does the testing, and just give the original service call some dummy callbacks. Note that if the service method didn’t return its AsyncToken there would be no way to wire a second callback. The Fluint async helper do two import operations: they handle the event or call the callback AND they mark the test method as an async method so the result is correctly reported by the test harness. You can read more about Async Testing in Fluint’s wiki. The rest of Fluint is your standard chain of crap borrowed from JUnit: test runner, test suites, and test cases.

Digging Deeper: It is equally critical to use dummy callbacks in the original service method call because in a failure situation they will cause Flash Player to error out instead of being caught by Fluint and reported as a test failure.

Files

The complete code is up on GitHub here: test_fluint_async. The code is MIT licensed and includes a working fluint.swc (see below) plus a mock async backend (so timeouts and faults are easily testable).

Alas, Fluint v1.1.0 was built incorrectly and is missing the TestResponder class (see issue 35). So if you want to try out Fluint in your project, I recommend you grab it from svn and build the swc yourself. Hopefully, this will all be fixed in the next release.

UPDATE: Fluint v1.1.1 was release on May 1, 2009 and fixes this issues and a few others. Download it here.