The AIR Install Badge is a very handy little flash application for delivering AIR applications to your users via the web. The badge allows your users to download and install both your application and the Adobe AIR runtime. Additionally, the install badge will automatically prompt users to upgrade if a previously installed version is detected. At Gorilla Logic, we use the AIR Install Badge on the FlexMonkey download page (free registration required).

Alas, flash is opaque to analytics. We have no idea what our users are doing inside the AIR Install Badge application. Are they installing? Or upgrading? No problem, we just need to write some code…

The Code

Using flash’s ExternalInterface, we can manually push the data out of flash and into javascript. Once we have the data in javascript, we have total control. One option is to use google analytics to store our badge data. In the case of FlexMonkey, we send the badge data along with the user’s credentials to our CRM platform, SalesForce.com.

Step 1: First, open AIRInstallBadge.as and add this to the top:

import flash.external.ExternalInterface;

Step 2: Next, add the ExternalInterface call to the top of the handleActinClick() function in AIRInstallBadge.as:

protected function handleActionClick(evt:MouseEvent):void {
    if (action == 'install' || action == 'upgrade') {
        //send data to js
        ExternalInterface.call('badgeJS',action);
    }
    ...
}

Since I only care about the install or upgrade actions, I’ll only send those out to javascript. Re-compile the badge and deploy.

Step 3: Last, add the badgeJS() javascript callback to the page containing the badge and do whatever you want with the incoming badge data:

function badgeJS(action) {
    //do metrics here...
    alert('badge action=' + action);
}

Conclusion

With an hour of effort, and a very small amount of code, we’ve managed to get the useful metrics of installs and upgrades out of the AIR Install Badge and into our analytics engine of choice. A job well done.

This post is a follow-up to my previous post, The Schizophrenic Programmer. I must have struck a chord, because I got a ton of fantastic feedback not just on this blog, but also on Reddit and DZone.

After further reflection, I still agree with what I said and my conclusions. Moving forward, I plan to learn fewer languages and less syntax. My daily pain is that too much of my knowledge is too tightly bound to the implementation details, and when I switch languages I can’t bring my solutions to bear. It sucks, and it makes me feel stupid. So the simple fix is to put my energy into learning more unbound concepts (like my soft skills of speaking and writing) and more easily transferable concepts (like APIs).

Analytics

I had a huge bump in traffic mostly due to Reddit and DZone that basically crushed my regular traffic down into the noise. My poor slice at Slicehost was taking a beating, so I quickly installed the Hyper Cache plugin. It’s dead simple and it really works. On Sunday morning, my load went from 0.2 to 0.02 instantly.

I also tracked my twitter analytics via bit.ly. As a side note, you can just put a plus “+” at the end of any bit.ly url to see it’s analytics. So, since the original post was shortened to http://bit.ly/cR6cnY, the analytics are available here: http://bit.ly/cR6cnY+.

Commentary

Lots of stuff was said, but I just wanted to touch on some of the key points.

1. I feel your pain – thanks. I find it comforting to know I’m not alone.
2. definition of schizophrenia – a couple of comments mentioned that my post had nothing to do with schizophrenia. First, I suggest you look it up: “contradictory or antagonistic qualities or attitudes”. My Erland knowledge is definitely in opposition with my Java, which is in opposition with my Actionscript, etc. If you don’t have this problem, good on you. Second, there’s thing called artistic license and if you are a writer you get to use it. I picked a sensational word for a reason. I welcome criticism of the choice, but it’s silly to take umbrage with its validity.
3. short sighted – my plan was called out as short sighted a couple of times. I tend to think of life and plans like this… There are three choices when it comes to plans. First, you can have no plan, and just drift along. Sometimes you get smashed into the rocks, and other times you wash up on a tropical paradise with beer, virgins, and fiber broadband. Second, you can have a short term plan that gets revisited and revised as things change. Third, you can have a short term plan, but be so unbelievably naive that you think it is actually some awesome long term plan. Life is agile. All plans are short term.
4. embrace and extend – a few people said don’t fight it and embrace the insanity. I really lust after new stuff, so it was easy to try to the embrace-the-insanity method first. Alas, it didn’t work out and I think it actually made matters worse. Thus, I decided to go with a new plan to fight against the insanity.
5. python – a couple people mentioned I should switch to python. Yes I’ve written many thousands of lines of python, but none in 2010 so far.

Thanks for reading. As always, I love feedback.

Hi, my name is Justin and I have a problem. I’m insane!¹

Being insane really sucks. But I’ve got a plan to get back on track. First, I’ll dole out some blame. Then, I’ll give a little background. Next, I’ll really dig into my current insanity. And finally, I’ll layout my plan for the future.

The Blame

I like to curse. And when I’m mad, I find it definitely helps to calm the rage. So to begin, I’d like to give a big fuck you to Andy Hunt and Dave Thomas, the pragmatic programmers. I love you guys but you get at least some of the blame for making me insane. Next, I’d like to give some blame to my wonderful employer, Gorilla Logic, and their penchant for landing such a diverse set of inspiring projects. You suck for building a candy store and filling it with the sweetest treats.

Unfortunately for me, the lion’s share of the blame is mine and mine alone. I’m insane today because of my lust for the new coupled with my desire to be great. I want to be a great dad, a great husband, and a great employee. Who wants to be average?

It’s just not possible that I’m alone. Who doesn’t like cool new stuff? Who doesn’t want to be great? There must be thousands, if not tens of thousands, of fellow developers suffering from insanity that goes undiagnosed. So this tale goes out to you, my functionally insane brethren. Together we can heal ourselves.

Who Am I?

I’d like to give a little background before I get in too deep with my current insanity. I’m a developer. I get paid to write code (woo hoo!) to solve problems. Blah, blah, blah, if you just fill in the rest with a bunch of geek stereotypes you won’t be far off. Yes, I like to IM my coworkers instead of just walking over and talking with them. Yes, I read slashdot. Yes, I read sci-fi. Yes, I played lots of Dungeons & Dragons.² Et cetera…

I consider myself a smart guy, just like every single developer I’ve ever met. I stay hungry and motivated to learn something new every day, every week, every year. Thanks to Andy and Dave, I learn one new language every year. And most importantly, I try very hard to listen or at least listen more than I argue. Sometimes it doesn’t work out so well, but that’s another tale…

How I Became Insane

I’m a big fan of “right tool for the job” in both life and development. In the realm of development, the concept of right tool for the job has metamorphosized into right language for the job, aka polyglot programming. On the face of it, polyglot programming has a lot of advantages (here’s a good talk by Dean Wampler), but it is one of the root causes of my insanity.

At Gorilla Logic, we are regular practitioners of polyglot programming. Our typical enterprise RIA project uses Java on the backend and Flex on the frontend. Of course, the mix of languages doesn’t stop there. I develop for the web, so when you talk about websites, WordPress and Drupal are immediately in the conversation, and thus PHP. And since a rich client-side experience is critically important, HTML, CSS, and Javascript are drawn into the mix. In the mobile world, Blackberry and Android are thankfully mostly Java, but anything Apple takes me into the non-GC’d world of Objective-C.

I’ve also been diligently learning my one language a year for a while now. And when you combine that with work, it has simply become too much. I’ve become scattered, my mind has become messy. Looking at the syntax level, the symptoms are acute. I can’t write a loop. I can’t match a pattern. I can’t build array operations. My context switching pain is severe, it can literally take hours before I get back up to speed with the language at hand.

But it’s not just the low-level stuff anymore. I’m finding that even at the highest levels of architecture, clean thought is hard to achieve. Different languages tend to espouse different patterns and paradigms that directly impact architecture level decisions. For example, if I wanted to write a server in PHP I might consider using OS support like fork and cron. In Java, I’m off in thread land. Erlang it’s processing and messaging. Scala is all about actors.

I find myself mixing paradigms and doing stupid stuff, like trying to fake Erlang’s message passing in PHP by multiple scripts communicating by repeatedly touching rows in a database. Dumb, but becoming harder to avoid as everything slowly bleeds together in my mind. I’m in the mud and I desperately what to be clean again.

My Current Insanity

Here’s a broad sample of what I’ve done lately (as in the last month or so) for work and personal projects:

Erlang – Work

Freshened my Erlang, wrote a sample custom module for ejabberd, played with Nitrogen.

Flex/AS3 – Work

I helped out a little bit with FlexMonkey 1.0, including the fuzzy pixel bitmap comparison support.

Flex/AS3 – Work

ScrumMonkey is another Gorilla Logic open-source project. I upgraded everything to work with Flex 4 and LCDS 3.0.

Java – Work

Fun with Spring, Hibernate, and Spring Actionscript. Can’t say any more about the project.

Javascript – Work

I helped develop FlexMonkium, a Selenium-to-FlexMonkey testing tool that enabled automated functional testing of hybrid web apps. I used Javascript and XUL.

Objective-C – Justin

Wrote and published a US states and capitals memorization helper app. Read more or get it from the app store.

Objective-C – Work

iPhone and iPad. Cool stuff. Can’t talk about it yet.

PHP – Justin

Released Viceroy, a one-column WordPress theme with a dash of pink.

PHP – Work

We needed to get our community feedback out of Google Groups and into our FlexMonkey Forum. I wrote a simple scrapper in PHP and blogged about it.

PHP – Justin

Various WordPress and Drupal side projects just for fun.

Ant, Maven, Bash, Rake – Work & Justin

Scripted a bunch of stuff…

And that’s just my current insanity: the languages that I’ve touched lately. But I’m not special! At least, not in this case. The list is similar for many co-workers, and many of my friends that are web developers.

The Sensible Plan

If I’m to blame, then I’ve got to be the one to fix it. So here’s my plan:

• Forget one language every year – Forget the syntax, forget the weirdness, and forget the whole ecosystem (frameworks, tools, community). Say goodbye and don’t look back. But before you leave a language behind, pick one core concept, one of the things the language does right, and take that with you. For example, when I forget Erlang, I’ll take the concept of concurrent programming with me. So I’ll remember stuff like immutability, message passing, and processes. When I forget Ruby, I’ll take DSLs with me. Ruby does lots of stuff well, like blocks, mixins, terse syntax, and meta-programming, but I always enjoyed using all the great DSLs the best, so that’s what I’ll remember.
• Don’t learn one programming language every year – Yep, I’m going against Andy & Dave. So no Haskel, Clojure, Duby, Go, or Reia for me this year. I’m obviously feeling a little full on languages right now, so I’ll take a break for a few years.
• Learn one or more APIs every year – Since Web 3.0 is all about APIs, I might as well spend some time learning more of them. The big boys are obvious: Twitter, Facebook, Flickr, Google Maps, but there are plenty more that are really interesting. From the practical side, every single web project either integrates with other APIs, wants their own API, or has some set of requirements that force a good SOA architecture (aka some internal set of APIs).
• Upgrade my soft skills – Slinging code is fun, but it can’t be the only thing I do if I want to be a great employee. So instead of picking up all these little bits of shiny tech, I’m going to going to focus my lust for the new on upgrading my soft skills. I write this blog to improve my writing, but I haven’t spoken at a big conference since CLEO/QELS in 1999 and I was really bad. So I hope to do some public speaking in 2010.

Basically, I’m hoping to elevate my kung-fu by going for more depth of knowledge. Then, I’ll do my best to control the relentless expansion of breadth. New and shiny is no longer sufficient. Talking things through with co-workers and writing this post is a great first step into the future. I’m already feeling optimistic about my path.

Footnotes

1. I’m not really insane, at least I don’t think so. I’m just functionally insane. For a glimpse into the mind of a real schizophrenic, I highly recommend Is There No Place On Earth For Me? by Susan Sheehan. It won the non-fiction pulitzer prize in 1983, so it’s a little bit dated now, but an unimaginable story.
2. Actually, my friends and I played a lot of Rolemaster (wikipedia) and Champions (wikipedia) which are superior role playing games. If you are a true fan, you’ll understand. If not, the image of boys with dice conjured by Dungeons & Dragons is a good takeaway.

UPDATE: I wrote a follow-up post to address all the great feedback: Schizophrenic Following

When we launched the new and improved Gorilla Logic website, we decided to bring all our open source projects together under one roof. In order to migrate all things FlexMonkey back to our website, we need to get our forum data migrated out of Google Groups. Alas, Google doesn’t provide any way to export data from Google Groups. The only way to preserve the amazing contributions from the FlexMonkey community was to scrape Google Groups. So that’s just what we did.

With a very minimal amount of PHP, I was able to walk the entire FlexMonkey Google Group, scrap all the topics (aka threads) and all the posts inside each thread. The first step was to build a generic scraper class that grabs an html page (using cURL) and parses out all unique outbound links.

Here’s the code for the Scraper class:

class Scraper {
    private $url = '';
    public $html = '';
    public $links = array();
 
    public function __construct($url) {
        $this->url = $url;
    }
 
    public function run() {
        $this->html = '';
        $this->links = array();
 
        //scrape url & store html
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $this->url);
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $this->html = curl_exec($ch);
        curl_close($ch);
 
         //parse html for all links
        $matches = array();
        preg_match_all('#<a.*?href\s*=\s*"(.*?)".*?>(.*?)</a>#i', $this->html, $matches);
 
        if ($matches !== false && count($matches) == 3) {
            for ($i = 0; $i < count($matches[1]); $i++) {
                $href = $matches[1][$i];
                $val = $matches[2][$i];
 
                //unique links
                if (!array_key_exists($href, $this->links)) {
                    $this->links[$href] = $val;
                }
            }
        }
    }
}

In the run() method, cURL is used to grab the html. Next, a regular expression is used to match all outbound links. The links are are stored in a hash, while making sure they point to unique urls.

Built on top of the generic Scraper class is a specialized Google Groups scraper class, aptly named GoogleGroupsScraper. For a given Google Group, the url of the main page (containing a list of most recent topics) is:

http://groups.google.com/group/[GROUP]/topics

And the url of a single topic (aka thread) is:

http://groups.google.com/group/[GROUP]/browse_thread/thread/[THEAD_ID]#

Where [GROUP] is the name of the Google Group, and [THREAD_ID] is some alphanumeric id. Most importantly, at the bottom of the main page is an Older » link that points to the next page of topics. The GoogleGroupsScraper exploits this to spider the entire group, recording topic title and topic url as it walks each page.

Next, each individual topic page is scraped by the GoogleGroupsTopicScraper class and parsed into a list of posts with author name, date, timestamp, etc. The topic scraper uses various regular expressions to extract and massage the html to extract the different parts of each post. In particular, the post body needs a lots of work to strip out any Google Groups specific links and code.

Lastly, the topics and their posts are assembled into an XML document with a nice big CDATA block around the post body to preserve the html content.

Here’s some sample output from the scraper:

<?xml version="1.0" encoding="UTF-8"?>
<scrape group="flexmonkey">
  <topic>
    <title>FlexMonkey User Group is now located at www.gorillalogic.com/flexmonkey!</title>
    <link>http://groups.google.com/group/flexmonkey/browse_thread/thread/fe9ed66bf56db88e#</link>
    <posts>
      <post idx="0">
        <author>Stu</author>
        <email>stu.st...@gorillalogic.com</email>
        <date>February 10, 2010 21:17:52 UTC</date>
        <timestamp>1265836672</timestamp>
        <body>
<![CDATA[
<p>People of FlexMonkey, <p>We have migrated the FlexMonkey discussion forum to <a href="http://www.gorillalogic.com/flexmonkey">http://www.gorillalogic.com/flexmonkey</a>. Please note that you will need to re-subscribe to the new forum to continue receiving FlexMonkey discussion messages. <p>-Stu <br>
]]>
        </body>
      </post>
    </posts>
  </topic>
  <topic>
    <title>Record button clicks based on Ids instead of names?</title>
    <link>http://groups.google.com/group/flexmonkey/browse_thread/thread/4f079b1959374f53#</link>
    <posts>
      <post idx="0">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 9, 2010 23:44:44 UTC</date>
        <timestamp>1265759084</timestamp>
        <body>...</body>
      </post>
      <post idx="1">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 10, 2010 00:05:44 UTC</date>
        <timestamp>1265760344</timestamp>
        <body>...</body>
      </post>
      <post idx="2">
        <author>Gokuldas K Pillai</author>
        <email>gokul...@gmail.com</email>
        <date>February 10, 2010 00:16:34 UTC</date>
        <timestamp>1265760994</timestamp>
        <body>...</body>
      </post>
      <post idx="3">
        <author>Shilpa</author>
        <email>shilpa.g...@gmail.com</email>
        <date>February 10, 2010 01:18:42 UTC</date>
        <timestamp>1265764722</timestamp>
        <body>...</body>
      </post>
...

Finally, there is a very simple PHP driver for the scraper that runs the scraping process:

require_once('GoogleGroupsScraper.class.php');
 
$scraper = new GoogleGroupsScraper('[GROUP]');
$scraper->run();
 
print $scraper->getXML();

And you run it as usual:

php scrape.php > output.xml

Just enter the name of the Google Group you wish to scrap, and away you go. Here are a couple of notes to help you along:

1. [GROUP] is the group name as it appears in the url, so no spaces, etc.
2. It’s not fast, so be patient, or modify the scraper code to generate some intermediate output.
3. Via a browser, Google Group displays 30 topics per page, but via PHP & cURL you only get 10. Probably some Cookie or User Agent magic going on.
4. Not much error handling. The error handling that exists isn’t very good. It will break.
5. Good luck!

Please download the code and use it however you wish. Hopefully, putting the code online and writing this post will save someone else some time when migrating data off Google Groups.

Files

• GoogleGroupsScraper.tgz

In Part 1, I covered the basic setup of LiveCycle Data Services 3 Beta sitting on a MySQL database. Now, I’ll get into the Flash Builder side of things, and talk about the Modeler plugin and model driven development.

Setup Modeler Plugin

1. Get Flash Builder 4 Beta (download)
2. Get the LCDS Modeler plugin (download)
3. Install the Modeler plugin:
   • Unzip to a temp folder, which will create a plugins folder
   • Copy the plugins folder’s contents directly into Flash Builder’s plugins folder (typically located in /Applications/Adobe Flash Builder Beta/plugins if you are on a Mac)
   • Restart Flash Builder

Model Driven Development

1. Create a new Flex project
   • In Flash Builder, right-click and say New > Flex Project
     
     enlarge
   • Choose J2EE for Application Server Type, and select LCDS
   • Click Next to configure your server
     
     enlarge
   • Uncheck Use Default Location, and fill in your root folder, url, and context (see screenshot)
   • Click Validate Configuration (this will fail if LCDS is not running), and then Finish
2. Create a new data model
   • Select the Data/Services tab in the bottom window
   • Click the Edit Active Data Model icon in the tab’s menubar (see screenshot)
     
     enlarge
   • This will create a new data model for your project, and bring up MyApp.fml in the Modeler’s design view (see screenshot)
   • To see the file in the Package Explorer, click the Filters icon (see screenshot), then uncheck .* and .model
3. Connect to the server
   • Switch to the Adobe Data Model perspective
   • In the RDS Dataview window, click the RDS Configuration icon
     
     enlarge
   • Set the Context Root to myapp (because our server is running at http://localhost:8080/myapp/)
   • Click Test Connection, and then OK
4. Verify the server’s connection to MySQL
   • In the RDS Dataview window, expand until we can see our tables
     
     enlarge
   • Right-click on a table, and select Show Table Contents
     
     enlarge
5. Edit the model
   • Just drag tables from the RDS Dataview to add them to the model
     
     enlarge
6. Browse the generated services and code
   • Once the model is updated, the builder automatically generates standard CRUD services. Switch back to the Flash Builder perspective, and you can view them in the Data/Services tab
     
     enlarge
   • The generated code is available in the Package Explorer. Files you don’t want to touch are prefixed with an underscore. The other files, like Player.as and PlayerService.as are available for you to customize.
     
     enlarge

More Docs

1. LCDS Modeler Guide (zip)

Conclusion

Once again, that’s it. I’ll have to learn something new about LCDS before I write more.