MacPorts - my new best friend

Perhaps it is just that I'm getting older, but I've resisted using package managers on Linux and Mac OS X for some time. I dunno, I guess I preferred to download the source and build it myself. What a sucker I am for punishment!

I think that I had a similar resistance to Apache Maven originally - I just didn't want the learning curve of a new tool.

I'd got to the point with a C++ project that I was fed up managing, building and sorting out the dependencies of various C/C++ based libraries. The catalyst for me was when I started using Cairo, a 2D vector and imaging library (which is great!). Cairo had many dependencies and the project recommended a number of package managers for my OS X build environment.

I ended up with MacPorts and haven't looked back. Managing those dependencies just got a whole lot easier. MacPorts was apparently started by a group of Apple engineers and it grew from there. At the time of writing there are now over 6,000 packages available through MacPorts.

MacPorts likes to put everything in /opt by default. For some bizarre reason I was against using /opt as my root folder for all MacPorts derived packages - talk about resisting change for no good reason! However I got over that and must take a leaf out here and try to resist such time consuming temptations. Who really cares where MacPorts puts stuff so long as at the end of the day your dependencies are being managed and your applications build?

The only thing I found myself changing with the default configuration was the /opt/local/etc/macports/variants.conf file. I wanted +universal as a default. My projects tend to build both 32 bit and 64 bit targets and so I needed MacPorts to do this by default (I kept forgetting to specify the variant on the command line).

All good. MacPorts is great and is another tool that helps me focus on the problem at hand.

Write less code

I think that old adage of "Sorry I wrote you such a long letter, if I had more time I'd write you a shorter one" certainly applies to software development as well. If you spend a great deal of time learning how to write less code by using more off-the-shelf toolkits and frameworks you will write less code in the end. Sounds obvious, but I still see developers re-inventing the wheel far too often.

The following code requires a steep learning curve around Java, Apache projects, iCal4j (a great calendaring toolkit for iCalendar) Spring, Spring DAO and Maven; and probably many more things. Look how straightforward the expression is though:

from(calendarEventEmailInboundEndpoint).to(
  "bean:calendarAttachmentTranslator").to(
  "bean:calendarEventTranslator").to(
  "bean:calendarEventDAO?method=save");

The above Java code shows a Camel route that reads emails from a pop3 or imap inbox (depending on how calendarEventEmailInboundEndpoint is configured), extracts iCalendar attachments from one or more emails, translates the iCal4j representation into a model form for persistence, and finally persists the calendar event(s) in a database.

Pretty sophisticated stuff for 4 lines of code. :-)

Maven Release Plugin - My new friend

One tool that slipped my radar until only this week was the Maven Release Plugin. If you are a Maven user and you release what you produce with it (I would hope so!) then you should take a look at this plugin.

My applications tend to be containers for the many pre-fabricated components of other projects I have. For instance I have 19 projects in my Eclipse workspace where 3 of them are the containers - if you like, the applications themselves.

Having lots of projects means that it can be easy to miss updating the version numbers for them when it comes time to perform a release; particularly missing that a project is being depended on as a snapshot.

In summary two big reasons to use the Maven Release Plugin are:

• it ensures that there are no snapshot dependencies; and


• it updates your pom's version number automatically.

In order to use the plugin I found myself having to add an scm element, an element for the plugin itself and a distributionManagement element.

The scm element let's the plugin know where to tag a release. Each time you make a release the plugin will create a tag for it in your source code repository. I was doing this as a manual task prior to the plugin so this feature is great.

I configured the plugin to ignore deploying my source code and javadoc to the repository. I actually had a problem with the maven source plugin given spaces contained in my path. I also didn't need to deploy the source code as I have it within my scm repository and I don't intend releasing the source to other parties.

Lastly you need to let the deploy plugin know how to deploy your project's artifacts in a maven repository i.e. the jar files in my case. The release plugin will invoke the deploy goal by default (actually it does a deploy and then "site deploy"). My projects generally rely upon the default goal with exception to those "application" projects. The application projects simply invoke the assembly plugin as I'm interested in producing a tar.gz file of the entire application. I therefore don't need to specify repository deployment details in my pom for these application projects as there is none.

All in all the maven release plugin saves me time and improves my release build quality.

Client/server or consumer/provider

Wow, I can't believe that it is now September and I've not posted anything since July. In a nutshell I landed a new airport customer (Subang Skypark - the Kuala Lumpur domestic airport) and so life has been terribly busy. As a result there's a lot of software development to be discussed since my last post. However I'll start with this little one as a result of an interesting conversation today.

I designed an interface today that describes a set of remote services. As the interface is a consumer of those services I came up with the name XXXConsumer (I didn't really use XXX though). A friend of mine asked why not call it XXXClient? I had to really think about that.

Two seconds later I blurted out that client and server had become an overloaded term and that the hip terms are now consumer and provider respectively. In other words I didn't really know why I preferred to use the word consumer and provider and put it down to the time I've spent with JMS and Apache Camel. These technologies often use the terms consumer and provider.

Wikipedia offer a nice definition of client and server that effectively implies that these names are used for fairly course-grained objects such as applications and systems. My assertion having thought about this a little more is that consumers and providers are used in a finer-grained context. By finer-grained I refer to the software components found within my application.

In addition client and server are certainly overloaded terms and typically imply that an application or system has just this one role. By contrast I often see that components are both consumers and providers of services. Perhaps this is another difference.

Thank you for allowing me to client^H^H^H^H^Hconsume your time!

Replacing jsvc for Java daemons

For some time I have been using the Apache Commons Daemon project to run and maintain my Java based services. jsvc did/does a reasonable job but I start to worry when a project has not been touched for a while.

In addition I am in the midst of deploying a Java based service on Mac OS X and so I am in the world of launchd. I wanted a better way of controlling Java based services; particularly one that did not fork multiple processes given launchd's garbage collection.

The principal issue with a Java service running in the background is that you need to have it respond asynchronously to various singals; particularly SIGTERM which of course is issued when the OS is being shutdown. My service needs to shut down gracefully.

Enter in a feature of Java 1.4.2 that I did not realise existed: Shutdown Hooks. In essence Shutdown Hooks provides your Java application an opportunity to respond to the application quitting.

Followers of this blog know that I am an Apache Camel addict. You will not therefore be surprised to find that what follows is an example of how to start up and shutdown a Camel context using Java's Shutdown Hooks.

public class EntryPoint {
  static Logger logger = 
    Logger.getLogger(EntryPoint.class.getName());

  static EntryPoint entryPoint;

  Main main;

  public static void main(String[] args) {
    entryPoint = new EntryPoint();

    Runtime.getRuntime()
     .addShutdownHook(new Thread() {
      public void run() {
        try {
          entryPoint.stop();
        } catch (Exception e) {
          logger.fatal(e.toString());
        }
      }
    });

    try {
      entryPoint.start();
    } catch (Exception e) {
      logger.fatal(e.toString());
    }

  }

  public void start() throws Exception {
    logger.info("Starting up");

    // Start up the context
    main = new Main();
    main.start();

    logger.info("Started");
  }

  public void stop() throws Exception {
    logger.info("Stopping");

    // Shutdown the context
    main.stop();

    logger.info("Stopped");
  }
}

A Camel based XML Payload HTTP polling provider

Wow, what a mouthful of a title that is.

The EAI Polling Consumer pattern is well documented. Polling consumers are particularly useful for HTTP clients such as AJAX applications. Their presence provides a means of implementing publish and subscribe.

What I needed was some code to service my AJAX consumer; a Polling Provider so to speak.

Given my immersion in Apache Camel I came up with a Polling Provider for XML objects provided by XMLBeans. The provider assumes that an HTTP endpoint is to be interacted with and thus sets the HTTP time headers appropriately.

The Polling Provider holds the notion that the xml object is updated by one thread and then consumed by another. Furthermore the time the xml object was last updated is retained. This allows HTTP consumers to specify the HTTP ifModifiedSince header and block if the condition is not met.

To update the provider (fidsDocumentPollingProvider is an instance of it and newFidsDocument is an XML document object):

fidsDocumentPollingProvider.update(
  newFidsDocument, new Date());

To retrieve the resource in a RESTful manner:

// Provide a RESTful service to retrieve FIDS data
from("jetty:http://0.0.0.0:9000/FIDSService/FIDS")
  .inOut("direct:getFIDS");

// Provide a component to process FIDS requests
from("direct:getFIDS")
  .process(fidsDocumentPollingProvider);

Here is the code I came up with for the provider. I would be very interested to hear of better ways to do this given Apache Camel.

package com.classactionpl.camel.xmlbeans;

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
import java.util.TimeZone;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.component.http.HttpProducer;
import org.apache.xmlbeans.XmlObject;

public class XMLHttpPayloadPollingProvider implements Processor {
  XmlObject xmlObject;
  Date lastModified;
  Lock lock = new ReentrantLock();
  Condition update = lock.newCondition();

  public void update(XmlObject xmlObject, Date lastModified) {
    lock.lock();
    try {
      this.xmlObject = xmlObject;
      // We don't need the milliseconds and they can upset
      // our comparisons given that they are not passed from the
      // outside world.
      this.lastModified = new Date((lastModified.getTime() / 1000) * 1000);
      update.signal();
    } finally {
      lock.unlock();
    }
  }

  public void process(Exchange exchange) throws Exception {
    lock.lock();
    try {
      boolean waitForUpdate = true;
      Date ifModifiedSince = null;
      if (lastModified != null) {
        String ifModifiedSinceStr = exchange.getIn().getHeader(
            "If-Modified-Since", String.class);
        if (ifModifiedSinceStr != null) {
          try {
            SimpleDateFormat rfc822DateFormat = new SimpleDateFormat(
                "EEE', 'dd' 'MMM' 'yyyy' 'HH:mm:ss' 'Z",
                Locale.US);
            ifModifiedSince = rfc822DateFormat
                .parse(ifModifiedSinceStr);
            waitForUpdate = (lastModified
                .compareTo(ifModifiedSince) <= 0);
          } catch (ParseException e) {
          }
        }
      }

      boolean provideDateHeader;

      if (waitForUpdate) {
        update.await(90, TimeUnit.SECONDS);
      }

      if (xmlObject != null) {
        if (ifModifiedSince == null) {
          ifModifiedSince = lastModified;
        }
        if (lastModified.compareTo(ifModifiedSince) > 0) {
          exchange.getOut().setBody(xmlObject.newInputStream());
          exchange.getOut().setHeader("Content-Type", "text/xml");
          provideDateHeader = true;
        } else {
          exchange.getOut().setHeader(
              HttpProducer.HTTP_RESPONSE_CODE, 304);
          provideDateHeader = true;
        }
      } else {
        exchange.getOut().setHeader(HttpProducer.HTTP_RESPONSE_CODE,
            404);
        provideDateHeader = false;
      }

      if (provideDateHeader && lastModified != null) {
        SimpleDateFormat rfc822DateFormat = new SimpleDateFormat(
            "EEE', 'dd' 'MMM' 'yyyy' 'HH:mm:ss' GMT'", Locale.US);
        rfc822DateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
        exchange.getOut().setHeader("Last-Modified",
            rfc822DateFormat.format(lastModified));

      }

    } finally {
      lock.unlock();
    }
  }
}

Titan Class goes into production

She's finished, well as finished as they always are - which of course means that they are never finished.

I am referring to my work over the past few months which is really just a concerted effort building on years of work. I'm now in a position to track vehicles, containers, you name it.

From a software development perspective there are many things that I'm happy about. I've previously discussed jQuery which really has eased the AJAX programming. From a backend perspective I am using the amazingly functional, scalable and distributable Apache Camel (also discussed in other posts).

One thing with the client-side that has really impressed me is the Google Earth plugin. I hope that you agree that it works very, very well (a link to the site is provided at the end of this post). Some of our potential customers will require their own maps along with their own projections and Google Earth will not handle that (Antarctica for example). However for many scenarios, Google Earth will be great.

Incidentally where we will be required to use specialised maps with funky projections, I have written a Java applet using JavaFX. The applet supports many types of map projections and is based on the work of Proj.4 and friends.

Here is a quick list of the OSS projects I utilised (and even contributed back to in some cases!):

Apache ActiveMQ,
Apache Camel
Apache Directory Server
Apache MINA
Apache Tomcat
Apache Web Server
GEOS
Eclipse
Hibernate
Hibernate Spatial
jQuery
JSON
JTS
JUnit
Linux
Log4J
Maven
PostgresQL
PostGIS
Proj.4
Spring
Subversion
xmlbeans

Go OSS!

Feel free to visit the tracking site and play around with the demo data.