Category Archives: Hackystat

Debugging Restlet connector problem

In the course of developing WattDepot, I ran into an annoying intermittent bug in my JUnit tests. I would sometimes get a failure in one particular test class, but not always in the same method of that class. The failure manifested as a 60 pause on the affected test, followed by the WattDepotClient method returning a 1001 miscellaneous failure status code. Maddeningly, it would only fail sometimes, making it much harder to track down (and making continuous integration comical). Further, running the test from within Eclipse would work fine every time, so I was unable to use the debugger to figure out what was going on.

Philip pointed out that this sounded like a classic deadlock problem between threads, perhaps in Derby which I’m using for persistence. He suggested that I use VisualVM to see if I could track down any deadlocks. Mac OS X comes with VisualVM installed as “jvisualvm”, and it’s pretty easy to use. Luckily, since the failure manifested as a 60 second pause, I could start the test, and then attach to the JUnit process and obtain thread dumps to see what was going on.

After a few thread dumps, I tracked it down to HTTP communication. The failure happens when the client is using PUT to send a new resource to the server, and the server is waiting for the end of the entity body from the client. This happens before any Derby call, so it looks like Derby is ruled out (at least for this bug).

WattDepot uses the Restlet framework to make it easier to implement the REST API, and to perform all the HTTP client and server work. Restlet provides a variety of connectors for both the client and server HTTP connections. In fact, there are enough options that it is somewhat confusing trying to pick one. Restlet has internal HTTP client and server connectors that come in the core Restlet jars. According to this email thread, the choice of connector is done automatically by scanning the classpath, with the first match winning.

When first setting up WattDepot, I based the set of Restlet jars I was using on Hackystat. Hackystat’s SensorBase includes org.restlet.jar (API classes), com.noelios.restlet.jar (reference implementation, including internal HTTP connectors), (client connector based on JDK HTTP code), and com.noelios.restlet.ext.simple_3.1.jar (server connector based on Simple framework). So it appears that WattDepot is using the Net connector for client HTTP connections, and the Simple connector for server connections, both overriding the internal HTTP connections in the reference implementation.

Since my problem was taking place in the HTTP code, I decided to try experimenting with removing Net and Simple from the classpath, thereby allowing the appropriate internal HTTP connector to kick in. Since I’m using Ivy and Ivy RoundUp for dependency management, this turns out to be as easy as changing the configuration parameter in the Restlet Ivy config, deleting the project “lib” directory and rerunning the tests.

After trying all combinations (all internal connectors, internal server & Net client, Simple server & internal client, Simple server & Net client), I found that only the combination of the Simple server connector and the Net client connector leads to my unit test failure. I guess I’m just lucky that way. 🙂

The solution is then to stop using either the Net client or the Simple server. Since the WattDepot server is likely to be the more performance-sensitive aspect of WattDepot, I opted to keep the Simple server on the assumption that it is higher performance than the internal Restlet server. It would be nice to figure out which of the variety of client and server connectors is recommended as the best performing, but this will do for now.

In the future I plan to post something to the Restlet mailing list to see if anyone else has run into this problem so it can be tracked down and perhaps fixed.

Java 1.6 & Eclipse on Mac OS X

These are some notes on using Java 1.6 on Mac OS X with Eclipse.

Mac OS X lagged behind for some time on adopting Java 1.6, but it was finally released as Java for Mac OS X 10.5 Update 1. Unfortunately, this release has some limitations: it only runs in 64-bit mode and only on Mac OS X 10.5 (Leopard). This meant no Java 1.6 support on Mac OS X 10.4, no support for PowerPC systems, and no support on the early 32-bit Intel Macs released in 2006.

To make Java 1.6 the default Java execution environment, run the Java Preferences application (found in the Utilities subfolder of the Applications folder). There you can drag the Java versions into any order you wish for both applications and applets. So to default to using Java 1.6 for everything, you can just drag Java SE 6 to the top of both the application and applet lists. The Java Preferences application will change around symlinks to correspond to your new choices. Mac OS X 10.6 (Snow Leopard) appears to include only Java SE 6, but includes both 32 and 64-bit versions, so it should allow Java 1.6 to be run on the oldest Intel Macs (no PPC because Snow Leopard does not support PPC).

Some command line scripts expect the JAVA_HOME environment variable to be set to the directory that contains the Java distribution being used. Also, some scripts might require different versions of Java. To support this, Apple introduced a new utility called /usr/libexec/java_home. By default, java_home just returns the home value appropriate based on the selection in the Java Preferences utility, but command line arguments can request different versions (see the manual page linked earlier). The output of java_home is intended to be assigned to the JAVA_HOME environment variable in a shell initialization script (like .profile or .cshrc).

This brings us to Eclipse, the Java development environment. Eclipse can be used to develop for multiple Java versions. When creating a Java project, Eclipse prompts for the desired JRE version in the new project dialog box. To ensure that Eclipse tracks the language differences between versions, also select the desired version in Preferences->Java->Compiler->Compiler compliance level to match your project.

However, despite these changes, Eclipse itself does not necessarily run using the Java version being used for the project being developed. Initially, the latest version of Eclipse (3.5) was released only in 32-bit mode for both Carbon (an older and deprecated Mac OS API) and Cocoa (the modern Mac OS API). Since Leopard only supported Java 1.6 in 64-bit mode, this meant that Eclipse was always running under Java 1.5. With the release of Eclipse 3.5.1, there are now 64-bit Cocoa downloads available, and these will run under Java 1.6. For some users, this may not be important, but for those doing Eclipse tool development (such as the Hackystat Eclipse sensor) it is very helpful. I use JAXB in some of my Ant build scripts, and JAXB is built into Java 1.6 but not Java 1.5. When running Eclipse in Java 1.5, many of my Ant scripts report spurious errors about being unable to locate JAXBException, but these all vanish when running Eclipse under Java 1.6. I should note that at least some people disagree with this advice, and suggest using the 32-bit Cocoa version on Leopard instead of the 64-bit version. Everyone apparently agrees that on Snow Leopard you want the 64-bit Cocoa version of Eclipse (unless you are on a 32-bit Intel Mac).

So if you want to use Java 1.6 for your application and run Eclipse under Java 1.6 there are two options:

  • Use Mac OS X 10.5 Leopard (with all software updates) on a Core 2 Duo Intel Mac, with Java Preferences set to use Java SE 6 as the default Java version, and use the 64-bit Cocoa version of Eclipse 3.5.1.
  • Use Mac OS X 10.6 Snow Leopard on an Intel Mac, and use the 64-bit Cocoa version of Eclipse 3.5.1 [note, I have not specifically tested this configuration, but it should work. Confirmed, this works]

WattDepot logging

This is the final blog post of the summer, as the Fall semester starts today. Didn’t get to spend as much time on implementation last week as I had planned. I now have unit tests for the authenticated placeholder User resources, and refactored the client to support accessing resources that require authentication. I’ve also adopted the HackystatLogger code (unable to use as library due to hard coded Hackystat-specific paths) so that I can do logging properly, so I fixed my code that was logging to System.out to use a logger.

The other task from last week was researching whole home electricity meters. The situation has improved from last December, and the TED 5000 appears to meet all of WattDepot’s needs. In particular, it now has a Gateway component that records data from the measurement component and makes it available over TCP/IP via an XML API. For $240, including a wireless handheld display, it looks like a really good deal. You can check out the web interface on a live demo unit they have set up. BrulTech makes a meter that looks like the only other real option, but it lacks the Gateway component, so you have to provide your own device to monitor the data and distribute it via TCP/IP.

Plans from last week:

  • Get logging working in WattDepot
    • done
  • Implement UserManager for concurrency control
    • not done, after discussion with Philip we’re thinking we should push this down into the DB layer
  • Implement memory storage of data
    • not done
  • Work on User resource
    • not done
  • Look for home meter solutions that could work with WattDepot
    • done

Other accomplishments from last week:

  • Provided feedback to Pavel on his dissertation proposal defense. He passed! Congrats!
  • Got tests working for authenticated resources, tracked down JUnit problem with Philip’s assistance (needed fork=yes)
  • Found and fixed a couple of minor problems in the hackystat-developer-example Ant build scripts

Pointers to work products:

Plans for this week:

  • Write DB abstraction layer
  • Implement simple memory storage of data
  • Work on User resource
  • Attend first REIS seminar this Thursday 4:30-5:30 PM

Cool links:

Google Gears

Google just released a beta version of Gears, which is a plugin for Firefox that allows web applications to work offline. It even synchronizes data manipulated offline when you go back online. Not sure if there is an immediate Hackystat application.