Tag Archives: TED5000

Using XPath to pick data out of XML

This week I wrote a WattDepot sensor for the TED 5000 home energy meter. The TED 5000 gateway (a small Internet-connected embedded computer) provides a URI that generates XML showing the current power data. First, I needed to figure out what the XML meant. Once that was done, I wanted a quick and simple way to pick out the 2 pieces of data from the XML that I care about using Java.

WattDepot uses JAXB extensively for XML processing, but that was kinda heavyweight for my needs here. I had heard about XPath, and it sounded like the right type of tool for just grabbing a little data from XML. Turns out that Java 1.5 and later have XPath built-in, so there’s no additional dependencies.

IBM has a good tutorial on using XPath from Java by Elliotte Rusty Harold. Unfortunately, I was confused initially because all the XPath examples in the tutorial are for finding all XML nodes in a document that meet certain criteria, whereas I knew exactly where in the XML tree my data was lurking. Luckily, it turns out that XPath is really a lot like a path in a filesystem (duh), so traversing the tree is easy.

Say you have the following XML from TED (some parts elided):

<LiveData>
  ...
  <Power>
    <Total>
      <PowerNow>2995</PowerNow>
      ...
      <PowerMTD>515227</PowerMTD>
      ...
    </Total>
  ...
  </Power>
</LiveData>

The XPath that would pull out the value from PowerNow is /LiveData/Power/Total/PowerNow/text(), and for PowerMTD it is /LiveData/Power/Total/PowerMTD/text(). Simple!

Here a code fragment that extracts those two values from an XML file (stealing liberally from the XPath tutorial linked above):

public class XPathTest {

  public static void main(String[] args) throws ParserConfigurationException, SAXException,
      IOException, XPathExpressionException {
    if (args.length != 1) {
      System.out.println("Need XML filename arg.");
      return;
    }
    DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();
    domFactory.setNamespaceAware(true);
    DocumentBuilder builder = domFactory.newDocumentBuilder();
    Document doc = builder.parse(args[0]);

    XPathFactory factory = XPathFactory.newInstance();
    XPath powerXpath = factory.newXPath();
    XPath energyXpath = factory.newXPath();
    XPathExpression exprPower = powerXpath.compile("/LiveData/Power/Total/PowerNow/text()");
    XPathExpression exprEnergy = energyXpath.compile("/LiveData/Power/Total/PowerMTD/text()");
    Object powerResult = exprPower.evaluate(doc, XPathConstants.NUMBER);
    Object energyResult = exprEnergy.evaluate(doc, XPathConstants.NUMBER);

    Double power = (Double) powerResult;
    Double energy = (Double) energyResult;
    System.out.println("Power from TED 5K: " + power + "W");
    System.out.println("Energy from TED 5K month to date: " + energy + "Wh");
  }
}

It’s nice to have a quick and easy way to make use of XML from Java in my toolbox.

Advertisements

it’s electric: TED data storage and plotting

I was checking on the website for The Energy Detective the other day looking for API info, and found that their page of 3rd-party applications had been updated, and included an application called it’s electric. it’s electric is a Java web application that queries the TED gateway frequently for the 1 second resolution power data, and stores it in a Berkeley DB. That alone is useful, as the TED has a segmented data storage system, keeping the 1 second resolution data only for an hour (and so on for coarser grained data).

It also provides a graphing system based on Google’s Annotated Timeline visualization, with some enhancements like automatically changing the resolution of the displayed data depending on the time interval displayed. Here’s a screenshot:

Screenshot of graph produced by it's electric

There’s a Google group for support and discussion, and the author Robert Tupelo-Schneck seems quite responsive. A jar file is provided on the group page (which I won’t link to since you should download the latest version), which includes the Java bytecode as well as the source, which is released under the AGPL license. The application is not large, consisting of 5 class files.

Compared to WattDepot, it’s electric seems considerably snappier. Presumably this is due in part to using Berkeley DB for persistence instead of an SQL database. The code also stores data in byte form, rather than higher-level Java objects and XML. Also, it’s electric occupies a clear functionality niche: it provides long-term storage of the finest-grained TED data (which is otherwise lost every hour), and provides graphing of that data from locations outside the home network.

I experienced some problems when scrolling around the data on the live it’s electric website, sometimes the graph would not update, or I was unable to scroll to where I wanted to apparently because new data was being loaded in for the current location.

Overall it’s electric looks like it could be useful for TED owners that want to hold on to that fine grained data, and want more options for displaying that data outside the home.