Monthly Archives: August 2009

WattDepot storage layer coming along

This is the first blog post of the fall semester, and finds me still grinding away on the WattDepot implementation. Things are going well, but not at the pace I would like. I’m supposed to have everything working by mid-September, which is going to require a major push.

Last week I finished the database abstraction layer, and wrote a memory-only database implementation. This will allow me to continue to push forward on all fronts while I figure out CouchDB. I’m still working on the tests of the DB layer, which should test the CouchDB storage as well as the memory implementation.

Philip pointed out last week that WattDepot needs some higher level resources to support the functionality needed for ICS 413/613. Specifically, it needs an Energy resource that can report on the energy produced or consumed by a Source over a period of time, and a Carbon resource that can compute the carbon emissions of a Source, given its carbon intensity (GHG emitted in g of CO2E per kWh). Both these new computed resources need to be added to the REST API, and then implemented.

Plans from last week:

  • Write DB abstraction layer
    • done
  • Implement simple memory storage of data
    • done, but still writing tests
  • Work on User resource
    • not done
  • Attend first REIS seminar this Thursday 4:30-5:30 PM
    • done

Other accomplishments from last week:

  • Met Blue Planet Foundation folks about their upcoming conservation contest
  • Realized that workshop in Florida I was thinking about attending is in October!
  • Became primary ICS representative to the GSO

Pointers to work products:

Plans for this week:

  • Finish writing DbManager tests
  • Implement User resource
  • Implement Source resource
  • Start reading CouchDB manual
  • Install CouchDB
  • Add Energy resource to REST API
  • Videotape REIS seminar
  • Write followup email to Blue Planet

Cool links:

  • None

WattDepot logging

This is the final blog post of the summer, as the Fall semester starts today. Didn’t get to spend as much time on implementation last week as I had planned. I now have unit tests for the authenticated placeholder User resources, and refactored the client to support accessing resources that require authentication. I’ve also adopted the HackystatLogger code (unable to use as library due to hard coded Hackystat-specific paths) so that I can do logging properly, so I fixed my code that was logging to System.out to use a logger.

The other task from last week was researching whole home electricity meters. The situation has improved from last December, and the TED 5000 appears to meet all of WattDepot’s needs. In particular, it now has a Gateway component that records data from the measurement component and makes it available over TCP/IP via an XML API. For $240, including a wireless handheld display, it looks like a really good deal. You can check out the web interface on a live demo unit they have set up. BrulTech makes a meter that looks like the only other real option, but it lacks the Gateway component, so you have to provide your own device to monitor the data and distribute it via TCP/IP.

Plans from last week:

  • Get logging working in WattDepot
    • done
  • Implement UserManager for concurrency control
    • not done, after discussion with Philip we’re thinking we should push this down into the DB layer
  • Implement memory storage of data
    • not done
  • Work on User resource
    • not done
  • Look for home meter solutions that could work with WattDepot
    • done

Other accomplishments from last week:

  • Provided feedback to Pavel on his dissertation proposal defense. He passed! Congrats!
  • Got tests working for authenticated resources, tracked down JUnit problem with Philip’s assistance (needed fork=yes)
  • Found and fixed a couple of minor problems in the hackystat-developer-example Ant build scripts

Pointers to work products:

Plans for this week:

  • Write DB abstraction layer
  • Implement simple memory storage of data
  • Work on User resource
  • Attend first REIS seminar this Thursday 4:30-5:30 PM

Cool links:

Revising WattDepot API

Philip looked at my diagram, and had more suggestions for improvement. I’ve now created three diagrams: one that shows the relationships between resources, one that shows examples of how sensor data might flow into WattDepot, and one that shows examples how sensor data from WattDepot might might be used. The WattDepot REST API introduction has been updated with the new diagrams. I also realized that for many applications, it will be necessary to combine the data from multiple Sources to map to real world concepts, for example a building’s usage might be recorded from multiple meters. I was planning to create a new type of resource for this, but Philip sagely pointed out that I should just allow Sources to include other Sources. So I’ve updated the API to support virtual Sources, which include the data from sub-Sources. Actually aggregating the data from multiple sources will require some additional thought, because in the general case the clocks of the meters are not synchronized, so the data for a particular point in time for a virtual source will require some work to calculate.

My implementation is behind schedule. I spent a lot of time figuring out how to get HTTP authentication working. This turns out to be really simple in Restlet, but it took me a long time to fully grok it. I now have HTTP Basic authentication working for the admin user, and placeholder User resources behind the authentication. I’m spending time figuring out the infrastructure required to get any of the stateful resources working (logging, concurrency control, persistance), but once I have them all working for the first resource, the rest should follow more quickly.

Plans from last week:

  • Implement user authentication in WattDepot
    • done
  • Implement User resource
    • started, currently just has placeholder
  • Implement memory storage of data
    • not done
  • Start reading CouchDB manual
    • not done
  • Finish reading docs for Google Visualization data source library
    • not done

Other accomplishments from last week:

Pointers to work products:

Plans for this week:

  • Get logging working in WattDepot
  • Implement UserManager for concurrency control
  • Implement memory storage of data
  • Work on User resource
  • Look for home meter solutions that could work with WattDepot

Cool links:

  • Enphase makes microinverters for solar panels. This animation of their Enlighten monitoring system shows how powerful it is to have data about the production from every panel.

Implementing WattDepot

After review of the WattDepot REST API, Philip suggested that I add a diagram to clarify the relationships between the resources, so I’ve done that. With the API complete (at least for now), I’ve moved onto the actual implementation. I decided to start with the easiest part, so the Health resource is working now, and I’ve done some refactoring to make it easier to write test cases, and to customize the server parameters through a properties file. Next, I’ll be implementing the User resource, which will require getting authentication working. Initially, I’ll just store the data in memory with no on-disk persistance, but later I’ll probably be storing the data in CouchDB.

Plans from last week:

  • Revise REST API based on feedback from Philip
    • done, added diagram to show how resources fit together
  • Start implementing REST API
    • done, implemented Health resource
  • Finish reading docs for Google Visualization data source library
    • not done, but probably should finish reading before implementing CouchDB storage

Other accomplishments from last week:

Pointers to work products:

Plans for this week:

  • Implement user authentication in WattDepot
  • Implement User resource
  • Implement memory storage of data
  • Start reading CouchDB manual
  • Finish reading docs for Google Visualization data source library

Cool links:

  • PollEverywhere is a service that allows you to run real-time polls with votes from text messages, tweets, and mobile and desktop web browsers. You can put make a poll into a slide in a PowerPoint presentation, or embed it in a web page. It’s free for up to 30 (32 for academia) voters per poll. That’s great for many classes and presentations.
  • Project Lombok is a library that provides Java @annotations to make implementation easier. For example, you can use @Data to provide getters, setters, toString, equals, and hashCode. Looks pretty sweet.
  • Stormpulse is pretty nice for tracking hurricanes, like hurricane Felicia which is heading for Hawaii. Luckily, it is very weak now so it will just mean a little more rain and wind than usual.

WattDepot API first draft

I finished the first draft of the WattDepot REST API. After discussion with Philip, I’ve removed the concept of a SensorDataType, and the canonical properties associated with sensor data will be listed in a PropertyDictionary. Looking forward to moving on to the implementation of the API.

Also, got confirmation that I have a research assistantship for the fall semester. Yay!

Plans from last week:

  • Finish first draft of WattDepot REST API
    • done!
  • Follow up on some contacts from Tech Hui conference
    • done
  • Read up on Google Visualization data source library
    • continuing to read, haven’t gotten all the way through

Other accomplishments from last week:

  • Upgraded to latest version of Eclipse

Pointers to work products:

Plans for this week:

  • Revise REST API based on feedback from Philip
  • Start implementing REST API
  • Finish reading docs for Google Visualization data source library

Cool links: