Category Archives: ICS699-Fall2009

Prepping for screencast

Last week I focused mostly on non-programming tasks of reviewing the NSF proposal and preparing for the WattDepot screencast. I did close one refactoring issue that I generated when rolling out the Google Visualization support.

This week will be focused on the issues identified as critical for a WattDepot 1.0 release.

Plans from last week:

  • Review grant proposal
    • done
  • Plan script for 10 minute screencast
    • in progress
  • Close some additional WattDepot issues
    • closed refactoring issue 24

Other accomplishments from last week:

  • Created WattDepot issues for items discussed with Philip

Pointers to work products:

Plans for this week:

  • Create stress tests for WattDepot (issue 32)
  • Benchmark WattDepot API using stress tests
  • Optimize Derby implementation based on performance data
  • Finish script for 10 minute screencast
  • Make 10 minute screencast?
  • Start support for energy counters in Energy class
  • CSDL firewall reorg

Cool links:

WattDepot, now with Google Visualizations

After discussion with Philip, decided to prioritize the Google Visualization data source work over performance testing. WattDepot now provides a data source for the Google Visualization API that allows plotting multiple types of data on the same graph, and expanding virtual sources into their component subsources. I have written a wiki page about using WattDepot’s Google Visualization data source.

Plans from last week:

  • Test performance of Derby implementation
    • deferred for now
  • Make GVisualizationServlet work with virtual sources
    • done
  • Review grant proposal?
    • not done, working on it this week
  • Close some more WattDepot open issues
    • closed some issues, but opened more new ones

Other accomplishments from last week:

  • Registered for Spring 2010 classes
  • Tracked down CSDL’s copy of Snow Leopard Server from UHM Bookstore
  • Pushed out new WattDepot release and notified 413/613 students of visualization availability
  • Watched some crazy big-wave surfing in Waimea Bay without getting sunburned

Pointers to work products:

Plans for this week:

  • Review NSF proposal
  • Plan script for 10 minute screencast
  • Close some additional WattDepot issues

Cool links:

  • Google Goggles is an Android application that lets you take pictures of objects (only some are recognized) and does a search for relevant information. Will do OCR on text found if it doesn’t know about that type of object.

WattDepot has persistence

The Derby persistence implementation is now complete. This week I will start performance testing the Derby persistence implementation, and work on the GVisualizationServlet so we can visualize virtual sources. Loading default data into WattDepot using Derby for storage (no network traffic) is running at ~876 sensor data entries per second, which seems pretty good.

Plans from last week:

  • Remove last of HashMap code from DerbyStorageImplementation
    • done, along with some other cleanup (like running unit tests on both DB implementations)
  • Start performance testing of Derby storage implementation
    • just started

Other accomplishments from last week:

  • Ate lots of delicious food

Pointers to work products:

Plans for this week:

  • Test performance of Derby implementation
  • Make GVisualizationServlet work with virtual sources
  • Review grant proposal?
  • Close some more WattDepot open issues

Cool links:

WattDepot Derby storage almost complete

The Derby persistence implementation is mostly complete. SensorData is temporarily still stored both in memory and in Derby, to simplify the initial implementation. The next steps during this short week are to remove the final in-memory tables, and start performance testing the Derby persistence implementation.

Plans from last week:

  • Keep working on DerbyStorageImplementation
    • almost done
  • Finish implementing missing methods in REST API
    • not done, deferred until Derby work is done

Other accomplishments from last week:

  • Wrote up directions on running WattDepot locally

Pointers to work products:

Plans for this week:

  • Remove last of HashMap code from DerbyStorageImplementation
  • Start performance testing of Derby storage implementation

Cool links:

  • New acronym I encountered: LOHAS. Lifestyles of Health and Sustainability (LOHAS) is a demographic defining a particular market segment related to sustainable living, “green” ecological initiatives, and generally composed of a relatively upscale and well-educated population segment.

WattDepot presentation & Derby started

Last week was holiday-shortened by Veterans’ Day. I spent most of my time working on my slides for my ICS 690 presentation, and helping the REIS seminar substitute videographer. Now that the presentation is done, I have begun implementing persistence in WattDepot using Derby.

Plans from last week:

  • Finish slides for presentation to ICS 690
    • done
  • Start working on DerbyStorageImplementation
    • done

Other accomplishments from last week:

  • Finally got video from 11/5/2009 REIS seminar uploaded
  • Attended ITS update seminar
  • Upgraded to 64-bit Cocoa version of Eclipse
  • Wrote up my Java 1.6 & Eclipse experiences

Pointers to work products:

Plans for this week:

  • Keep working on DerbyStorageImplementation
  • Finish implementing missing methods in REST API

Cool links:

Yet more WattDepot work

Last week I got WattDepot ready for the next ICS 413/613 assignment. This included adding fuel types to each of the sources, and then implementing each of the new CLI commands to ensure that they could actually be implemented 🙂 Along the way I made some changes to the client that only became apparent when running the client independent of the server.

I also changed the server so that it can load default resources from one big file instead of one resource per file. This greatly speeds startup when there are 100K+ resources to load (like Oscar-generated sensor data for 2 months).

I’m still working on slides for my presentation for ICS 690 this week. After that is done, I will shift to working on persistence in WattDepot using Derby.

Plans from last week:

  • Turn TODO.txt into Google Code issues
    • done
  • Implement instantaneous carbon emission rate resource in WattDepot
    • not done, ended up using emissions over 1 hr of energy as basic measure
  • Write up code review description for Philip
    • not done
  • Prepare draft presentation for ICS 690
    • in progress
  • Make GVisualizationServlet support virtual sources
    • not done
  • Start working on DerbyStorageImplementation
    • not done

Other accomplishments from last week:

  • Prepared documents for REIS RA renewal application
  • Sped up WattDepot server startup from minutes to a few seconds
  • Implemented ICS 413/613 CLI assignment
  • More problematic video from REIS seminar. Double Arg.

Pointers to work products:

Plans for this week:

  • Finish slides for presentation to ICS 690
  • Start working on DerbyStorageImplementation

Cool links:

  • N/A

Refactoring WattDepot

I spent most of last week working on refactoring some cruft out of WattDepot. The core data types (Source, SensorData, and User) now all have proper constructors and all the code that was in utility classes has been folded into the datatypes themselves. This makes the rest of the system easier to read, but it means it will be a pain if I need to modify the XML schema in the future. If I need to make schema modifications, perhaps I will use the JAXB schema compiler in the other direction to go from code to XML schema.

There is still more refactoring that needs to be done, especially around the interpolated resources. I need to start creating issues in the Google Code site instead of my current tracking via TODO.txt.

Next week on Thursday 11/12 I will be giving an hour-long presentation on my research. Philip will be out of the office for the latter half of that week, so I want to have a draft presentation ready by the end of this week. The other important deliverable for this week is a carbon emission rate resource for WattDepot, which will be needed for the next ICS 413/613 assignment.

Other WattDepot features that need to be implemented are persistence and bringing the Google visualization servlet up to the rest of WattDepot (supporting virtual sources, authentication, etc).

Plans from last week:

  • Make list of refactoring changes to implement
    • mostly done, should be turned into Google Code issues
  • Start designing real persistence layer
    • not done. Philip and I agree that for expediency’s sake I should stick with Derby for now
  • Make Saunders demo with virtual source Google visualization?
    • not done, requires visualization data source improvements
  • Start planning for next 413/613 assignment
    • done, only new feature needed is instantaneous carbon emission rate
  • Write WattDepot server installation and configuration wiki page
    • deferred: without real persistence, probably nobody should be using WattDepot just yet, so no point in writing up installation instructions that will need to be changed

Other accomplishments from last week:

  • Produced a list of “known good” values from the current WattDepot public server for the ICS 413/613 students
  • Wrestled with bad video from REIS seminar, probably due to dirty camcorder heads. Arg.

Pointers to work products:

Plans for this week:

  • Turn TODO.txt into Google Code issues
  • Implement instantaneous carbon emission rate resource in WattDepot
  • Write up code review description for Philip
  • Prepare draft presentation for ICS 690
  • Make GVisualizationServlet support virtual sources
  • Start working on DerbyStorageImplementation

Cool links:

  • Google Wave is pretty interesting (just got an invite last week). In theory, it seems pretty useful, but until many of the people who I collaborate with have accounts, it’s hard to tell if it will be useful in practice.

WattDepot first release available

As usual, the last 10% took longer than expected, and I spent all last week finishing WattDepot up. It now has all the functionality required by the first WattDepot assignment in ICS 413/613. Virtual sources are working (except for some exceptional cases), and interpolated resources like power, energy, and carbon all work. There’s also a new wiki page explaining how to use the WattDepot client library. To help those unfamiliar with the physical concepts behind electrical power, I wrote a page explaining those concepts. There is now a public distribution of WattDepot available, but no installation documentation so not really useable outside the class context yet.

While getting WattDepot to this point was pretty exhausting, there’s still a laundry list of refactoring and features needed to make it more generally functional. I’ll be working on that this week.

Plans from last week:

  • Finish implementing Power resource GET for a timestamp
    • done
  • Design and implement Energy resource GET for a range of times
    • done
  • Design and implement Carbon resource GET for a range of times
    • done
  • Refactor some of the bogus WattDepot code that has accumulated
    • started, but much more to be done

Other accomplishments from last week:

  • Discovered that PMD failures were not failing Hudson builds for projects derived from hackystat-developer-example. Fixed for wattdepot, oscar-project, and hackystat-developer-example, but others could be lurking.

Pointers to work products:

Plans for this week:

  • Make list of refactoring changes to implement
  • Start designing real persistence layer
  • Make Saunders demo with virtual source Google visualization?
  • Start planning for next 413/613 assignment
  • Write WattDepot server installatoin and configuration wiki page

Cool links:

  • UHM thesis/dissertation LaTeX style. This has been worked on by many hands, and I recently passed on the link to ICS MS student who passed his defense and now needs to work on formatting. 🙂

Oscar support almost done

As planned, the past week has been all about building out the WattDepot functionality required by Oscar, specifically the first assignment using Oscar data in ICS 413/613. I’ve made massive progress, but still not quite finished with the requirements for the assignment. Planning to finish that up today, and then probably do some refactoring and prioritization of additional WattDepot functionality.

Plans from last week:

  • Finish input client for OSCAR data
    • done
  • Design and implement Source summary resource
    • done
  • Implement GET SensorData for a range of times
    • done
  • Design and implement Power resource GET for a timestamp
    • done for normal sources, almost done for virtual sources
  • Design and implement Energy resource GET for a range of times
    • not done
  • Design and implement Carbon resource GET for a range of times
    • not done
  • Implement virtual sources
    • done, though as new features are added they will need virtual source loving

Other accomplishments from last week:

  • Discovered that PMD failures were not failing Hudson builds for projects derived from hackystat-developer-example. Fixed for wattdepot, oscar-project, and hackystat-developer-example, but others could be lurking.

Pointers to work products:

Plans for this week:

  • Finish implementing Power resource GET for a timestamp
  • Design and implement Energy resource GET for a range of times
  • Design and implement Carbon resource GET for a range of times
  • Refactor some of the bogus WattDepot code that has accumulated

Cool links:

BMO data importing, onward to Oscar

It took longer than I had planned for, but I now have a client that polls Building Manager Online for data every 15 minutes, and inputs that data into WattDepot, where it can be displayed in a Google Visualization API graph. Discovered that Saunders’ Acquisuite box is set to only upload meter data every hour, so the graphed data is not as fresh as it could be.

This week has a singular focus: implement all the WattDepot functionality required to support the first ICS 413/613 assignment on WattDepot. Philip has written up the assignment, and now it is just a matter of implementation.

Philip got Google PowerMeter working with his TED 5000, which is cool. Hopefully we can extract data directly from PowerMeter, thus avoiding the need to reconfigure firewalls to allow direct access to the TED.

Plans from last week:

  • Finish WattDepot client for Building Manager Online
    • done, finally
  • Implement input client for OSCAR data
    • in progress, should be done by end of day on Monday
  • Write API for Power resource
    • not done
  • Write API for Carbon resource
    • not done
  • Start implementing Power resource
    • not done

Other accomplishments from last week:

Pointers to work products:

Plans for this week:

  • Finish input client for OSCAR data
  • Design and implement Source summary resource
  • Implement GET SensorData for a range of times
  • Design and implement Power resource GET for a timestamp
  • Design and implement Energy resource GET for a range of times
  • Design and implement Carbon resource GET for a range of times
  • Implement virtual sources

Cool links:

  • None to report.