Prepping for screencast

Last week I focused mostly on non-programming tasks of reviewing the NSF proposal and preparing for the WattDepot screencast. I did close one refactoring issue that I generated when rolling out the Google Visualization support.

This week will be focused on the issues identified as critical for a WattDepot 1.0 release.

Plans from last week:

• Review grant proposal
  • done
• Plan script for 10 minute screencast
  • in progress
• Close some additional WattDepot issues
  • closed refactoring issue 24

Other accomplishments from last week:

• Created WattDepot issues for items discussed with Philip

Pointers to work products:

• WattDepot subversion repository

Plans for this week:

• Create stress tests for WattDepot (issue 32)
• Benchmark WattDepot API using stress tests
• Optimize Derby implementation based on performance data
• Finish script for 10 minute screencast
• Make 10 minute screencast?
• Start support for energy counters in Energy class
• CSDL firewall reorg

Cool links:

• Google Chrome for Mac OS X is now in beta

WattDepot, now with Google Visualizations

After discussion with Philip, decided to prioritize the Google Visualization data source work over performance testing. WattDepot now provides a data source for the Google Visualization API that allows plotting multiple types of data on the same graph, and expanding virtual sources into their component subsources. I have written a wiki page about using WattDepot’s Google Visualization data source.

Plans from last week:

• Test performance of Derby implementation
  • deferred for now
• Make GVisualizationServlet work with virtual sources
  • done
• Review grant proposal?
  • not done, working on it this week
• Close some more WattDepot open issues
  • closed some issues, but opened more new ones

Other accomplishments from last week:

• Registered for Spring 2010 classes
• Tracked down CSDL’s copy of Snow Leopard Server from UHM Bookstore
• Pushed out new WattDepot release and notified 413/613 students of visualization availability
• Watched some crazy big-wave surfing in Waimea Bay without getting sunburned

Pointers to work products:

• WattDepot subversion repository
• Wiki page on using WattDepot’s Google Visualization data source

Plans for this week:

• Review NSF proposal
• Plan script for 10 minute screencast
• Close some additional WattDepot issues

Cool links:

• Google Goggles is an Android application that lets you take pictures of objects (only some are recognized) and does a search for relevant information. Will do OCR on text found if it doesn’t know about that type of object.

WattDepot has persistence

The Derby persistence implementation is now complete. This week I will start performance testing the Derby persistence implementation, and work on the GVisualizationServlet so we can visualize virtual sources. Loading default data into WattDepot using Derby for storage (no network traffic) is running at ~876 sensor data entries per second, which seems pretty good.

Plans from last week:

• Remove last of HashMap code from DerbyStorageImplementation
  • done, along with some other cleanup (like running unit tests on both DB implementations)
• Start performance testing of Derby storage implementation
  • just started

Other accomplishments from last week:

• Ate lots of delicious food

Pointers to work products:

• WattDepot subversion repository

Plans for this week:

• Test performance of Derby implementation
• Make GVisualizationServlet work with virtual sources
• Review grant proposal?
• Close some more WattDepot open issues

Cool links:

• Turns out you can close Google Code issues with version control commit messages. Pretty cool.
• Mariah Power, makers of the Windspire small-scale wind turbine, have released an iPhone app that supposedly uses the microphone to estimate wind speeds at your location. I have downloaded it, but not tried it yet.

WattDepot Derby storage almost complete

The Derby persistence implementation is mostly complete. SensorData is temporarily still stored both in memory and in Derby, to simplify the initial implementation. The next steps during this short week are to remove the final in-memory tables, and start performance testing the Derby persistence implementation.

Plans from last week:

• Keep working on DerbyStorageImplementation
  • almost done
• Finish implementing missing methods in REST API
  • not done, deferred until Derby work is done

Other accomplishments from last week:

• Wrote up directions on running WattDepot locally

Pointers to work products:

• WattDepot subversion repository
• REIS seminar web pages

Plans for this week:

• Remove last of HashMap code from DerbyStorageImplementation
• Start performance testing of Derby storage implementation

Cool links:

• New acronym I encountered: LOHAS. Lifestyles of Health and Sustainability (LOHAS) is a demographic defining a particular market segment related to sustainable living, “green” ecological initiatives, and generally composed of a relatively upscale and well-educated population segment.

WattDepot presentation & Derby started

Last week was holiday-shortened by Veterans’ Day. I spent most of my time working on my slides for my ICS 690 presentation, and helping the REIS seminar substitute videographer. Now that the presentation is done, I have begun implementing persistence in WattDepot using Derby.

Plans from last week:

• Finish slides for presentation to ICS 690
  • done
• Start working on DerbyStorageImplementation
  • done

Other accomplishments from last week:

• Finally got video from 11/5/2009 REIS seminar uploaded
• Attended ITS update seminar
• Upgraded to 64-bit Cocoa version of Eclipse
• Wrote up my Java 1.6 & Eclipse experiences

Pointers to work products:

• WattDepot subversion repository
• REIS seminar web pages
• Presentation slides

Plans for this week:

• Keep working on DerbyStorageImplementation
• Finish implementing missing methods in REST API

Cool links:

• This is Restaurant Week Hawaii

Java 1.6 & Eclipse on Mac OS X

These are some notes on using Java 1.6 on Mac OS X with Eclipse.

Mac OS X lagged behind for some time on adopting Java 1.6, but it was finally released as Java for Mac OS X 10.5 Update 1. Unfortunately, this release has some limitations: it only runs in 64-bit mode and only on Mac OS X 10.5 (Leopard). This meant no Java 1.6 support on Mac OS X 10.4, no support for PowerPC systems, and no support on the early 32-bit Intel Macs released in 2006.

To make Java 1.6 the default Java execution environment, run the Java Preferences application (found in the Utilities subfolder of the Applications folder). There you can drag the Java versions into any order you wish for both applications and applets. So to default to using Java 1.6 for everything, you can just drag Java SE 6 to the top of both the application and applet lists. The Java Preferences application will change around symlinks to correspond to your new choices. Mac OS X 10.6 (Snow Leopard) appears to include only Java SE 6, but includes both 32 and 64-bit versions, so it should allow Java 1.6 to be run on the oldest Intel Macs (no PPC because Snow Leopard does not support PPC).

Some command line scripts expect the JAVA_HOME environment variable to be set to the directory that contains the Java distribution being used. Also, some scripts might require different versions of Java. To support this, Apple introduced a new utility called /usr/libexec/java_home. By default, java_home just returns the home value appropriate based on the selection in the Java Preferences utility, but command line arguments can request different versions (see the manual page linked earlier). The output of java_home is intended to be assigned to the JAVA_HOME environment variable in a shell initialization script (like .profile or .cshrc).

This brings us to Eclipse, the Java development environment. Eclipse can be used to develop for multiple Java versions. When creating a Java project, Eclipse prompts for the desired JRE version in the new project dialog box. To ensure that Eclipse tracks the language differences between versions, also select the desired version in Preferences->Java->Compiler->Compiler compliance level to match your project.

However, despite these changes, Eclipse itself does not necessarily run using the Java version being used for the project being developed. Initially, the latest version of Eclipse (3.5) was released only in 32-bit mode for both Carbon (an older and deprecated Mac OS API) and Cocoa (the modern Mac OS API). Since Leopard only supported Java 1.6 in 64-bit mode, this meant that Eclipse was always running under Java 1.5. With the release of Eclipse 3.5.1, there are now 64-bit Cocoa downloads available, and these will run under Java 1.6. For some users, this may not be important, but for those doing Eclipse tool development (such as the Hackystat Eclipse sensor) it is very helpful. I use JAXB in some of my Ant build scripts, and JAXB is built into Java 1.6 but not Java 1.5. When running Eclipse in Java 1.5, many of my Ant scripts report spurious errors about being unable to locate JAXBException, but these all vanish when running Eclipse under Java 1.6. I should note that at least some people disagree with this advice, and suggest using the 32-bit Cocoa version on Leopard instead of the 64-bit version. Everyone apparently agrees that on Snow Leopard you want the 64-bit Cocoa version of Eclipse (unless you are on a 32-bit Intel Mac).

So if you want to use Java 1.6 for your application and run Eclipse under Java 1.6 there are two options:

• Use Mac OS X 10.5 Leopard (with all software updates) on a Core 2 Duo Intel Mac, with Java Preferences set to use Java SE 6 as the default Java version, and use the 64-bit Cocoa version of Eclipse 3.5.1.
• Use Mac OS X 10.6 Snow Leopard on an Intel Mac, and use the 64-bit Cocoa version of Eclipse 3.5.1 [note, I have not specifically tested this configuration, but it should work]

Yet more WattDepot work

Last week I got WattDepot ready for the next ICS 413/613 assignment. This included adding fuel types to each of the sources, and then implementing each of the new CLI commands to ensure that they could actually be implemented Along the way I made some changes to the client that only became apparent when running the client independent of the server.

I also changed the server so that it can load default resources from one big file instead of one resource per file. This greatly speeds startup when there are 100K+ resources to load (like Oscar-generated sensor data for 2 months).

I’m still working on slides for my presentation for ICS 690 this week. After that is done, I will shift to working on persistence in WattDepot using Derby.

Plans from last week:

• Turn TODO.txt into Google Code issues
  • done
• Implement instantaneous carbon emission rate resource in WattDepot
  • not done, ended up using emissions over 1 hr of energy as basic measure
• Write up code review description for Philip
  • not done
• Prepare draft presentation for ICS 690
  • in progress
• Make GVisualizationServlet support virtual sources
  • not done
• Start working on DerbyStorageImplementation
  • not done

Other accomplishments from last week:

• Prepared documents for REIS RA renewal application
• Sped up WattDepot server startup from minutes to a few seconds
• Implemented ICS 413/613 CLI assignment
• More problematic video from REIS seminar. Double Arg.

Pointers to work products:

• WattDepot subversion repository
• REIS seminar web pages

Plans for this week:

• Finish slides for presentation to ICS 690
• Start working on DerbyStorageImplementation

Cool links:

• N/A

Refactoring WattDepot

I spent most of last week working on refactoring some cruft out of WattDepot. The core data types (Source, SensorData, and User) now all have proper constructors and all the code that was in utility classes has been folded into the datatypes themselves. This makes the rest of the system easier to read, but it means it will be a pain if I need to modify the XML schema in the future. If I need to make schema modifications, perhaps I will use the JAXB schema compiler in the other direction to go from code to XML schema.

There is still more refactoring that needs to be done, especially around the interpolated resources. I need to start creating issues in the Google Code site instead of my current tracking via TODO.txt.

Next week on Thursday 11/12 I will be giving an hour-long presentation on my research. Philip will be out of the office for the latter half of that week, so I want to have a draft presentation ready by the end of this week. The other important deliverable for this week is a carbon emission rate resource for WattDepot, which will be needed for the next ICS 413/613 assignment.

Other WattDepot features that need to be implemented are persistence and bringing the Google visualization servlet up to the rest of WattDepot (supporting virtual sources, authentication, etc).

Plans from last week:

• Make list of refactoring changes to implement
  • mostly done, should be turned into Google Code issues
• Start designing real persistence layer
  • not done. Philip and I agree that for expediency’s sake I should stick with Derby for now
• Make Saunders demo with virtual source Google visualization?
  • not done, requires visualization data source improvements
• Start planning for next 413/613 assignment
  • done, only new feature needed is instantaneous carbon emission rate
• Write WattDepot server installation and configuration wiki page
  • deferred: without real persistence, probably nobody should be using WattDepot just yet, so no point in writing up installation instructions that will need to be changed

Other accomplishments from last week:

• Produced a list of “known good” values from the current WattDepot public server for the ICS 413/613 students
• Wrestled with bad video from REIS seminar, probably due to dirty camcorder heads. Arg.

Pointers to work products:

• WattDepot subversion repository
• REIS seminar web pages

Plans for this week:

• Turn TODO.txt into Google Code issues
• Implement instantaneous carbon emission rate resource in WattDepot
• Write up code review description for Philip
• Prepare draft presentation for ICS 690
• Make GVisualizationServlet support virtual sources
• Start working on DerbyStorageImplementation

Cool links:

• Google Wave is pretty interesting (just got an invite last week). In theory, it seems pretty useful, but until many of the people who I collaborate with have accounts, it’s hard to tell if it will be useful in practice.

WattDepot first release available

As usual, the last 10% took longer than expected, and I spent all last week finishing WattDepot up. It now has all the functionality required by the first WattDepot assignment in ICS 413/613. Virtual sources are working (except for some exceptional cases), and interpolated resources like power, energy, and carbon all work. There’s also a new wiki page explaining how to use the WattDepot client library. To help those unfamiliar with the physical concepts behind electrical power, I wrote a page explaining those concepts. There is now a public distribution of WattDepot available, but no installation documentation so not really useable outside the class context yet.

While getting WattDepot to this point was pretty exhausting, there’s still a laundry list of refactoring and features needed to make it more generally functional. I’ll be working on that this week.

Plans from last week:

• Finish implementing Power resource GET for a timestamp
  • done
• Design and implement Energy resource GET for a range of times
  • done
• Design and implement Carbon resource GET for a range of times
  • done
• Refactor some of the bogus WattDepot code that has accumulated
  • started, but much more to be done

Other accomplishments from last week:

• Discovered that PMD failures were not failing Hudson builds for projects derived from hackystat-developer-example. Fixed for wattdepot, oscar-project, and hackystat-developer-example, but others could be lurking.

Pointers to work products:

• WattDepot subversion repository
• REIS seminar web pages
• WritingWattDepotClients page
• PhysicalConcepts page

Plans for this week:

• Make list of refactoring changes to implement
• Start designing real persistence layer
• Make Saunders demo with virtual source Google visualization?
• Start planning for next 413/613 assignment
• Write WattDepot server installatoin and configuration wiki page

Cool links:

• UHM thesis/dissertation LaTeX style. This has been worked on by many hands, and I recently passed on the link to ICS MS student who passed his defense and now needs to work on formatting.

Oscar support almost done

As planned, the past week has been all about building out the WattDepot functionality required by Oscar, specifically the first assignment using Oscar data in ICS 413/613. I’ve made massive progress, but still not quite finished with the requirements for the assignment. Planning to finish that up today, and then probably do some refactoring and prioritization of additional WattDepot functionality.

Plans from last week:

• Finish input client for OSCAR data
  • done
• Design and implement Source summary resource
  • done
• Implement GET SensorData for a range of times
  • done
• Design and implement Power resource GET for a timestamp
  • done for normal sources, almost done for virtual sources
• Design and implement Energy resource GET for a range of times
  • not done
• Design and implement Carbon resource GET for a range of times
  • not done
• Implement virtual sources
  • done, though as new features are added they will need virtual source loving

Other accomplishments from last week:

• Discovered that PMD failures were not failing Hudson builds for projects derived from hackystat-developer-example. Fixed for wattdepot, oscar-project, and hackystat-developer-example, but others could be lurking.

Pointers to work products:

• WattDepot subversion repository
• REIS seminar web pages

Plans for this week:

• Finish implementing Power resource GET for a timestamp
• Design and implement Energy resource GET for a range of times
• Design and implement Carbon resource GET for a range of times
• Refactor some of the bogus WattDepot code that has accumulated

Cool links:

• UH supports authentication pass-through by MAC address on its WiFi network now. I’ve wanted this for like 3+ years, nice to see it finally happen.