Monthly Archives: March 2010

The null ritual

Philip and I were discussing the design of my dissertation experiment, and he pointed me at an interesting book chapter titled “The Null Ritual: What You Always Wanted to Know About Significance Testing but Were Afraid to Ask“. It’s fascinating reading, as it walks through a lot of false beliefs about significance testing as used by psychologists in experiments. I found that my understanding of significance testing was definitely incorrect in the ways described in the chapter.

The “null ritual” from the title is described as:

  1. Set up a statistical null hypothesis of “no mean difference” or “zero correlation.” Don’t specify
    the predictions of your research hypothesis or of any alternative substantive hypotheses.
  2. Use 5% as a convention for rejecting the null. If significant, accept your research hypothesis.
  3. Always perform this procedure.

The problem is that the null hypothesis test is p(D|H0), or the probability of obtaining the observed data given that the null hypothesis is true. When doing an experiment, any real world scientist will have a hypothesis that they are testing and usually hope that they can prove that it is true using the data from the experiment. What we really want is p(H1|D), or the probability of our hypothesis being true given the observed data. However, we need Bayes’ rule to draw a conclusion about the hypothesis and that requires the prior probabilities of the hypotheses, which are often not available to us beforehand.

The chapter also brings out the controversies in statistics between different approaches and goals of particular techniques, which is usually glossed over in teaching of statistics.

I’m planning to follow the authors’ recommendation in my research: “In many (if not most) cases, descriptive statistics and exploratory data analysis are all one needs.”

Advertisements

Rebuild Hawaii Consortium March 2010 meeting

I attended the Rebuild Hawaii Consortium quarterly meeting last week. I had never attended any of their meetings before, and I was somewhat surprised at the sizable number of people in attendance (40? 50?). It was held in a large stadium-style conference room at the Hawaii Convention Center. I had checked the agenda in advance, and thought I could arrive at 10 AM and still see everything I wanted to, but apparently the agenda changed since it was posted on the website.

The talk I missed that I wish I had seen was by Luis Vega on the Hawaii National Marine Renewable Energy Center. His slides look very interesting, lots of hard-nosed cost comparisons of wave and OTEC electricity generation.

Paul Norton have a talk on Zero Energy Buildings, which was interesting. I attended his REIS seminar where he covered some of the same things, but this was focused on ZEB. Some points I found particularly interesting:

  • The introduction of air conditioning leads to a 70% increase in electricity use
  • The key conceptual shift is thinking about the monthly cost of a home being the mortgage + utility bill.
  • The efficiency / photovoltaic balance point is the point at which adding generation via PV is the same cost as additional efficiency measures
  • A cost neutral design (monthly cost is same as a home built to code) that uses efficiency and PV results in an 85% reduction in home electricity usage
  • Once major efficiency measures are in place (solar water heating, efficient lighting & air conditioning, insulation), the major remaining load is appliance plug loads
  • In one military housing complex on Oahu, there is a 4x difference in electricity usage between houses with identical efficiency measures. Presumably the differences are due to appliance purchases and behavior.
  • In a group of homes in Las Vegas, the difference was 5x
  • Further, the differences were fairly continuous: there is no nice average plateau
  • PV inverters on the neighbor islands have been causing problems because the utility frequency can sag during periods of high usage. By default, the inverters are set to disconnect from the grid when the frequency drops below 59.3 Hz, so inverters all over turn off, which puts additional strain on the utility, exacerbating the problem. Reducing that threshold frequency to 57 Hz can help. Thus there is a lot of research still to be done on renewable integration.

Another presentation was on HCEI and smart grid initiatives at PACOM. They are working on a project called SPIDERS that is trying to address the fact that access to electricity is a critical need for the military. One thing I was stunned to learn was that people living in military housing don’t pay for electricity! Thus they have no financial incentive at all to reduce their energy usage. Slide 8 shows an actual graph of HECO’s demand and generation for one particular day. Our work on OSCAR was all based on vague outlines of what the demand curve looks like, so it was great to see it “in the flesh”.

There was a lot of good information at the meeting, so I’m planning to attend in the future. Next meeting is June 2.

Community-Based Social Marketing workshop

Today I attended an all day workshop on Community-Based Social Marketing by Dr. Doug McKenzie-Mohr. While it may have originally been intended as a workshop, I think the number of participants was doubled to over 100 people, so it ended up being mostly a long lecture with a lot of question-taking.

I was already somewhat familiar with the CBSM from reading a condensed version of the method on the web. CBSM is an attempt to modularize and standardize a process for “fostering sustainable behavior” across a range of domains. It’s based on a wide variety of social psychology research on how to get people to actually change their behavior. One major takeaway is that big mass-media campaigns to promote behavior change are not very effective for the money spent on them.

CBSM process has 5 steps

  • Selecting the behavior(s) that you wish people to adopt
  • Assess the barriers and benefits people face adopting the behaviors
  • Develop strategies to foster the behavior changes
  • Run a pilot project that tests your action plan
  • Implement your plan broadly and evaluate its effectiveness

The development section suggests the use of a variety of psychological “tools” that can help people change their behavior such as making public commitments, social diffusion, social norms, and prompts. The design of the Kukui Cup competition and the supporting website is already strongly influenced by these tools.

The target audience for the workshop and CBSM in general are government and NGOs that have a mission to foster some type of behavior change, which is somewhat different from our situation with the Kukui Cup. The Kukui Cup can be thought of as a sort of applied research into using these CBSM techniques but with a very narrow market segment (first-year college students) and making extensive use of a customized website. Rather than focus on encouraging a small number of behaviors in participants to achieve our goals of increased energy conservation and energy literacy, we are giving participants a smorgasbord of options via the website and trying to figure out which ones work the best based on what the participants do.

I was struck by how little the Internet and WWW were mentioned in the workshop. CBSM is clearly labor intensive compared to a traditional mass media campaign, but the claim is that CBSM delivers better results (i.e. more desired behavior) than informational campaigns. From a CBSM perspective, the Kukui Cup seeks to determine how much of that traditional CBSM labor can be embodied in the website, and whether the web-based CBSM retains the effectiveness of traditional CBSM. Does a web community provide the same benefits as real-world community? At least with the Kukui Cup the web community will mirror a series of small real world communities: floors of a dorm.

That said, there are some cautions from the workshop that are worrisome. There was a lot of emphasis on surveying the people the campaign is targeting, and running a pilot study. This makes a lot of sense, and from a certain perspective, the first Kukui Cup could be thought of as a pilot, though it will be a bigger pilot than most due to the infrastructure required to make it a competition. Obviously the risk is that the whole Kukui Cup could flop (nobody uses the website, general apathy towards the competition, no energy conservation, etc), which would pose significant problems for my graduation timeline. 🙂 Doug related a story of his own biggest failed campaign, which turned out to be his own dissertation project!! At great expense he created a media campaign across Canada that cost $100K to encourage people to participate in policy meetings in their communities. The ads included a 1-800 number, and his organization braced for tens of thousands of calls. Number of calls received: 8.