The Annual Fedora Linux Upgrade

For the past several years, the end of the year has been the time for me to upgrade the home Linux system from Fedora N to Fedora N+2.  The Fedora people have the reasonable policy of only providing updates on the most recent two Fedora Linux distributions, and they release a new distribution every six months.  So, in order to keep my home Linux system up-to-date, I upgrade to the latest Fedora distribution once a year.  I began with Fedora Core 2, am now running Fedora 12, and have run all the even-numbered releases in between.

This year’s upgrade seemed like it had more than the usual number of issues, in part because I’m doing more things with my Linux system these days.  Since some of them might be avoided had I been better prepared, I thought I would describe what I ran into.

Performing the Upgrade

I downloaded the Fedora 12 DVD distribution via BitTorrent soon after it came out.  As part of the reciprocal nature of BitTorrent, I left my client running until it had uploaded at least two times as much data as I had downloaded.  I suggest others do the same.  I verified the checksum on the distribution and burned a DVD from it.

The DVD booted fine, but the installer wouldn’t recognize the DVD drive when it came to performing the upgrade.  This is apparently a long-standing problem that has existed since Fedora 7, which has been closed as a “can’t be fixed” bug.  I haven’t gone through all of the text in the bug in detail, but I have trouble understanding why the installer, anaconda, can’t talk to the drive any more.  I must have run into this on either the Fedora 8 or Fedora 10 upgrades, but other than a vague sense of déjà vu, I don’t remember solving this problem.

Fortunately, one of the options available is a network installation.  So I created a share on my network-attached storage box, and loaded the distribution DVD onto it.  This is done by mounting the ISO image as a “loop” filesystem:

# mount -o loop,ro Fedora-12-i386-DVD.iso /mnt
# mount -t cifs -o username=admin //nas/Fedora12 /mnt2
# cd /mnt2
# tar c /mnt | tar xv

One of the options was to install via http, but after enabling http on the NAS and attempting the installation, the installer crashed.  I then enabled NFS and used it successfully for the install.

Graphics

I have an Nvidia graphics board in my system, and as a result the upgrade didn’t have working graphics when it installed.  I disabled the ‘nouveau’ video driver (although not sure that was necessary) and installed the nvidia_kmon package and graphics started working again.

Non-standard software

As expected, I needed to recompile the sendmail ‘milters’ that I use for SenderID (sid_milter) and DKIM (opendkim), since they’re not part of the standard Fedora distribution.  I also found that I needed to re-install the python_daemon package used by my solar power data connection daemon.

PostgreSQL

I had only recently started using PostgreSQL, since it is used by the DAViCal calendar server I installed recently.  I discovered the hard way that new versions (such as 8.3->8.4) of PostgreSQL change the structure of the data they store on disk in a way that is incompatible with previous versions.  I should have backed up the PostgreSQL database prior to the upgrade, but didn’t (since I had backed up the entire disk).  After checking a few machines I have access to and finding the proper (old) version of PostgreSQL on none of them, I gave up and recreated the database.  I hadn’t lost that much because we had just started using the calendar server.  A lesson for the future, though.

BIND

I operate a BIND name server on my system for my domain and as a secondary for a couple of friends’ domains.  My DNS has two views:  An internal view that includes hosts on NATted networks available only internally, and an external view that shows only public addresses.  The internal view allows recursion since internally we use my Linux machine as the primary name server, but this stopped working.  A little research revealed a change that was made in BIND 9.4.1P1 on the configuration of allowing recursion.  Rather than the ‘recursion no;’ or ‘recursion yes;’ statements in the view definitions in the BIND configuration file, these statements are needed in the general section of the file:

allow-recursion { mylan; };
allow-query { any; };
allow-query-cache { mylan; };

DAViCal and PHP

The DAViCal calendar server software uses PHP extensively.  While I have had a little experience with its use in MovableType and MediaWiki, I was unprepared when all I got was a white screen (in the browser window) when trying to use DAViCal.  At first I thought that there was something wrong with my Apache web server configuration, but after considerable testing found that PHP was working correctly, but something happened with respect to DAViCal since Fedora 10.

After much flailing, I found that the configuration line (in the suggested DAViCal configuration for httpd):

php_value error_reporting "E_ALL & ~E_NOTICE"

doesn’t work because the error flag symbols aren’t available to Apache so all error reporting was turned off.  Replacing this with a value of 22527 caused errors to be reported (this value also disables warnings about deprecated code usage).  It was now visible that the setting

php_value open_basedir 1

is the wrong thing; open_basedir takes as its value the list of directories that should be available to PHP scripts.  I replaced this with:

php_value open_basedir "/usr/share/davical/:/usr/share/awl/:/etc/davical:/tmp"

Including /tmp in the open_basedir is perhaps not a best practice, but in this stance it seemed like it would be OK.

With respect to both of these changes, I wonder how it ever worked under Fedora 10.

Reverse Path Filtering

At this point, I should point out that my Linux machine has two interfaces, one to the network with public IP addresses, and one to the NAT (network address translation) network.  The Linux machine does not route packets between the networks though; this and the NAT function is done by a Cisco 881 router.  The primary reason for the interface to the NAT network is to give the Linux machine a direct gigabit Ethernet connection to the network-attached storage box.

I noticed that some nodes within the house were mysteriously not working.  After considerable packet sniffing and head scratching, I found that it was not possible to ping the public IP address of the Linux machine from the NAT network.  The failing hosts were making DNS requests to that public address, which were also failing.

It turns out that by default Fedora 12 is configured by default to perform reverse path filtering, whereby incoming packets are ignored if they aren’t coming from the interface having a route to the source address.  In order to change this, one needs to edit /etc/sysctl.conf to change the line:

net.ipv4.conf.default.rp_filter = 1

The right-hand side needs to be changed from 1 to 0.  I’m not positive of the default in Fedora 10, but I don’t remember changing this before so I suspect that the default has been changed.

Summary

Having made the above changes, Fedora 12 is working fine, and it’s definitely worth it to be on a release that’s getting bug-fix updates, especially since some of those correct security vulnerabilities.  Perhaps I just wasn’t diligent enough in reading the release notes this time.  In any case, if I run into any problems upgrading to Fedora 14, I’ll know what the new issues are.

Calendars for the Family

For some time, my wife and I have been looking for a way to coordinate our family calendars electronically.  Some time ago, I purchased an iPod Touch and thought its calendar might be appropriate, but at the time the only synchronization it supported was with Microsoft Exchange or with Apple’s MobileMe service.  Since I already operate a Linux server at home, I’m not willing to also invest in MobileMe, and running an Exchange server seemed like too much effort just for calendar synchronization.

Apple introduced CalDAV support in the iPhone 3.0 software last June, which cleared that particular logjam.  One option is to use a service like Google Calendar as the calendar repository, but my wife doesn’t want to store her calendar off-premises.  This gives me a perfect excuse to learn something about calendaring by running a CalDAV server on our home (Fedora) Linux server.

I started by checking to see whether there was an installable CalDAV package for Fedora, but there isn’t just yet.  It seems that the Fedora community is looking at that for a future Fedora release, but hasn’t decided on what to include yet.  So I started searching around for “CalDAV Linux” and found several candidates.

Darwin Calendar Server — I started with this calendar server, which seemed to be a popular choice.  As the name Darwin suggests, this is a calendar server that is oriented toward MacOS, although it will run under other Unix/Linux variants.  This largely manifests itself in the locations that it keeps various files.  Many of those using this server seem to be using it in “test” mode via the included “run” script.  I got the server working briefly this way, but the documentation in that script warns against using it for production.  Instead one is supposed to install the server “properly” — a task I found to be difficult to do.  It seemed to scatter bits and pieces of the server and packages on which it was dependent (such as the Twisted programming framework) across many different and unfamiliar places on my system, which in turn required other changes.  I never did get the server working in this production mode, and soon went looking for an easier solution.

mod_caldav — After the experience with Darwin, this package, which is an extension module for the Apache HTTP Server, looked very attractive.  After solving several dependencies, I ran into build errors with one of them, mod_dav_acl.  It turns out that one first needs to patch the Apache HTTP Server before building the module.  That wasn’t at all attractive, because it meant that I would now need to manually apply any changes to the HTTP Server, such as security patches that were distributed as part of the Fedora update process.  Also, the patches applied to an earlier version of the code, and not the version I was running.  I gave up on this one, too.

DAViCal — This CalDAV server, previously known as the Really Simple CalDAV Server (RSCDS), was the next one I tried.  It’s “native” to the Debian variant of Linux, which uses a somewhat different packaging arrangement for software distribution from Fedora, but is otherwise quite similar.  Again, there were a few dependencies that needed to be taken care of, but generally these were available by installing the necessary Fedora packages.

Installation of DAViCal went fairly smoothly.  It uses PostgreSQL as its underlying database, which I had used only rarely as most other applications I have use MySQL.  I got DAViCal running relatively quickly, although I had to work around a couple of glitches in the installation documentation that I have since reported.  I have successfully used it with iCal, iPod touch, Sunbird (PC), and Lightning (both Mac and PC).

Sharing the family calendars required that I learn a bit more about CalDAV.  Using DAViCal’s excellent Web administration interface, it was easy to create an account for each member of the family and set up the desired access permissions.  I then attempted to log in as myself and create calendars under their names.  Unfortunately this meant that now I had a private calendar under my account with their name on it; I wasn’t sharing anything at all.  It took a bit of manual URL entering to actually share calendars with family members.  Different clients seem to handle this somewhat differently:  For example, iCal and iPod Touch require that you create separate accounts (with the same username and password) for each of the calendars you want to share in this manner.  Lightning fortunately doesn’t.  Also, the Apple clients default to different ports and a somewhat different URL structure than is native to DAViCal.  This can be dealt with if you’re willing to do a little work with the Apache configuration files.

At this point, and having “given” calendar services to my family for Christmas, it got to be time to update from Fedora 10 to Fedora 12 (more on that in an upcoming blog post). This hurt in a couple of different ways.  First, the Fedora update changed the version of PostgreSQL, which changes the on-disk layout of the database in an incompatible way.  PostgreSQL assumes that you have backed up the database before doing the upgrade, and as a PostgreSQL newbie, I hadn’t.  Fortunately we hadn’t populated the calendar much yet and could afford to start over.  Second, PHP didn’t seem to work any more.  A couple of parameters in the DAViCal documentation were set incorrectly; I’m not sure how they worked under Fedora 10 but I’m working with the DAViCal folks on getting the documentation corrected.

Was it worth it?  Absolutely.  My family loves having calendars they can all refer to and update that show what everybody is doing.  I just need to keep that database well backed up now!

An Unexpected Waste of Energy

We recently moved the furniture in our bedroom, enabling me to use my trusty Kill-A-Watt to measure the energy consumption of some things that were hard to reach previously.  By now we’re quite aware of what the consumption of TVs, radios, and satellite receivers is.

We got quite a surprise, however, when we measured the consumption of our electric mattress pad when turned off.  Like many contemporary mattress pads, ours (labeled “Safe & Warm”) uses low voltage in response to concerns that many people have about adverse health effects of electric fields.  There is a power supply that sits on the floor and provides low voltage to the heating element, and also connects to the user control (on/off switch and temperature dial).  As a dual-control mattress pad, there were two controls and power supplies.

Mattress pad power supply

My first hint came when picking up the power supply.  It had not been used for several months, yet was warm.  The Kill-A-Watt told the story:  4 watts of standby power consumed while turned “off” for each side of the bed.  That works out to 70 kWh per year, or a cost of $10.50 per year at 15 cents per kilowatt-hour.  I’m really kicking myself for not unplugging the pad during the summer months, at least, during the 5 years or so that we had it.

Unlike the satellite receiver (which downloads program information from the satellite while on standby) and the TV (which is waiting for commands from its IR remote control when on standby), I can’t think of any justification at all for the mattress pad to be consuming 8 watts continuously.

A good clue was a small label on the power supplies:  “When operating properly, this power supply remains warm.”  But I expected that meant when it was operating, not when it was “off”.  It’s ironic that one of the selling points of mattress pads such as this is that they allow one to save energy by turning down your home’s thermostat in the winter.

We’ll be shopping for a new mattress pad soon, and from what I have been able to tell nearly all of them seem to use about the same general design.  Hopefully the design has been improved in the past few years.  I’ll probably be taking the Kill-A-Watt with me to stores as we decide on a new one.  I’ll add a comment below if I find one that seems particularly good.

Energy WHAT?!

We were moving some furniture this past weekend, and I was moving the satellite receiver in the bedroom when I came upon this:

That’s right, it’s the satellite receiver I wrote about a couple of weeks ago, the one that takes 11 watts turned on and 10 watts turned “off”, and it has an Energy Star sticker on top.  But I associate Energy Star with computer monitors (especially the older CRTs) that turn themselves almost completely off when not in use.  How is this possible?

I went to the Energy Star website to research the criteria for set-top boxes to be listed as Energy Star compliant.  According to the page on set-top boxes for consumers:

Set-top boxes that have earned the ENERGY STAR are at least 30 percent more efficient than conventional models.

I’m not at all sure what a “conventional model” is.  Presumably there are satellite receivers that are even worse, but DirecTV offers a very limited choice sso I’m not sure you can actually get one.  So I went on to the information for “partners” (equipment manufacturers).  The program requirements list a number of different types of set-top boxes (such as cable, satellite, IP, terrestrial), along with optional capabilities (additional tuners, advanced video processing, DVR, and so forth).  Limits are given in kWh/year for each type of set-top box, with additional allowances for each additional capability present.  There are two tiers of limits: Tier 1, effective January 1, 2009, and Tier 2, effective January 1, 2011.

The Tier 1 limit for a satellite set-top box with no additional capabilities is 88 kWh/year.  But what bothers me is how that’s calculated:

kWh_Base = 0.365 * (14 * P_tv + 10 * P_sleep)

In other words, the formula assumes that the receiver is on 14 hours a day.  While some people must use their TVs that much, it’s hard to believe that this is typical, even in the US.  The result is that Psleep (the power consumption when turned “off”, more accurately characterized as sleep mode) does not count for as much as it should.

Under this formula, my set-top box consumes 93 kWh/year.  This doesn’t qualify under the current limits, but it’s very possible that they were different in 2004 when it was built. The criteria continue to tighten; the Tier 2 limit, effective 2011, is 55 kWh, a step in the right direction.  But they need to rethink the duty cycle in the test criteria, as well as whether these are actually 30% more efficient than anything, as claimed on the consumer Energy Star website.

What I don’t like about RSS

Really Simple Syndication (RSS, and a similar protocol, Atom) has been touted for several years as the way to keep track of many things: blog posts and comments, wiki edits, news headlines, even email messages and Twitter feeds. Nevertheless, email remains the de facto notification mechanism for most people and applications. I use RSS quite regularly to manage my blog reading, but I’m not very satisfied with it. Here’s why:

RSS is a pull medium

Any time I want to use an RSS reader, I need to ask my RSS reader to collect (pull) the RSS summary from each of the “feeds” (typically blogs) that I subscribe to. This works reasonably well for content that changes relatively slowly like many blogs, but works less well for headlines and especially Twitter. There is no mechanism for notifying me when something new shows up; I need to poll for it.

Lack of multi-device integration

I read blogs from at least three devices: my home machine, my work laptop, and my iPod Touch. I also read my home email from all of those places. The difference is that when I read an email message, all three places show the message as having been read. Not so with RSS. The devices don’t have a way of communicating with each other, so each time I change devices everything looks unread. Subscriptions are similarly disaggregated; I have to subscribe to each feed on each device. I’d rather, when I find a new blog (or something) worth reading, to be able to add it to all devices at once.

Poor reading organization

Rather than organizing the information by feed, I’d rather have a more flexible way of prioritizing the information. This is more of a client software problem than a protocol problem, but it’s still a reason I don’t like RSS as much as I might. I’d like it if my RSS reader organized the information it presents to me by subject (perhaps using keywords or tags to do so), or according to a more complex sorting algorithm that either I could specify or it would learn from my reading behavior.

Older information is lost

RSS feeds typically syndicate only relatively items, and are often limited to only the most recent N articles, where N is typically 20 or so. If N increases, the size of the feed increases proportionally, as does the overhead for the reader which has to match the current feed against its stored database to see what has already been read. However, lower values of N fail if the reader doesn’t regularly retrieve the feed. One currently has to expect that articles may be lost when you go on vacation, and that limits RSS to non-critical tasks.

---

Unless we want to continue to use email as a notification mechanism for everything, it would be useful to have a syndication mechanism that doesn’t suffer from the limitations of RSS (or Atom). A new mechanism could be designed that is more tailored to this purpose than email, and can be designed from the start to avoid the many ways that email is abused.  For example, Mark Cuban recently talked about the use of WebHooks and PubSubHubbub to overcome the pull limitations of RSS and similar protocols.  These and other (push) protocols have the potential to make the use of the Internet a much more responsive, event-driven experience.

Thoughts on the Cash for Clunkers Program

I’m going to stick my neck out a bit and blog about something that I don’t think I know that much about:  economics.  Hopefully some astute reader will set me right through some insightful comments if I’m all wet.

I have been watching the Cash for Clunkers feeding frenzy with some concern.  The government contributed first $1 billion, then another $2 billion, to incentivize consumers to trade older, less efficient vehicles for new, somewhat more efficient ones.  This was billed as both an environmental and economic incentive.

On the environmental side, many commentators have observed that the benefits aren’t entirely clear:  the program only required a modest improvement in fuel economy, which might be overshadowed by the environmental impact of manufacturing the new car and disposing of the old one.  This is a concept known as life cycle assessment, which I find fascinating but which hasn’t yet, to my knowledge, analyzed something as big and complex as an automobile in detail.  The disposal cost is aggravated by the requirement that the car being returned be rendered undriveable.

On the economic side, I’m all for stimuli that create jobs by building infrastructure.  The most famous example of this was FDR’s creation of agencies such as the Works Progress Administration and the Tennessee Valley Authority during the Great Depression.  These projects created infrastructure, much of which we’re still using, that enabled future growth in addition to creating jobs in the short term.

But I see a fundamental difference with Cash for Clunkers.  What’s happening generally is that people are accelerating their purchases of new vehicles using the program.  With the exception of the remaining life on the “clunkers” being destroyed, the program isn’t creating new demand, it’s accelerating future demand.  In other words, it’s mining demand that might occur during the recovery from the current recession and pulling it into the present.  It’s making the current recession a bit shallower but wider, and not much different in total volume, at least as far as the automotive industry is concerned.

Are there benefits to a shallower recession?  Sure.  Not as many workers will be affected, and those that are might reasonably make career changes to other industries.  This is much harder than it seems, though — these other industries may not exist where the workers live, and it’s very hard to sell a home in an area with a weak economy.  I’m also concerned when I hear that automakers are ramping up capacity to meet the demand of Cash for Clunkers; it is such a short-term program that this is likely to be the wrong thing to do.

Many of the things in President Obama’s stimulus plan build infrastructure for the future, but I wish there was more emphasis there.  I won’t be at all surprised if I hear that the automotive industry takes longer to recover than other parts of the economy.  After all, we have already gotten the clunkers off the road.

Power Switches that Aren’t

A few weeks ago, I noticed a colleague at work had a Kill-A-Watt power usage meter, which he allowed me to borrow.  It’s a great device, and relatively inexpensive, so I quickly bought one for myself.

I went around the house measuring all sorts of devices, particularly things like my networking gear that is on all of the time, some of the computers, and entertainment equipment.  As expected, many of these devices draw some power in their standby modes; this is necessary if they have a clock, or can be turned on via a wireless remote.

I was expecting the DVR (DirecTV HD model) to take a significant amount of power on standby, but I was shocked by the comparison between “off” and “on” power usage:

On: 40 watts
Off: 38 watts

That’s right, there is only a 2 watt difference between being turned on and turned off!  It seems that all that the “power” button must do is to turn on the front-panel LEDs and turn on the video output circuitry.  Of course, there are a lot of nice features you get by having the receiver effectively on all the time:  the program guide is continuously up to date and the remote recording feature allows one to request a recording via the Web or an iPhone app.

So I checked our bedroom set, which is connected to an older-model, non-HD, DirecTV receiver:

On:  11 watts
Off: 10 watts

Only 1 watt different!  The program guide still needs updating for this receiver, but of course one can’t remotely request a recording from it.

Does it have to be that way? I’d be happy with a program guide that’s a few hours old, and with remote recording that requires that I give it a little bit of advance notice.  So why not have the receiver (and recording processor and disk in the DVR) switch to a “real” standby mode most of the time, with a timer that wakes it every so often to receive instructions and updates from the satellite?  It would also wake when a remote-control or front-panel command is received.

How much power are we talking about? At the end of 2008, DirecTV had 17.6 million subscribers.  Let’s assume (I think very conservatively) that there are 10 million DVRs and 10 million non-DVRs (since some subscribers have more than one TV).  Suppose, through power cycle management, we can reduce power consumption 90% of the time to 2 watts for the non-DVRs (typical of home audio equipment) and to 10 watts tor the DVRs (typical of a sleeping home computer).  That works out to a savings of 324 megawatts, just for DirecTV customers.  Other satellite receivers, DVRs, and cable set-top boxes have this problem as well, although I don’t know to what extent.

Another way to look at this is that a more efficient DVR could save about 221 kWh per year, and a more efficient non-DVR receiver could save about 65 kWh/year.

The irony is that on both receivers, the button is labeled “Power”.  It’s almost as if they felt they needed that button as a feel-good measure.

Airline Code (somewhat) Sharing

My family and I just returned from vacation in Hawaii, and experienced “code sharing” on United Airlines with rather unexpected results.

For anyone not familiar with the term, code sharing is the common practice of airlines to co-brand a partner airline’s flight with their own flight number.  This is often seen on long-haul and international flights, where a United flight might be co-branded with flight numbers from partners like Air New Zealand or Lufthansa.  In this case, we were travelling on Island Air flight 417 from Kona to Maui, sometimes known as United 4978.

The code sharing process is far from seamless:

Appearance on reservation systems: Some reservation systems and not others showed the existence of UA 4978.  Although I used various third-party reservation systems to search for flights, I booked directly with United so that I could use a discount coupon from a previous delayed flight (more on that below).  But United’s website didn’t show the inter-island flight; it tried to route me from Kona to Maui via Los Angeles.  I had to call and speak to a reservation agent in order to book the itinerary with the codeshare flight.  This has got to be a bug; I can’t imagine that United intends it this way.

Coupons don’t work: I explained to the agent that I intended to use a $200 discount coupon, and she helpfully split my reservation from that of my wife and daughter so that I could do that (although I wonder why that is necessary) and she told me I could purchase the ticket at a United ticket counter.  I did so the next day, and was told that I couldn’t use the coupon on that itinerary, because of the Island Air flight.  Apparently the intent is to make it as hard as possible to use those coupons (a fact we should all heed when being asked to give up our seats on overbooked flights), but United should consider the itinerary “all United” when their flight number is on all the flights.  They did attempt to split the Island Air flight into a separate itinerary, which would have increased the fare by about $160, and I didn’t consider that to be good value for my $200 coupon.

Information system disconnect: When we arrived at the Island Air ticket counter at Kona, my daughter’s E-ticket was listed under the name “Change Name”.  After much pressing of keys and a conversation with someone, and asking how we booked the reservation, the agent told us that she couldn’t change the name; United would have to do that.  We walked with her over to the United counter where she was listed under the correct name and they issued a paper ticket for her, which worked around the problem.  At this point I was very glad that I had booked directly with United; they would undoubtedly have tried to blame the booking agent had I not booked directly.

Airlines would better achieve their objectives with code sharing by working more closely with their code sharing partners to make the process more cooperative and seamless, and would provide better customer service as a byproduct.  In addition, they should be required to treat flights on which they apply their flight numbers as their own to the greatest extent possible so that the “operating carrier” is truly that and that code sharing isn’t an excuse to shrug and say, “Well, it isn’t our flight.”

Photovoltaic, Part 3: Results

We have had the photovoltaic system in place for about 2 1/2 years now.  What are the results?  The simple answer:  We haven’t paid more than a metering charge since.  But there’s a lot more to it than that.

In the first year, the system generated 7380 kWh of energy, 3977 kWh from the south array and 3403 kWh from the west array.  Our net energy usage for that period was 1182 kWh (we used 1182 kWh more than we generated), but our bill was zero because we generated more than we used during peak hours.  Here in California, Pacific Gas & Electric averages the usage for “net energy metering” customers on an annual basis, which takes into account seasonal variations in usage.  They send monthly statements summarizing this, such as this bill at the end of the first year:

As I mentioned in the last post, the inverters have serial interfaces that allow one to read, in real time, the status of the unit.  Xantrex has free monitoring software (for Windows) available, so when the system was first installed I had it hooked up to my Windows PC in order to generate a year’s worth of baseline data.  The Xantrex software generated a CSV file each day with the day’s measurements, plus another file that summarized the daily output of the system.  I was able to generate graphs fairly laboriously by importing the data into Microsoft Excel.  But it was great to see what the system was doing and to verify that both arrays were producing power as expected.

For the most part I stopped collecting data after the first year, in part because it seemed like a contradiction to keep a PC running mostly to collect data from the photovoltaic system.  But I do have a Linux system that runs all of the time (it is my mail server), so earlier this year I made a programming project out of importing the old data into MySQL and writing an application that would query the inverters periodically and insert the results into the database.  The result now is that it’s fairly easy to generate graphs like the following:

Graph for May 17, 2009

I had been noticing this year that the output from the system seemed a bit lower; I had remembered having several 30 kWh by this time in 2007, but none this year.  I compared the graph above against the baseline two years earlier and found the earlier graph to look almost exactly the same, but higher.  The output of the system had decreased about 8.5% over the two years.  What happened?  It appears that, even though the panels didn’t look excessively dirty, they’re quite sensitive to dirt.  On June 1, I cleaned the panels with a long-handled mop and squeegee, and it appears that things are back to 2007 levels (we have had an unusual cloudy spell so it’s difficult to be more definite).  I expect to be cleaning the panels more often in the future.

You’ll also see that the output from the west array isn’t that much lower than the south array.  The difference is more pronounced in the winter (due to sun angles and shorter days) but I suspect that the west array is actually contributing more financially due to time-of-day metering.  Morning fog can also cause the west array to be more productive than the south array.

A similar graph taken during the winter months tells a very different story:

In addition to the shorter day and lower sun angle, this graph shows the impact of shading on array output.  This is probably most visible on the west array about 2 PM.  As the sun angle goes down, objects further away can shade the array from the sun.  Even relatively small partial shading of the array has a major impact.

Here’s a summary of the output of the system over the course of a year (2007):

System output for 2007

As you can see, the output of the system varies widely between winter and summer.  In addition, you can see more consistent results during the summer, due to the relatively dry and sunny summers here in California.

One of our neighbors to whom I showed our data collection and analysis setup described it as having significant “toy value”.  He’s right; I have had a good time collecting, analyzing, and showing off the data.  But it has also taught me a lot about the sensitivities of the system to things like shading, solar panel dirt, and alternative panel placements which I hope is useful to others as well.

Photovoltaic Part 2: Specifications and Installation

In an earlier post, I described the process leading up to the installation of our house’s photovoltaic system.  This post describes the system itself and its installation.

As described previously, the design of our system was done by the contractor, based on the electric bills we provided and input such as the fact that we wanted the system to be somewhat sensitive to aesthetics, not wanting to cover the front of our house (which faces south) with solar panels.  Instead, we opted to have half of the panels on the front, in an area that isn’t as visible from the street, and half on the back, facing west.

South-facing array (front)

West-facing array (rear)

The system consists of:

• 24 each Sanyo HIP-200BA3 panels (12 panels each facing south and west)
• 2 each Xantrex 3.8 GT series inverters (1 each for the south and west arrays)
• 1 lot mounting hardware, AC disconnect, etc.

The Sanyo HIP-200BA3 panels are rated at 200 W (180 W guaranteed) and are about 52 x 35 inches in size (132 x 89 cm), with a total area of 12.7 square feet per panel.  One of the things I like about these panels compared with others I have seen is that the frame is anodized black, which is much less conspicuous than ones that are their natural aluminum color.  The total area covered by the panels is about 305 square feet.

Xantrex inverters and line connection

The Xantrex inverters operate independently in order to better optimize their efficiency for the different time of day that the two arrays peak.  The use of two inverters did increase the cost of the system somewhat, but provided good efficiency while maintaining the aesthetics of the house.  Each inverter has a serial output that can be used to monitor the inverter output from a remote PC using software available from Xantrex.  More on that in a future posting describing the results from the system.  The inverters also have a display on the front of each unit and a very unusual but practical user interface:  knock on the front of the inverter to change the display.

The system cost (in mid 2006) was about $36,000 for the hardware and $5,500 for installation.  We got substantial incentives and rebates at the time, but these vary with time and locality.

The installation of our system took place in Fall, 2006, with the installation being performed by Renewable Alternative Technologies of Gilroy, California.  They did an excellent job; with all of the support posts I was concerned about possible roof leaks, but there haven’t been any at all.  You can see some more pictures of our system on their website under “Los Altos – 4.2 KW Grid-tied System.”