Open Source Blog

A couple of quick points about this Harvard “Bill of Rights” study
May 23, 2007

The study, written by Harvard associate professor Alan MacCormack and based in part on a study funded by Microsoft, is something of a work of art.

In many ways it would be better to ignore it, but having taken a good look at it here are a couple of points worth making:

In any kind of research environment 34 respondents is not “more than sufficient” for making sweeping statements about the opinions of thousands of people.

Only 16 of those respondents were actually contributing to projects that use the GPL/LGPL

Based on their answers to questions about their motivations, respondents were placed into three groups, which were based on a bastardization of the four clusters identified by Lakhani and Wolf in their 2005 study Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects.

There is a reason why Lakhani and Wolf used four clusters: “The four-cluster solution provided the best balance of cluster size, motivational aggregation, stability and consistency”. As soon as you mess with the clusters, you mess with that balance, stability and consistency.

Changing Lakhani’s four group cluster into a three group cluster might have suited the needs of MacCormack, but it destroyed the point of making any direct comparisons between the two studies.

MacCormack then compared “the majority” viewpoints of respondents in each group on a number of issues and found that only half of group 3 (that’s 3.5 people) agreed with the FSF’s approach to GPLv3, whereas all of groups 1 and 2 were against it.

“Based on Lakhani’s research, Group Three represents 19% of the community. Thus our results suggest the actions of the FSF may only be favoured by approximately 10% of the broader community” he concluded.

As an example of how pointless it is to extrapolate between the two studies once the clusters have been messed with, it is worth considering that in Lakhani’s original four group cluster 42% of those in cluster 3 agreed with the philosophy that “the code should be open”, which would have aligned them, in MacCormack’s three group cluster, with group 3 “the philosophers”.

Instead it is assumed that the percentage of respondents in Lakhani’s group 3 all agree with those of MacCormack’s group 2 “the intellectuals”.

It might not seem like that much of an anomaly – it’s only 42% of 29% after all, but given that Lakhani and Wolf managed to find 684 people to interview, it represents 83.3 people – more than twice as many as even answered MacCormack’s study.