From: Dan DeMaggio <dmag [at] umich [dot] edu>
To: jhowison [at] syr [dot] edu, crowston [at] syr [dot] edu
CC: Edward J. Valauskas, Chief Editor, First Monday
Subject: Comments on Social Structure
Date: 31 Mar 2005 14:39:54I have to comment on your article "The Social Structure of Free and Open Source Software Development," [First Monday, volume 10, number 2 (February 2005)] at http://firstmonday.org/issues/issue10_2/crowston/index.html
In your introduction you say
"We suggest, therefore, that it is wrong to assume that FLOSS projects are distinguished by a particular social structure merely because they are FLOSS."In your paper, you equate "Social Structure" with "interactions."
I must say I’ve never heard anyone equate FLOSS with interactions before.
Let’s look at the definitions of Cathedral and Bazaar:
http://www.google.com/search?q=define:bazaar&btnG=Google+Search http://www.google.com/search?q=define:cathedral&btnG=Google+SearchCathedral: "The principal church of a province or diocese, where the throne of the bishop is placed."
Bazaar: "A large marketplace with many streets and stalls."
The cathedral definately implies a social structure, the bazaar implies the absence of social structure. Neither implies anything about interactions. In a cathedral, the congregation can talk among themselves as easily as in a bazaar.
In fact, ESR’s CatB even says "the halo developers work [.. on ..] parallel subtasks and interact with each other very little."
http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s05.htmlNext, you say
"Our findings suggest that FLOSS projects might have to work hard to achieve the expected development advantages which have been assumed to flow from ‘going open.’"Huh? This does not follow from your previous statement. You are implying that the FLOSS development advantages come from users/developers talking among themselves. They do not, and I do not see any references to support your position.
The FLOSS advantages come from the license (just having an open license will attract developers), the level playing field (everyone has equal rights), the lack of managers or deadlines (biggest impediments to code quality), and the "release early and often" philosophy (which commercial users won’t tolerate.)
None of your references say "code quality is related to the number or type of interactions between users." In fact, the Cox data suggests that while user to developer communication was important, the (unaffected) user to user communication was not important.
[In your discussion about kill–files, you state "The flow would have been one-way towards the centre." Really, it would be one–way OUT from the center, since no information can get in.]
You write:
"[ESR] praises social structures which appear to be centralized [...] this right of "ownership" is informal but strongly normative and often persists in the same individual that founded the project — a feature found, for example, in the Linux project."You suggest that Linus ‘owns’ the Linux kernel, and therefore it’s centralized. Nothing could be further from the truth. Hardly any users run Linus’s "official" kernel. Each Linux vendor maintains their own kernel with custom patches. Because most people use a linux distribution, very few people run Linus’s kernel unmoddified. Patches often go thru various people’s kernel trees (–dj, –mm, –ac, –rms, etc.) before getting to Linus’s tree. ESR says "Rogue patches are unusual, and generally not trusted," but he has a big footnote explaining that friendly forks are ok.
(True Story: I maintain my own kernel for some hardware at work that we sell. The code is not good enough to be integrated into the mainline kernel, so it will forever be a fork.)
I think the truth is: FLOSS is agnostic about centralized vs decentralized. Think about a group of programmers trying to architect their code. In one scenario, a strong leader listens to a few vocal members and chooses the ‘right’ direction. If people disagree, they leave. In another scenario, the programmers all just talk for a while, and the ‘right’ direction emerges. People who disagree leave. You would give these cases different labels. I see them as the same process, just that the first one is a more efficient version of the second. For technical matters, the ‘right’ decision is obvious once it gets out, the question is how to get it out.
You write;
"[...] nonetheless [Linus’ Law] also appears to require that the communications structure of bug–fixing will include many participants, each expanding on reports, providing alternative conceptualizations of problems or attempting solutions."Huh? Linus’s Law can be rephrased as "Debugging is parallelizable." Parallelizable means that multiple people can act independently (without communications overhead, or waiting for each other, etc.). Only the people who have found a problem (or solution to a problem) will speak up. The people who looked into a problem/program and found nothing will be silent.
For finding bugs, Linus’s law does not even require that the ‘users’ be able to talk amongst themselves. (Each one can download the program and try it, and report bugs back to the maintainers.) This directly contradicts your assertion.
For the case where one user posts a bug and another fixes it, you will record one interaction (the reply to the bug with a patch) even if 100’s of people looked for the bug but had no news to report. It is quite possible that the patch was not directly associated with the original bug report, in which case your data would record zero interactions.
That brings up a related point: Your paper does not explain why you think looking at the SF bug database is a valid research method. I can think of dozens of problems with your data. For example:
— Not all projects actually use the SF bug reporting database. Many times users post bugs before realizing that the maintainers don’t actually use the bug DB. If the maintainer merely closes the bug with "report it on our mailing list," your data will look centralized.
— Many projects will have a few people in charge of the bug database (responding, closing, etc). This will skew your results to look centralized when it’s not.
— Especially in large projects, developers tend to have a "core" mailing list or IRC channel where they can discuss things without the riff–raff.
(True story: I’ve talked chatted with a gaim developer using gaim. During our conversation, I mentioned a bug to him. But I’ve never posted to their mailing list or bug database.)
— Even worse, some developers may be friends (and discuss things on the phone) or work for the same company (and discuss things in person/meetings). On the public mailing list or bug list, it may appear that they *never* talk to each other because they speak with one voice.
— On smaller projects, users may e–mail bug reports and patches directly to the author(s). These private e–mails may not be captured on any public list.
— You haven’t defined interaction. If someone posts a bug and 100 people read it, isn’t that 100 interactions? Some of those people actually got information from that bug post, even if they didn’t respond. What if some fraction of them searched for the bug and didn’t find it? Is that an interaction or not?
— The degree to which the mailing list duplicates/superceeds the bug list will vary by project. I would argue that you are missing far more data than you realize by not using mailing lists. (But watch out, many projects don’t use SF’s mailing list features. The bigger and/or more well organized projects tend to have their own lists.) Also, the same problems as above will affect the mailing list.
I think when people talk about "Centralized vs Decentralized," they are really talking about "Rigid, top–down centralization" compared to "Flexible, bottom–up, temporary centralization just for efficiency." It's like the difference between the Army building a house and a town barn–raising. From the outside, they look similar: One person barking out orders to several other people.
Calling the barn–raising ‘centralized’ is misleading at best. The "guy in charge" is not in charge, and can be replaced at any time. (It’s even more critical in software, because you can fork software, but you can’t fork a house.) The people are not following him so much as they are following a common goal. The communication is very efficient because they have a shared vision. (Unlike typical management which is always contradicting themselves or holding back information, which prevents a shared understanding.)
-=Dan=-
From: James Howison <jhowison [at] syr [dot] edu>
To: Dan DeMaggio <dmag [at] umich [dot] edu>, Edward J.Valauskas <ejv [at] uic [dot] edu>
CC: Kevin Crowston <crowston [at] syr [dot] edu>
Subject: Comments on Social Structure
Date: 9 Apr 2005 01:14:14Thanks for your comments on our recent paper and for sharing your experiences with FLOSS projects.
Let us first say that academic research is a little like open source: the more contributions, the better. It is a little like open source software also in that we make our data and analysis scripts available so others can correct or build on them. A final similarity: while conceptual discussion and evidence from personal experience is great, in the end, what counts is to "show us the data" from a broad selection of projects.
That said, onto your comments.
"In your introduction you say ‘We suggest, therefore, that it is wrong to assume that FLOSS projects are distinguished by a particular social structure merely because they are FLOSS.’ In your paper, you equate ‘Social Structure’ with ‘interactions’. I must say I’ve never heard anyone equate FLOSS with interactions before."We equate FLOSS development team structure (not FLOSS) with interactions, as our paper is a study of the structure of FLOSS development teams, including both formal and informal structures. Patterns of communications are important evidence for informal structures and there is a long research tradition of their use starting in the 1960s and continuing today.
"Let’s look at the definitions of Cathedral and Bazzar: http://www.google.com/search?q=define:bazaar&btnG=Google+Search
http://www.google.com/search?q=define:cathedral&btnG=Google+SearchCathedral: ‘The principal church of a province or diocese, where the throne of the bishop is placed.’
Bazzar: ‘A large marketplace with many streets and stalls.’
The cathedral definately implies a social structure, the bazzar implies the absence of social structure. Neither implies anything about interactions."
The literature, both research and practitioner, argues that some of the benefits of FLOSS development come from the way they are social organized. To that end we point out the organizational content of metaphors like cathedrals, bazaars and town councils, as used by ESR and Alan Cox and discuss the academic research of Cox, 1998; Gacek, et al., 2001; Moon and Sproull, 2000; Mockus, et al., 2002 and Krishnamurthy, 2002 all of whom, it seems to us, argue that there is something interesting about the way FLOSS is organized that relates to the benefits of FLOSS development.
"In a cathedral, the congregation can talk among themselves as easily as in a bazzar."And yet cathedrals produce and enforce silence amongst their visitors, do they not? More importantly workers building a cathedral (since that is what the metaphor is really about) do not decide the goals of their tasks through discussion amongst themselves: they come from on high, from the architect via various levels of hierarchy. On the other hand, there is no architect in a Bazaar; instead, the outcome of the day’s activities comes from interactions amongst decentralized participants.
"In fact, ESR’s CatB even says "the halo developers work [.. on ..] parallel subtasks and interact with each other very little."
http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s05.html"ESR’s excellent piece, as we discuss in the paper, is somewhat inconsistent when read to understand what it proposes would be a typical structure of a FLOSS project. The section you draw attention to would have made an interesting addition to a long paper. It doesn’t change our point that the various sources lead to different conclusions about the expected structure.
"Next, you say "Our findings suggest that FLOSS projects might have to work hard to achieve the expected development advantages which have been assumed to flow from ‘going open.’"Huh? This does not follow from your previous statement. You are implying that the FLOSS development advantages come from users/developers talking among themselves. They do not, and I do not see any references to support your position."
The first half of the paper is an argument addressing the practitioner and academic literature highlighting the way that suggested benefits of FLOSS development is linked to social structures, sometimes centralized, sometimes decentralized.
It is great to hear from the personal experience of developers who have different opinions and we found the next section of your letter intriguing.
"The FLOSS advantages come from the license (just having an open license will attract developers),"Unfortunately, that is just not the case. Of the 98,000 or so SourceForge projects over the past five years up to February this year, nearly all of which have OSI approved licenses, only 67 percent have ever had more than one developer registered at any time during their lifetime. (Crowston, Howison and Annabi, under review, working paper at http://floss.syr.edu/StudyP/Crowston2005APortfolio.pdf; the data are available via http://ossmole.sf.net). Our data are clearly saying that open licenses by themselves are not enough to attract developers.
"the level playing field (everyone has equal rights),"This is only true for a restricted definition of rights, like the (important) right to download, compile and run the software. It is simply not true that everyone has equal rights more broadly: project originators and leaders have (and should have) more rights over their projects than other contributors. This inequality is if anything more pronounced in more successful projects, such as Linux or Apache. Furthermore, a project in which all communication flows through one individual, who answers and thus shapes the groups understanding of each discussion, isn't likely to produce a project with equal influences. We don’t, however, have the data yet to examine the link between communications and influence in depth. Our research group has published a theory of emergent leadership in FLOSS projects which you might find interesting: http://floss.syr.edu/publications/leadership2005.pdf
"the lack of managers or deadlines (biggest impediments to code quality), and the ‘release early and often’ philosophy (which commercial users won’t tolerate.)"We’d agree that a lack of deadlines is definitely related to the success of open source. For one thing they facilitate the involvement of volunteers who are somewhat unreliable (which is understood and accepted in the projects we know well). Projects like Eclipse and the Mozilla Foundation, however have plenty of managers involved (though their authority comes from persuasive ability rather than compulsion) and plenty of open source projects follow a slower release cycle than "early and often", Debian, for example. But competing anecdotes are not data — more work is needed to establish the prevalence of these practices and their relation to project effectiveness.
"None of your references say ‘code quality is related to the number or type of interactions between users.’ In fact, the Cox data suggests that while user to developer communication was important, the (unaffected) user to user communication was not important.
[In your discussion about kill–files, you state ‘The flow would have been one–way towards the centre.’ Really, it would be one–way OUT from the center, since no information can get in.]"
It depends a lot on from where you were measuring the list, from inside a developer’s inbox where you rightly point out that you wouldn’t see the messages at all, or from an archive on the mail server where you would see the messages. More interestingly, did the external participants know that they were being ignored? If so did they stop speaking? Just because they didn’t stop but weren’t read should one exclude their messages entirely? The situation would be interesting to study in detail. Perhaps, however, Alan Cox’s call has been heeded and kill files like that are no longer in use?
"You write:
"[ESR] praises social structures which appear to be centralized [...] this right of "ownership" is informal but strongly normative and often persists in the same individual that founded the project — a feature found, for example, in the Linux project."You suggest that Linus ‘owns’ the Linux kernel, and therefore it’s centralized. Nothing could be further from the truth. Hardly any users run Linus’s "official" kernel. Each Linux vendor maintains their own kernel with custom patches. Because most people use a linux distribution, very few people run Linus’s kernel unmoddified. Patches often go thru various people’s kernel trees (–dj, –mm, –ac, –rms, etc.) before getting to Linus’s tree. ESR says "Rogue patches are unusual, and generally not trusted," but he has a big footnote explaining that friendly forks are ok.
(True Story: I maintain my own kernel for some hardware at work that we sell. The code is not good enough to be integrated into the mainline kernel, so it will forever be a fork.)"
This was interesting, thanks for sharing your experiences. Nonetheless the virtual memory, SE linux episodes and the Bitkeeper decision (and Linus’ current search for a replacement http://lkml.org/lkml/2005/4/6/121) show both that many developers do care about getting things accepted into the mainstream kernel and that Linus is in full and firm control of that process and wider processes of the Linux project. It would be interesting to get data on how widespread the practice of parallel source code trees is outside the Linux project. Certainly the activities of package management in distributions like Debian and Gentoo, while not being forks per se, demonstrate the ability for others to utilize open code in ways not anticipated by the authors. Again, data rather than competing anecdotes are needed.
"I think the truth is: FLOSS is agnostic about centralized vs decentralized. Think about a group of programmers trying to architect their code. In one scenario, a strong leader listens to a few vocal members and chooses the ‘right’ direction. If people disagree, they leave. In another scenario, the programmers all just talk for a while, and the ‘right’ direction emerges. People who disagree leave. You would give these cases different labels. I see them as the same process, just that the first one is a more efficient version of the second. For technical matters, the ‘right’ decision is obvious once it gets out, the question is how to get it out."We agree that the task focus of FLOSS projects does play a major role in understanding how decisions are made. And from a task perspective perhaps the first (a benevolent dictatorship model) may be more effective. But there is nothing to say that it will emerge, nor that it can always sustain a community. As we point out in our conclusion, it would be great to conduct an evolutionary analysis to see how project teams change over time. Happily the data are out there.
"You write;
"[...] nonetheless [Linus’ Law] also appears to require that the communications structure of bug–fixing will include many participants, each expanding on reports, providing alternative conceptualizations of problems or attempting solutions."Huh? Linus’s Law can be rephrased as "Debugging is parallelizable." Parallelizable means that multiple people can act independently (without communications overhead, or waiting for each other, etc.). Only the people who have found a problem (or solution to a problem) will speak up. The people who looked into a problem/program and found nothing will be silent.
For finding bugs, Linus’s law does not even require that the ‘users’ be able to talk amongst themselves. (Each one can download the program and try it, and report bugs back to the maintainers.) This directly contradicts your assertion.
For the case where one user posts a bug and another fixes it, you will record one interaction (the reply to the bug with a patch) even if 100’s of people looked for the bug but had no news to report. It is quite possible that the patch was not directly associated with the original bug report, in which case your data would record zero interactions."
You raise two interesting points here. Let us address them one by one. Firstly we agree that there is much that occurs in FLOSS development that leaves no traces in the repositories and mailing lists. Not all CVS checkouts result in check–ins and much effort may have been expended in failed attempts to replicate or fix a bug. Perhaps it is even this ability to work silently with no fear of public failure that allows so many to ‘take a shot’ at code or bug fixes. We urge FLOSS researchers, therefore, to go beyond the publicly available data and to speak to developers. Time diary studies might be appropriate. As tempting as the data available is, comprehensive research is going to need to go beyond it.
Secondly, we relied on ESR’s characterization of Linus’s law for our analysis in this part of the paper: "Somebody finds the problem ... and somebody else understands it." and another fixes it. It is hard to see that this collaboration could operate completely absent communications between the participants in bug fixing, bug reports, bug characterizers and bug fixers, be they users or developers. Certainly the thread lengths we found demonstrate that there is plenty of discussion in bug repositories.
"That brings up a related point: Your paper does not explain why you think looking at the SF bug database is a valid research method. I can think of dozens of problems with your data. For example:
— Not all projects actually use the SF bug reporting database. Many times users post bugs before realizing that the maintainers don’t actually use the bug DB. If the maintainer merely closes the bug with "report it on our mailing list," your data will look centralized.
— Many projects will have a few people in charge of the bug database (responding, closing, etc). This will skew your results to look centralized when it’s not.
— Especially in large projects, developers tend to have a "core" mailing list or IRC channel where they can discuss things without the riff–raff.
(True story: I’ve talked chatted with a gaim developer using gaim. During our conversation, I mentioned a bug to him. But I’ve never posted to their mailing list or bug database.)
— Even worse, some developers may be friends (and discuss things on the phone) or work for the same company (and discuss things in person/meetings). On the public mailing list or bug list, it may appear that they *never* talk to each other because they speak with one voice.
— On smaller projects, users may e–mail bug reports and patches directly to the author(s). These private e–mails may not be captured on any public list.
— You haven’t defined interaction. If someone posts a bug and 100 people read it, isn’t that 100 interactions? Some of those people actually got information from that bug post, even if they didn’t respond. What if some fraction of them searched for the bug and didn’t find it? Is that an interaction or not?
— The degree to which the mailing list duplicates/superceeds the bug list will vary by project. I would argue that you are missing far more data than you realize by not using mailing lists. (But watch out, many projects don’t use SF’s mailing list features. The bigger and/or more well organized projects tend to have their own lists.) Also, the same problems as above will affect the mailing list."
You raise a number of possible problems with the data we used and the way we operationalized our theory. The inherent tradeoff in analyzing a large amount of data is increased uncertainty about the details of each piece. We discuss most of these problems and our decisions in the paper. To recap:
Bug–fixing is only one interesting activity of a team but it is an important piece of the FLOSS development model (it is the point of Linus’s Law after all) that provides insight into the team structure, which is the object of our study. Other researchers may believe that other data are more useful and their analyses will provide an interesting counter–point to ours. It would be great to analyze mailing lists, and we say so in the paper and are currently doing so in our research. In fact, we initially intended to include a SNA analysis of the mailing lists in the paper but found that only 52 projects in our sample had a developer mailing list on SourceForge. Since the SNA analysis of those lists showed no significant differences in the distribution of centralities from those in the bug forums, rather than halve the number of projects considered, we choose only to report bug repository data. We do mention this in the paper. But clearly, combining data from multiple sources will provide a richer picture of the interactions, and should be done. The data are available should you wish to do so.
It is clear that not all teams use the repository nor do they use it similarly. These are acknowledged limitations of the paper. We did check that the projects in our sample were actively using the bug tracker by requiring 100 bugs in the tracker and manually by examining a sample of home pages for bug-posting instructions. Other analysis (not reported in the paper but available on http://floss.syr.edu) shows that many of these bugs are closed over a range of time periods, indicating that the teams are using the tracker in the expected way. We’ve also analyzed bug reports from the Savannah and Apache bug trackers and gotten the same pattern of results. Still we acknowledge that the types of things you point out may go on and would reduce the accuracy of the analysis. If you believe that they affect our results, the data are available for you to show where we went wrong.
We present our definition of interactions in the methods section of the paper. Others may choose to use different definitions of interactions and again, the data are available, if anyone wishes to do so. If analysis with those definitions leads to wildly different results, then there’s something to talk about; without the analysis, it’s just speculation.
"I think when people talk about "Centralized vs Decentralized," they are really talking about "Rigid, top–down centralization" compared to "Flexible, bottom–up, temporary centralization just for efficiency." It's like the difference between the Army building a house and a town barn–raising. From the outside, they look similar: One person barking out orders to several other people.
Calling the barn–raising ‘centralized’ is misleading at best. The "guy in charge" is not in charge, and can be replaced at any time. (It’s even more critical in software, because you can fork software, but you can’t fork a house.) The people are not following him so much as they are following a common goal. The communication is very efficient because they have a shared vision. (Unlike typical management which is always contradicting themselves or holding back information, which prevents a shared understanding.)"
Again, we reiterate that we don’t pretend that communications structure is control or decision making structure, although they are likely related in some way. On the other hand, it seems implausible that the person barking out the orders at a barn raising is chosen at random (and indeed, published reports of barn raisings note the existence of a power structure with one person in charge, experienced individuals serving as crew chiefs, new members taking minor roles or even just watching to learn how it works — sounds a lot like a FLOSS project, in fact). Getting a firm grip on leadership and influence in FLOSS projects, and its relationship to team growth and project success will be a longer process than this one study, but one we, and other researchers, are actively engaged in. Recently Gregorio Robles has put together http://libresoft.urjc.es/planet/, meta blog of research in the field and Karim Lakhani hosts the excellent http://opensource.mit.edu research repository. Hopefully the results of this research will take us beyond the current anecdotal discussions of FLOSS which are, it seems to us, based heavily on the respective author’s experience of a few projects.
We’ve encountered many open source communities at conference like ApacheCon, FOSDEM, CodeCon and OSCon, who are trying to build active projects and have found that simply choosing an OSI license and ‘hanging out the shingle’ are not enough. They also did not find many ‘how to build a successful and active community’ resources available in the FLOSS community (we’d love to hear of any that First Monday readers find useful). We hope that the deeper understanding of the range of FLOSS social organization that further research should produce will thus be of benefit to the FLOSS community at large.
James Howison and Kevin Crowston
Syracuse University
School of Information Studies
348 Hinds Hall
Syracuse, NY 13244–4100 USA
This work is licensed under a
Creative Commons License.
Letter to the Editor
First Monday, volume 10, number 4 (April 2005),
URL: http://firstmonday.org/issues/issue10_4/letters/index.html