A Comparison Study on the coordination patterns in oss community(仮)
Shinsuke Matsumoto, Yasutaka Kamei, Masao Ohira and Ken-ichi Matsumoto
-------------------- review 1 -------------------- OVERALL RATING: 2 (accept) REVIEWER'S CONFIDENCE: 4 (expert) Is this paper a good candidate for a major presentation with a discussant?: 2 (maybe) ----------------------- REVIEW -------------------- Brief summary: The authors argue, based a case study of Apache & Netscape, that highly motivated people who bridge user and developer communities are the most effective way to ensure success of open source projects. The study considered both the static social structure of open source projects, and the dynamic change in the structure. While it is well known that highly motivated developers are the corner stones of the community, the "bridging behaviour" of key people, between developers and users, is an interesting phenomenon. Evaluation: THe paper offers an interesting case study. The results are somewhat speculative, but they are potentially interesting and relevant, and suggest further topics for study and discussion. There are some problems with the work. THe authors don't seem to have done any aliasing resolution. People use multiple email aliases; without resolving these, the data and social networks are suspect. Definitions are scattered too much in the paper. There should be a separate section that defines all the metrics. IN fact, I could not find <k> defined anywhere. i assumed this was degree? I suggest using detailed captions. Figures, tables etc are much easier to follow if they are fully self-describing. Some other comments: Figure 7 is very hard to read. Please add a detailed caption. Page 1, right column, end of second para, last sentence: "developers who communicated with more developers contributed more to source code" (reword) section 6.1, para 2 "Highly motivated developers are in constant communication with each other"
-------------------- review 2 -------------------- OVERALL RATING: 2 (accept) REVIEWER'S CONFIDENCE: 4 (expert) Is this paper a good candidate for a major presentation with a discussant?: 2 (maybe) ----------------------- REVIEW -------------------- Brief summary: The manuscript presents a study of Apache and Netscape email mail-group and newsgroup communication networks. Email addresses that contribute to both user and developer groups were a particular target. The links between email addresses are defined as an immediate post-response relationship in the discussion threads. Average path length and clustering coefficients are presented for entire user/developer network, for the user network, and for the developer network. Only developer networks are substantially different (presumably at least partially because Netscape developers are collocated and have alternatives to newsgroup communication). The top five emails for Apache and Netscape that participate in developer and user lists are tracked over time finding that the Netscape email addresses had uneven participation over the years, presumably reflecting down-sizing, churn, and eventual demise of the project. Evaluation: Clearly, understanding communication networks in general, and the roles of developer/user-group-spanning individuals is of interest. Furthermore, work demonstrates that some of the dramatic differences between the two projects are also somewhat reflected in some of the network measures. There are, however, a number of questions about what exactly the findings may mean. First question that jumps out is the differences between mail-groups (where each message is mailed to each subscriber) and newsgroups (where anyone interested can read some of the messages at some point in time). This push versus pull is likely to affect the reader/poster behavior and may explain at least some of the observed differences. The presented explanation that it is irrelevant is not believable. The second big question relates to the definition of the target group (posters to user and developer lists). Are they indeed moderating feature request transition from users to developers as hypothesized in the paper or are they serving some other roles? There appears to be no attempt to validate that assumption. What are the minimal requirements (say number of posts or responses in either or both lists) to qualify them as moderators of user requirements? The third big issue relates to exact meaning of the presence of post-response pair in the newsgroups and in mailing lists. Does it really establish a social connection (can the prolific responders recall even a tenth of the email addresses they have helped) or is it reflecting something else, and if so what? Without answers to these questions it is hard to judge what the measures and charts mean, so the remaining comments are minor. The absence of clear differences in the summary measures for the entire and user groups is quite disconcerting. The projects are, after all, drastically different and it is surprising that several network measures have quite similar values. Could it be that other measures would capture the difference, or was it an unlikely case that the particular aspects that these measures captured were similar? If so why? The difference in the graph layout charts, while pronounced, are not likely to be reproducible (both in terms of layout algorithm and, most importantly, in terms of interpretation). It is not clear if the results of the dynamic analysis reflect noise or signal. In any case it is not clear what the findings are, and even less clear what they may mean. Text, especially sect. 4.2.2 needs numerous corrections and clarifications.
リポジトリ: http://tun.naist.jp/svn/Empirical/shinsuke-m/STC_2008/