![[PukiWiki]](image/wadanyan.png "[PukiWiki]")
[[伊原]] **The 3rd International Workshop on Knowledge Collaboration in Software Development 2009 [#hf995dab] KCSD 2009~ URL:http://se.naist.jp/kcsd2009/ :title| Differences of Time between Modification and Re-modification: An Analysis of a Bug Tracking System :paper|&attachref(KCSD2009_v6.pdf);~ &attachref(KCSD2009_v6.tex);~ :presentation|pptx：&attachref();~ ppt：&attachref();~ *** Reviews [#c0d97db9] ### Review 1 ### EVALUATION (points in favour and against, and comments for improvement): This paper presents an analysis of the amount of time required to resolve a bug that was opened for the first time compared to a bug that has been reopened. The paper finds that bugs that are reopened are more quickly closed than those that are opened for the first time. While this result may not be terribly surprising, it is nice to have some empirical data supporting the finding. The paper has two main shortcomings: 1) The paper is difficult to read and contradicts itself (I believe for language reasons) several times. For instance: a) The abstract says: "time to resolve a firstly reported bug was shorter" b) The intro says: "time required for re-modifying a bug would be shorter" c) 5.2 says: "time to modify firstly-modified bugs [ノ] is longer" d) Table 1 indicates that the time to modify a new bug is longer. While items a and c are consistent (as are b and d), they directly contradict one another. This could be resolved by changing the language, for example: The time required to close a new bug was found to be longer than to close a bug that had been reopened. Using 'new bug' and 'reopened bug' would be easier to follow than 'modifications' or 'firstly reported bug' and 're-modification'. The paper would benefit immensely from being edited by a native english speaker as several tense changes and incomplete sentences impede understanding of the core concepts of the paper. 2) The results require much greater discussion. For instance, Table 2 shows that the results are statistically significant, and while I believe this, I do not think the table (or analysis) is necessary. From Table 1 it is clear enough that there is a large difference. More interesting than a statistical analysis of the time difference would be a real discussion about why this time difference matters, how it can be increased or decreased, or how projects can use this information to adapt their development practices. Certainly we can imagine several qualitative reasons why the results turned out this way, just as we could hypothesize their foil. A description, supported by the analyzed data, could add important data in support of this finding. For example, I would expect new bugs to take longer to close because they would tend to languish initially while developers understood the problem (as mentioned in the paper), or simply because the developer never got to them for a longer period of time. I could also imagine a new bug being addressed more slowly because a discussion needs to happen about whether the bug is even a bug, if it is a dupe, and if it should be fixed. These support the findings of the paper. At the same time, I could imagine that a bug could take longer to close subsequent times because it was clearly a difficult problem as the bug needed to be reopened: in these cases the bug would need to be considered and tested at greater length, or the bug was reopened because once it was applied it caused greater harm to the system. While these ideas are all simply hypotheses, the authors have the data to give a sense of _why_ new bugs took longer to close than reopened bugs. I STRONGLY believe that this 'why' is far more important than the simple fact that reopened bugs are closed more quickly. Without this 'why' the findings of the paper are not actionable to projects who want to learn from this finding and improve their own development practices. Other points: - The related work section is very brief and should more comprehensively address the relationship of this work to the 5 cited pieces of work. - The related work mentions 2 case studies (apache and mozilla), but apache is never mentioned again. A second case study would greatly expand the external validity of this study. - There is no discussion of the experimental validity or other limitations of the case study. - Extraneous information is included several times in the paper (e.g., the list of bug tracking systems, or the description of why Firefox is popular); these serve only to fill space and to detract from the primary message of the paper. - Fig 1: what is tossed? Do you mean reassigned? - In the paper the title is mentioned several times (e.g., abstract: In this paper, ノ, or the conclusion: In this paper, ノ. In both cases this conveys no additional information, in addition to being a sentence fragment. - The page headers (pg. 2+) are incorrect. - Section 5 should better describe why these bugs were selected as I have to assume there have been more than 10,917 bugs closed for firefox between v1 and v3. ### Review 2 ### EVALUATION (points in favor and against, and comments for improvement): The paper compared the time difference between first-time reported bugs and re-opened bugs in the mozilla firefox project. re-opened bugs tend to take shorter time. The Discussions section made the wrong claim. The data and Section 5 show that re-opened bugs take shorter time, but the Dicussions section claims (p.4) "As a result of this study, we observed that time to modify firstly reported bugs was shorter than time to re-modify a bug in spite of the number of assigned developers." It then continues to claim that "In this paper, we did not analyze time to resolve bugs, considering every developer’s skills and priority and/or severity of bugs." Well, I think the whole paper is about the time to resolve bugs. If not, what are those times? Please clarify those points. ---------------END-------------------------------------------
