伊原/KCSD2009 の編集

[[伊原]]
**The 3rd International Workshop on Knowledge Collaboration in Software Development 2009 [#hf995dab]
KCSD 2009~
URL:http://se.naist.jp/kcsd2009/

:title| Differences of Time between Modification and Re-modification: An Analysis of a Bug Tracking System

:paper|&attachref(KCSD2009_v6.pdf);~
&attachref(KCSD2009_v6.tex);~

:presentation|pptx：&attachref();~
ppt：&attachref();~

*** Reviews [#c0d97db9]
### Review 1 ###
EVALUATION (points in favour and against, and comments for improvement):

This paper presents an analysis of the amount of time required to resolve a bug
that was opened for the first time compared to a bug that has been reopened. The
paper finds that bugs that are reopened are more quickly closed than those that
are opened for the first time.

While this result may not be terribly surprising, it is nice to have some
empirical data supporting the finding. The paper has two main shortcomings:

1) The paper is difficult to read and contradicts itself (I believe for language
reasons) several times.

For instance:

a) The abstract says: "time to resolve a firstly reported bug was shorter"
b) The intro says: "time required for re-modifying a bug would be shorter"
c) 5.2 says: "time to modify firstly-modified bugs [ノ] is longer"
d) Table 1 indicates that the time to modify a new bug is longer.

While items a and c are consistent (as are b and d), they directly contradict
one another. This could be resolved by changing the language, for example: The
time required to close a new bug was found to be longer than to close a bug that
had been reopened. Using 'new bug' and 'reopened bug' would be easier to follow
than 'modifications' or 'firstly reported bug' and 're-modification'.

The paper would benefit immensely from being edited by a native english speaker
as several tense changes and incomplete sentences impede understanding of the
core concepts of the paper.

2) The results require much greater discussion. For instance, Table 2 shows that
the results are statistically significant, and while I believe this, I do not
think the table (or analysis) is necessary. From Table 1 it is clear enough that
there is a large difference. More interesting than a statistical analysis of the
time difference would be a real discussion about why this time difference
matters, how it can be increased or decreased, or how projects can use this
information to adapt their development practices.

Certainly we can imagine several qualitative reasons why the results turned out
this way, just as we could hypothesize their foil. A description, supported by
the analyzed data, could add important data in support of this finding. For
example, I would expect new bugs to take longer to close because they would tend
to languish initially while developers understood the problem (as mentioned in
the paper), or simply because the developer never got to them for a longer
period of time. I could also imagine a new bug being addressed more slowly
because a discussion needs to happen about whether the bug is even a bug, if it
is a dupe, and if it should be fixed. These support the findings of the paper.
At the same time, I could imagine that a bug could take longer to close
subsequent times because it was clearly a difficult problem as the bug needed to
be reopened: in these cases the bug would need to be considered and tested at
greater length, or the bug was reopened because once it was applied it caused
greater harm to the system. While these ideas are all simply hypotheses, the
authors have the data to give a sense of _why_ new bugs took longer to close
than reopened bugs. I STRONGLY believe that this 'why' is far more important
than the simple fact that reopened bugs are closed more quickly. Without this
'why' the findings of the paper are not actionable to projects who want to learn
from this finding and improve their own development practices.

Other points:

- The related work section is very brief and should more comprehensively address
the relationship of this work to the 5 cited pieces of work.

- The related work mentions 2 case studies (apache and mozilla), but apache is
never mentioned again. A second case study would greatly expand the external
validity of this study.

- There is no discussion of the experimental validity or other limitations of
the case study.

- Extraneous information is included several times in the paper (e.g., the list
of bug tracking systems, or the description of why Firefox is popular); these
serve only to fill space and to detract from the primary message of the paper.

- Fig 1: what is tossed? Do you mean reassigned?

- In the paper the title is mentioned several times (e.g., abstract: In this
paper, ノ, or the conclusion: In this paper, ノ. In both cases this conveys no
additional information, in addition to being a sentence fragment.

- The page headers (pg. 2+) are incorrect.

- Section 5 should better describe why these bugs were selected as I have to
assume there have been more than 10,917 bugs closed for firefox between v1 and
v3.

### Review 2 ###
EVALUATION (points in favor and against, and comments for improvement):

The paper compared the time difference between first-time reported bugs
and re-opened bugs in the mozilla firefox project. re-opened bugs tend
to take shorter time.

The Discussions section made the wrong claim. The data and Section 5 show
that re-opened bugs take shorter time, but the Dicussions section claims
(p.4)
"As a result of this study, we observed that time to modify firstly reported
bugs was shorter than time to re-modify a bug in spite of the number of assigned
developers."

It then continues to claim that
"In this paper, we did not analyze time to resolve bugs, considering every
developer’s skills and priority and/or severity of bugs."

Well, I think the whole paper is about the time to resolve bugs. If not, what
are those times?

Please clarify those points.

---------------END-------------------------------------------

タイムスタンプを変更しない