For the full study, we measured the agreement (a) for each combination of (round, group) separately (i.e., two raters per comment), and (b) across both groups (i.e., more than two raters per comment).
We select these four publicly available repair techniques that have been evaluated against the Defects4J benchmark and ignore others which either focus on another language [29, 40, 56] or are not publicly available [22, 24].
![[PukiWiki]](image/wadanyan.png "[PukiWiki]")
