![[PukiWiki]](image/wadanyan.png "[PukiWiki]")
[[reviews]] *IWESEP2016 REVIEWS PAPER 19[#wba44397] **TITLE: Clustering OSS License Statements Toward Automatic Generation of License Rules [#mc1a75c5] **AUTHORS: Yunosuke Higashi, Yuki Manabe and Masao Ohira [#a61a0e6f] ***----------- REVIEW1 ----------- [#db2be394] This paper presented an approach to automatically cluster files with license statements that are unable to be detected by existing rule-based license identification tools. The authors used grep and hierarchical clustering to group these files. The approach was applied on 2838 unknown license files of Debian version 7.8.0. Points in favour This paper tackles an interesting problem which is both relevant in research and practice. The research problem is well motivated. The paper is well-structured and the writing is clear. Points against In the grep step, to identify the BSD licenses, the authors used the discriminant conditions and the flow conditions shown in Figure 2. There are four clauses in that figure. It is not clear to me what do those clauses mean (e.g., what is "neither the name"? what is "all advertising materials"?). Please briefly describe them in the paper.    -BSD3の条件"neither the name"と"all advertising materials"の説明 In the discussion, the authors explained the rationale behind the not "so high" precision (0.89). But they did not explain the reasons behind the even lower recall (0.64).    -RECALLの言及がない The authors calculated the precision/recall/F-value for the grep step. For the clustering step, the authors used another set of metrics (ratio of SLC, ratio of SF, etc.). However, these metrics are not standard performance evaluation metrics. It would be useful to show the actual precision/recall/F-value just as the grep step. -SLCやSFのメトリクスはスタンダードじゃない．grepステップのように，precision/recallで示すべき ***----------- REVIEW2 ----------- [#sf8e4d16] This paper proposes a method to more efficiently check the licenses in software projects by building on a regular-expression-based license identification tool, and cluster together similar unknown licenses for human perusal. Overall, I think that the method seems to be practically useful and would make a good contribution to the workshop. One major issue with the paper, however, is the clarity of the technical description. Specifically, in Section III.c. the alpha and beta conditions were a bit difficult to understand. For condition alpha it is easy enough to understand by looking at the equation, but what is the particular motivation behind the heuristic threshold in Equation (2). Why does it make sense? The beta condition was quite difficult to understand even after several readings. This seems to be a little bit like the "exchange algorithm" in the following paper, is this correct? Algorithms for bigram and trigram word clustering Sven Martin, Joerg Liermann, Hermann Ney Speech communication -条件αとbetaの説明がわかりにくい．条件betaは，以下の論文と少し似ているようにも見える There were quite a few typos (e.g. "sometimes sometimes", "hight", "pourpose") that could be caught either with a spell checker or a careful read of the paper. I'd suggest using a spell checker and reading through the paper again carefully. It might also be a good idea to use a grammar checker such as the one built in to Microsoft Word. Also, it is not "a data", but "a datum" or "a piece of data". Also, I believe it's usually called F-measure, not F-value? -タイポグラフィー It would also be interesting to use a more supervised approach to machine learning to identify code with different licenses. For example, every time a user annotates a particular cluster of licenses, this annotated data could be used to train and improve a machine learning model. -教師あり学習アプローチを使ってみれば，より面白いものになるだろう ***----------- REVIEW3(dmg?) ----------- [#i099f376] The paper describes an approach to automatically infer license rules from license descriptions marked as UNKNOWN by the Ninka tool. These new license rules could then be used to improve the effectiveness of the Ninka tool (that require a predefined set of license rules to be able to identify source code licenses).  To date, Ninka can identify 112 licenses. The proposed approach consists in three steps. The Ninka tool is applied on source code files and files with license statements reported as UNKNOWN by Ninka are clustered. Next, each cluster is analysed to extract patterns in the license descriptions. Finally the obtained patterns are used to generate license rules. However, this paper elaborates only on the first step of the proposed approach. Overall, the paper addresses an interesting problem. The proposed approach could improve the precision/recall of rule based tools like Ninka and FOSSology.  However, certain aspects of the work needs to be clarified. For example the authors states that « The GPL/BSD-related licenses could be distinguishable not by using rigid combinations of regular expressions but by finding license names and versions including “or later” options in license statements. » This statement is not entirely true since a licensor could add some conditions and―or exceptions to an existing license. For example, in the case of BSD3, by adding four more sentences to it we get Apache Licences v1.1. How does this affect the accuracy of the proposed approach? -Ninka，Fossologyのどの部分をサポートするのかを明確にする必要ある -"GPL/BSDは，正規表現を使うことで，バージョンと，オプションを分類することができる．"この記述のが完全に正しいというわけではない． -開発者が追加した条項や，リンク例外，Apache v1.1などを分類するべき．これらは，研究における精度にどのような影響を与えるか The use of clustering to infer license rules is interesting, however, unless the number of existing licenses is known beforehand, it is challenging to obtain a good clustering Contrary to what is claimed in the paper, Ninka performs normalization to resolve the problem of misspelling and notation variances.  It would have been interesting to see a discussion about this in the paper -論文に描かれていることに反して，Ninkaは，スペルミスや，バージョンのバリエーションを標準化する．これについて議論しているのを見るのは興味ぶかいとされるだろう? The paper should be proof-read to correct the multiple typos. Some Typos: clusters which are are suitable -> clusters which are suitable a large number of licenses sometimes sometimes must be checked to be reused -> a large number of licenses sometimes must be checked to be reused For the GLP license family -> For the GPL license family Pourpose-> Purpose
