Our empirical evaluation suggests that our
approach can learn useful API usage models which can provide
recommendations with higher levels of accuracy than the baseline
n-gram model.
In this section we present 8 findings of our empirical study, grouped
by their relevance to research questions raised in Section 1.
In this section, we answer the first research question by comparing
SketchFix’s repair efficacy with other repair techniques—
Astor [35], Nopol [7], ACS [58] and HDRepair [25].
The main intent of the study was to evaluate the effectiveness of the layout distortion metric, L (Section 3.3), in minimizing layout disruptions and producing attractive pages.
Despite being a fairly expected phenomenon, it informs us that software obfuscation at
this point is still not a general interest to mobile development.
These results show that our approach was effective in identifying patches that could reduce the amount of distortion in a solution that was able to pass the mobile friendly test.
We found that in contrast to the distribution of all apps in App Store regarding their categories, the distribution of obfuscated apps has a vastly different pattern. This pattern varies further when the impact of third-party libraries is considered.
Based on our analysis, we found two dominant reasons that applied to all of the nine subjects.
On average, MFix improved the mobile friendliness score of a subject by 33% Overall, these results show that our approach was able to consistently improve a subject’s mobile friendliness score.
On average, the readability rating of the repaired pages showed a 17% improvement over original versions (original = 5.97, repaired = 6.98).
In fact, overall, our repaired versions were rated slightly higher than original versions for attractiveness (avg. original = 6.50, avg. repaired = 6.67 and median original = 6.02, median repaired = 7.12).
We investigated the results to understand why two subjects did not pass the GMFT. The patched version of the first subject, gsmhosting, contained a content sizing problem.
We may infer that although developers working on security-sensitive business sectors do view malicious reverse engineering as a nonneglectable threat, the obfuscation applied to their works is mostly for protecting the information encapsulated in the apps rather than the design and implementation of the software.
As can be seen from the chart, finding the repair for the mobile friendly problems (phase
3) was the most time consuming, taking up almost 60% of the
total time.
Overall, these results indicate that MFix was very effective in generating repaired pages that (1) users preferred over the original version, (2) considered to be more readable, and (3) that did not suffer in terms of visual aesthetics.
Due to limited space, we only list categories with significant relevance to the discussions in Finding A.2. It may cause confusion that a small number of apps or libraries do not employ symbol renaming.