Skip to content

Commit

Permalink
Merge pull request #2 from ArDoCo/feature/docs
Browse files Browse the repository at this point in the history
Revise documentation of Benchmark
dfuchss authored Aug 5, 2022
2 parents 8561a49 + 85876db commit 0237ea1
Showing 7 changed files with 127 additions and 1 deletion.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @Gram21 @dfuchss @Hossiphi
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2022 ArDoCo
Copyright (c) 2022 Sophie Corallo, Dominik Fuchß, Jan Keim

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,38 @@
# Benchmark
This repository contains a benchmark for traceability link recovery (TLR) between textual Software Architecture Documentation (SAD) and Software Architecture Models (SAM).

Each project of the benchmark is structured as follows:

* The README of each project contains some information about the used languages and lines of code created with [cloc](https://github.com/AlDanial/cloc).
* The folder `pcm` contains a Palladio Component Model (PCM) of the system. It has at least the repository view (components) of the system.
* The text file(s) in the project folder contains the SAD of the projects as plain text.
* The file `goldstandard.csv` contains the traceability links between SAD and SAM. It links the model elements by id with the sentences by their number (index starting at 1).

## Using the benchmark
In order to provide an easy approach to use the benchmark, we provide an example TLR approach called [Simple Tracelink Discovery (STD)](https://github.com/ArDoCo/SimpleTracelinkDiscovery/) that uses this benchmark in its [evaluation](https://github.com/ArDoCo/SimpleTracelinkDiscovery/tree/main/src/test/java/io/github/ardoco/simpletracelinkdiscovery/eval).
Therefore, the benchmark is linked to the STD repository via a [git subtree](https://github.com/ArDoCo/SimpleTracelinkDiscovery/tree/main/src/test/resources/benchmark).

## References
> **Note**
>
> Our LICENSE does only apply to the PCM models and the Gold Standards (CSV files). The texts are licensed w.r.t. to the actual projects.
### BigBlueButton
BigBlueButton (BBB) is a non-scientific application that provides a web conferencing system with the focus on creating a "global teaching platform".
The [documentation of BBB](https://docs.bigbluebutton.org/2.4/architecture.html) is licensed under LGPL.
Therefore, the text we extracted from their documentation is licensed according to the [license of BBB](https://bigbluebutton.org/open-source-project/open-source-license/) under LGPL.

### MediaStore
MediaStore is a "model application built after the iTunes Store".
Its architecture was used for exemplary performance analyses on software architecture models.
The text we extracted from their documentation originates from the publication [Modeling and Simulating Software Architectures: The Palladio Approach](https://books.google.de/books?id=g6BSDQAAQBAJ).

### Teammates
TEAMMATES is an open-source "online tool for manageing peer evaluations and other feedback paths of your students".
The documentation of TEAMMATES is part of their [repository](https://github.com/TEAMMATES/teammates).
Therefore, the text we extracted from their documentation is licensed according to the [license of TEAMMATES](https://github.com/TEAMMATES/teammates/blob/master/LICENSE) under GPL-2.0.

### Teastore
Teastore is a scientific application that is used as a "micro-service reference test application".
The [documentation](https://web.archive.org/web/20201102180945/https://github.com/DescartesResearch/teastore/wiki/Services) of Teastore was part of their [repository](https://github.com/DescartesResearch/teastore).
Therefore, the text we extracted from their documentation is licensed according to the [license of Teastore](https://github.com/DescartesResearch/TeaStore/blob/master/README.md) under Apache-2.0.
34 changes: 34 additions & 0 deletions bigbluebutton/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# BigBlueButton Statistics (cloc)
```
--------------------------------------------------------------------------------
Language files blank comment code
--------------------------------------------------------------------------------
JSON 84 182 0 100980
JavaScript 949 10983 10387 68930
JSX 342 5919 1382 47177
CSS 39 3350 1079 28297
Scala 421 4861 2439 22317
Java 379 5058 5219 21372
XML 62 494 1301 20119
Ruby 60 1443 1667 6588
Bourne Shell 148 1133 615 3556
Groovy 18 589 516 3116
Bourne Again Shell 9 448 484 2308
YAML 14 54 345 1147
SVG 20 8 16 1065
HTML 8 138 129 762
Markdown 19 256 0 580
Python 2 110 64 512
Scheme 6 86 0 381
Maven 2 9 23 240
Gradle 4 21 8 159
DOS Batch 2 44 4 130
SQL 1 8 0 67
ERB 1 2 0 33
Cucumber 2 5 0 21
Dockerfile 1 2 0 13
TOML 1 0 1 2
--------------------------------------------------------------------------------
SUM: 2594 35203 25679 329872
--------------------------------------------------------------------------------
```
16 changes: 16 additions & 0 deletions mediastore/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# MediaStore Statistics (cloc)
```
-----------------------------------------------------------------------------------
Language files blank comment code
-----------------------------------------------------------------------------------
Java 97 1126 1978 3709
Maven 23 154 43 1193
XML 24 2 25 590
Visualforce Component 22 0 0 378
XHTML 5 59 8 214
CSS 1 63 0 68
SQL 4 0 0 7
-----------------------------------------------------------------------------------
SUM: 176 1404 2054 6159
-----------------------------------------------------------------------------------
```
23 changes: 23 additions & 0 deletions teammates/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# TEAMMATES Statistics (cloc)
```
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
JSON 93 60 0 91232
Java 826 22948 11357 90733
TypeScript 782 7652 5963 53747
HTML 298 706 85 16447
Sass 121 441 4 2129
Markdown 35 764 0 1622
XML 10 40 35 854
YAML 13 42 114 696
Gradle 1 69 11 509
Bourne Shell 3 32 112 128
CSS 2 27 6 120
JavaScript 6 4 0 100
DOS Batch 1 21 2 68
Dockerfile 3 6 0 11
-------------------------------------------------------------------------------
SUM: 2194 32812 17689 258396
-------------------------------------------------------------------------------
```
25 changes: 25 additions & 0 deletions teastore/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# TeaStore Statistics (cloc)
```
-----------------------------------------------------------------------------------
Language files blank comment code
-----------------------------------------------------------------------------------
Java 200 2591 7933 12295
CSS 5 58 24 7598
YAML 32 44 59 2824
JavaScript 13 644 250 1848
JSON 4 0 0 1280
Maven 14 61 24 991
XML 24 18 37 769
JSP 17 35 10 760
Markdown 3 138 0 352
SVG 1 0 0 288
Bourne Shell 5 27 10 209
Visualforce Component 13 0 0 154
SQL 2 31 48 146
Lua 2 14 52 122
Python 1 11 37 87
Dockerfile 7 23 12 67
-----------------------------------------------------------------------------------
SUM: 343 3695 8496 29790
-----------------------------------------------------------------------------------
```

0 comments on commit 0237ea1

Please sign in to comment.