New command-line interface (options) #181

zvr · 2025-01-17T12:24:32Z

This (finally) adds the brand new CLI.
It introduces a number of new command-line options, and essentially re-implements the top-level code of spec-parser.

As can be seen in the updated README, one can now call and only generate what is required -- or nothing at all, if only input validation is needed.

usage: main.py [-h] [-V] [-d] [-v] [-f] [-n]
               [-o OUTPUT]
               [-j] [-J dir]
               [-m] [-M dir]
               [-p] [-P dir]
               [-r] [-R dir]
               [-t] [-T dir]
               [-w] [-W dir]
               input_dir

Generate documentation from an SPDXv3 model.

positional arguments:
  input_dir             Path to the input 'model' directory.

options:
  -h, --help                                Show this help message and exit
  -d, --debug                               Print debug output
  -f, --force                               Force overwrite of existing output directories.
  -j, --generate-jsondump                   Generate a dump of the model in JSON format.
  -J, --output-jsondump OUTPUT_JSONDUMP     Output directory for JSON dump file.
  -m, --generate-mkdocs                     Generate mkdocs output.
  -M, --output-mkdocs OUTPUT_MKDOCS         Output directory for mkdocs files.
  -n, --no-output                           Perform no output generation, only input validation.
  -o, --output OUTPUT                       Single output directory for all output types.
  -p, --generate-plantuml                   Generate PlantUML output.
  -P, --output-plantuml OUTPUT_PLANTUML     Output directory for PlantUML files.
  -r, --generate-rdf                        Generate RDF output.
  -R, --output-rdf OUTPUT_RDF               Output directory for RDF files.
  -t, --generate-tex                        Generate TeX output.
  -T, --output-tex OUTPUT_TEX               Output directory for TeX files.
  -v, --verbose                             Print verbose output
  -V, --version                             Show program version number and exit
  -w, --generate-webpages                   Generate web pages output.
  -W, --output-webpages OUTPUT_WEBPAGES     Output directory for web pages.

Fixes #152
Closes #161
Closes #179

Signed-off-by: Alexios Zavras (zvr) <[email protected]>

zvr · 2025-01-17T12:25:25Z

I will wait before merging this, to give a chance to update the workflows in other repos.

goneall · 2025-01-21T19:39:13Z

@zvr - Should we start tagging releases for the spec parser? That way we can update the CI's to reference the tag and not worry about breaking them when we update the spec parser. The other advantage is that the change to update the version in the CI will trigger the CI and test out the changes before unrelated merges happen.

To implement this suggestion:

Tag the current main (e.g. 0.1.0)
Update the CI in spdx-spec repo to reference that tag rather than main
Update the CI in spdx-3-model to reference the tag rather than main
Merge this PR
Create a new tag (e.g. 1.0.0)
Update the CI in spdx-spec repo to use the new command line and update the reference in the CI
Update the CI in spdx-3-model repo to use the new command line and update the reference in the CI

bact · 2025-01-23T05:42:44Z

The new command line looks great. Thank you @zvr

Also agrees with @goneall suggestions on versioning/tagging.

Workflows to be updated (in different branches, may including support/3.0):

Validation on spdx-3-model repo: https://github.com/spdx/spdx-3-model/blob/main/.github/workflows/validate-pr.yml
- Relevant lines: https://github.com/spdx/spdx-3-model/blob/a745f63e8643d5ae0f0851fcfa6836085308f80b/.github/workflows/validate-pr.yml#L24
Publication on spdx-spec repo: https://github.com/spdx/spdx-spec/blob/develop/.github/workflows/publish_v3.yml
- Relevant lines: https://github.com/spdx/spdx-spec/blob/61a649da8ca27924ac1ca8d2a061cb228839b24c/.github/workflows/publish_v3.yml#L40-L43

zvr · 2025-01-23T09:35:29Z

@bact On the workflows:

for the model repo, there is no change needed, the invocation with -n is correct. One might delete the call with -h while setting it up (line 21), which does not serve any purpose
for the spec repo, from a quick look the changes should be:
- fix the invocation of spec-parser on line 151
- no need to install a fake pandoc (lines 145-149)
- probably no need to do all the copying around (lines 166-187, 200-208, etc.) or creating directories (lines 152-155), if the invocation (first sub-bullet above) specifies where the output for each type (mkdocs and rdf) should go.

zvr · 2025-01-23T09:54:23Z

I disagree on the tagging, especially if combined with the understanding that "tags are immutable".

There is no plan to support more than one (the latest) version of spec-parser. So, any changes/enhancements/bug fixes will only be done in a sequential linear way, on a single branch. I mean, if we decide that "superclasses" output need to be changed (cf. spdx/spdx-3-model#960) the changes will only be applied to the latest version (for example, the one with the updated CLI options, if it has been merged). Therefore we should really discourage use of older versions.

Giving "version" tags is helpful when the "API" changes in an incompatible way, so that callers can still use the old one (as Gary described). But this is not happening "often" (or ever). This change on this PR is the only such change in the last two years and I have announced it... a year(?) ago. On the other hand, there will be changes in the functionality (for example, generating the web pages to point the RDF endpoints to), without change in the calling API. Having an immutable tag on a version prevents users from getting the latest enhancements/fixes for this.

There will be more API changes coming (e.g. including an option to specify which languages should be handled, when multi-language input is available), but my plan is to have all these changes made in a backwards compatible manner -- that's why I took so long to design and implement the system in this PR.

In the end, if any script/automation really needs to use a specific instance of spec-parser, it can always use the corresponding commit-id.

bact · 2025-01-23T10:17:27Z

for the model repo, there is no change needed, the invocation with -n is correct. One might delete the call with -h while setting it up (line 21), which does not serve any purpose

Do we like to have --debug here as well so the PR author have more information ?

(Update: PR proposed here: Print verbose info during PR validation (target: 'develop' branch) spdx-3-model#963)

bact · 2025-01-23T10:41:21Z

runparams.py

+        parser = argparse.ArgumentParser(description="Generate documentation from an SPDXv3 model.")
+
+        parser.add_argument("input_dir", type=str, help="Path to the input 'model' directory.")
+
        parser.add_argument("-d", "--debug", action="store_true", help="Print debug output")


Suggested change

parser.add_argument("-d", "--debug", action="store_true", help="Print debug output")

parser.add_argument("-d", "--debug", action="store_true", help="Print spec-parser debug information.")

~~To tell that this will be more verbose than --verbose~~

To tell that this is for spec-parser debugging purpose.

No, debug is not "more than verbose". It's a completely different functionality.

I agree that the current implementation with logging levels, DEBUG is a superset of INFO, but this might not stay the same way, once more messages get added.

Thank you. I have update the description accordingly.

zvr · 2025-01-23T10:53:40Z

Do we like to have --debug here as well so the PR author have more information ?

No, --debug is for debugging the spec-parser software; it should never be used by people not interested in modifying spec-parser.

bact

Put in help description to specifically say that --debug is for the debug of spec-parser (not the debug of the Markdown spec file) as described by the author in the comment section.

General stylistic suggestions. No need to change if feel unnecessary.

bact · 2025-01-23T13:31:50Z

runparams.py

+        parser = argparse.ArgumentParser(description="Generate documentation from an SPDXv3 model.")
+
+        parser.add_argument("input_dir", type=str, help="Path to the input 'model' directory.")
+
        parser.add_argument("-d", "--debug", action="store_true", help="Print debug output")


Thank you. I have update the description accordingly.

bact · 2025-01-23T13:32:35Z

runparams.py

+        parser.add_argument("-r", "--generate-rdf", action="store_true", help="Generate RDF output.")
+        parser.add_argument("-R", "--output-rdf", type=str, help="Output directory for RDF files.")
+        parser.add_argument("-t", "--generate-tex", action="store_true", help="Generate TeX output.")
+        parser.add_argument("-T", "--output-tex", type=str, help="Output directory for TeX files.")
        parser.add_argument("-v", "--verbose", action="store_true", help="Print verbose output")


Suggested change

parser.add_argument("-v", "--verbose", action="store_true", help="Print verbose output")

parser.add_argument("-v", "--verbose", action="store_true", help="Print verbose information.")

Other descriptions have a fullstop.

bact · 2025-01-23T13:35:04Z

runparams.py

+        parser.add_argument("-m", "--generate-mkdocs", action="store_true", help="Generate mkdocs output.")
+        parser.add_argument("-M", "--output-mkdocs", type=str, help="Output directory for mkdocs files.")


Suggested change

parser.add_argument("-m", "--generate-mkdocs", action="store_true", help="Generate mkdocs output.")

parser.add_argument("-M", "--output-mkdocs", type=str, help="Output directory for mkdocs files.")

parser.add_argument("-m", "--generate-mkdocs", action="store_true", help="Generate MkDocs output.")

parser.add_argument("-M", "--output-mkdocs", type=str, help="Output directory for MkDocs files.")

Keep the stylistic cases. Similar to PlantUML and TeX.

bact · 2025-01-23T13:36:40Z

README.md

+  -m, --generate-mkdocs                     Generate mkdocs output.
+  -M, --output-mkdocs OUTPUT_MKDOCS         Output directory for mkdocs files.


Suggested change

-m, --generate-mkdocs Generate mkdocs output.

-M, --output-mkdocs OUTPUT_MKDOCS Output directory for mkdocs files.

-m, --generate-mkdocs Generate MkDocs output.

-M, --output-mkdocs OUTPUT_MKDOCS Output directory for MkDocs files.

Keep the original casing. Similar to PlantUML and TeX.

bact · 2025-01-23T13:37:18Z

README.md

+  -h, --help                                Show this help message and exit
+  -d, --debug                               Print debug output


Suggested change

-h, --help Show this help message and exit

-d, --debug Print debug output

-h, --help Show this help message and exit.

-d, --debug Print spec-parser debug information.

Keep consistent with other descriptions that have a fullstop.

bact · 2025-01-23T13:37:40Z

README.md

+  -R, --output-rdf OUTPUT_RDF               Output directory for RDF files.
+  -t, --generate-tex                        Generate TeX output.
+  -T, --output-tex OUTPUT_TEX               Output directory for TeX files.
+  -v, --verbose                             Print verbose output


Suggested change

-v, --verbose Print verbose output

-v, --verbose Print verbose information.

Keep consistent with other descriptions that have a fullstop.

bact · 2025-01-23T13:53:22Z

Btw, in the option descriptions, we have two kinds of "output".

Screen output, as in --debug and --verbose
File output, as in TeX/MkDocs/HTML/etc generation (--generate-, --output- and --no-output)

I suggest that we change the descriptions of --debug and --verbose to use the word "information" (or "log", etc) instead of "output" to clearly differentiate the two.

Adds brand new CLI and changing top-level code

d369681

Signed-off-by: Alexios Zavras (zvr) <[email protected]>

bact approved these changes Jan 23, 2025

View reviewed changes

bact reviewed Jan 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New command-line interface (options) #181

New command-line interface (options) #181

zvr commented Jan 17, 2025

zvr commented Jan 17, 2025

goneall commented Jan 21, 2025

bact commented Jan 23, 2025 •

edited

Loading

zvr commented Jan 23, 2025

zvr commented Jan 23, 2025

bact commented Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025 •

edited

Loading

zvr Jan 23, 2025

bact Jan 23, 2025

zvr commented Jan 23, 2025

bact left a comment

bact Jan 23, 2025

bact Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025

bact Jan 23, 2025

bact Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025 •

edited

Loading

bact commented Jan 23, 2025 •

edited

Loading

	parser.add_argument("-d", "--debug", action="store_true", help="Print debug output")
	parser.add_argument("-d", "--debug", action="store_true", help="Print spec-parser debug information.")

	parser.add_argument("-v", "--verbose", action="store_true", help="Print verbose output")
	parser.add_argument("-v", "--verbose", action="store_true", help="Print verbose information.")

		parser.add_argument("-m", "--generate-mkdocs", action="store_true", help="Generate mkdocs output.")
		parser.add_argument("-M", "--output-mkdocs", type=str, help="Output directory for mkdocs files.")

		-m, --generate-mkdocs Generate mkdocs output.
		-M, --output-mkdocs OUTPUT_MKDOCS Output directory for mkdocs files.

		-h, --help Show this help message and exit
		-d, --debug Print debug output

	-v, --verbose Print verbose output
	-v, --verbose Print verbose information.

New command-line interface (options) #181

Are you sure you want to change the base?

New command-line interface (options) #181

Conversation

zvr commented Jan 17, 2025

zvr commented Jan 17, 2025

goneall commented Jan 21, 2025

bact commented Jan 23, 2025 • edited Loading

zvr commented Jan 23, 2025

zvr commented Jan 23, 2025

bact commented Jan 23, 2025 • edited Loading

bact Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

zvr Jan 23, 2025

Choose a reason for hiding this comment

bact Jan 23, 2025

Choose a reason for hiding this comment

zvr commented Jan 23, 2025

bact left a comment

Choose a reason for hiding this comment

bact Jan 23, 2025

Choose a reason for hiding this comment

bact Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

bact Jan 23, 2025

Choose a reason for hiding this comment

bact Jan 23, 2025

Choose a reason for hiding this comment

bact Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

bact Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

bact commented Jan 23, 2025 • edited Loading

bact commented Jan 23, 2025 •

edited

Loading

bact commented Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025 •

edited

Loading

bact Jan 23, 2025 •

edited

Loading

bact commented Jan 23, 2025 •

edited

Loading