init: hello world jobspec

Signed-off-by: vsoch <[email protected]>

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,73 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team leader @vsoch. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org

.github/dev-requirements.txt

@@ -0,0 +1,4 @@
+pre-commit
+black==23.3.0
+isort
+flake8

.github/workflows/main.yaml

@@ -0,0 +1,28 @@
+name: jobspec test
+
+on:
+  push:
+    branches:
+      - main
+  pull_request: []
+
+jobs:
+  formatting:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Setup black linter
+      run: conda create --quiet --name black pyflakes
+
+    - name: Check Spelling
+      uses: crate-ci/typos@7ad296c72fa8265059cc03d1eda562fbdfcd6df2 # v1.9.0
+      with:
+        files: ./README.md
+
+    - name: Lint and format Python code
+      run: |
+        export PATH="/usr/share/miniconda/bin:$PATH"
+        source activate black
+        pip install -r .github/dev-requirements.txt
+        pre-commit run --all-files

.gitignore

@@ -0,0 +1,5 @@
+jobspec.egg-info
+build
+dist
+env
+.eggs

.pre-commit-config.yaml

@@ -0,0 +1,26 @@
+exclude: "examples"
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.3.0
+    hooks:
+      - id: check-added-large-files
+      - id: check-case-conflict
+      - id: check-docstring-first
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      - id: mixed-line-ending
+
+  - repo: local
+    hooks:
+      - id: black
+        name: black
+        language: python
+        types: [python]
+        entry: black
+
+      - id: isort
+        name: isort
+        args: [--filter-files]
+        language: python
+        types: [python]
+        entry: isort

COPYRIGHT

@@ -0,0 +1,15 @@
+Intellectual Property Notice
+----------------------------
+
+HPCIC DevTools is licensed under the MIT license (LICENSE).
+
+Copyrights and patents in this project are retained by
+contributors. No copyright assignment is required to contribute to
+HPCIC DevTools.
+
+SPDX usage
+------------
+
+Individual files contain SPDX tags instead of the full license text.
+This enables machine processing of license information based on the SPDX
+License Identifiers that are available here: https://spdx.org/licenses/

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022-2023 LLNS, LLC and other HPCIC DevTools Developers.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

MANIFEST.in

@@ -0,0 +1,6 @@
+recursive-include jobspec *
+prune env*
+global-exclude .env
+global-exclude *.py[co]
+recursive-exclude .git *
+global-exclude __pycache__

NOTICE

@@ -0,0 +1,21 @@
+This work was produced under the auspices of the U.S. Department of
+Energy by Lawrence Livermore National Laboratory under Contract
+DE-AC52-07NA27344.
+
+This work was prepared as an account of work sponsored by an agency of
+the United States Government. Neither the United States Government nor
+Lawrence Livermore National Security, LLC, nor any of their employees
+makes any warranty, expressed or implied, or assumes any legal liability
+or responsibility for the accuracy, completeness, or usefulness of any
+information, apparatus, product, or process disclosed, or represents that
+its use would not infringe privately owned rights.
+
+Reference herein to any specific commercial product, process, or service
+by trade name, trademark, manufacturer, or otherwise does not necessarily
+constitute or imply its endorsement, recommendation, or favoring by the
+United States Government or Lawrence Livermore National Security, LLC.
+
+The views and opinions of authors expressed herein do not necessarily
+state or reflect those of the United States Government or Lawrence
+Livermore National Security, LLC, and shall not be used for advertising
+or product endorsement purposes.

README.md

@@ -0,0 +1,23 @@
+# jobspec (python)
+
+> Translation layer for a jobspec specification to cluster execution
+
+![img/jobspec-bot.png](img/jobspec-bot.png)
+
+This is intended to work with the prototype [rainbow](https://github.com/converged-computing/rainbow) scheduler.
+Jobspec is an entity of [flux-framework](https://flux-framework.org).
+
+**under development**
+
+## License
+
+HPCIC DevTools is distributed under the terms of the MIT license.
+All new contributions must be made under this license.
+
+See [LICENSE](https://github.com/converged-computing/cloud-select/blob/main/LICENSE),
+[COPYRIGHT](https://github.com/converged-computing/cloud-select/blob/main/COPYRIGHT), and
+[NOTICE](https://github.com/converged-computing/cloud-select/blob/main/NOTICE) for details.
+
+SPDX-License-Identifier: (MIT)
+
+LLNL-CODE- 842614

jobspec/core/__init__.py

@@ -0,0 +1,4 @@
+# Expose the latest version as core Jobspec
+# If the user wants an earlier version (when we have them)
+# they can import it
+from .v1 import JobspecV1 as Jobspec

jobspec/core/converter.py

@@ -0,0 +1,97 @@
+import shlex
+
+
+def from_compatibility_spec(
+    compspec, command, nodes, name=None, tasks=1, jobspec_version=1, attributes=None
+):
+    """
+    Generate a jobspec from a compatibility spec.
+    """
+    # Start with a basic one we will modify
+    js = new_simple_jobspec(command, nodes, name, tasks, jobspec_version)
+
+    # This is an optional lookup that can filter down to attributes of interest
+    attributes = attributes or {}
+
+    # The first task is lammps
+    task = js["tasks"][0]
+
+    # Generate task resources based on compatibility metadata
+    # Note that each of these has associated graphs we aren't using
+    resources = {}
+    for compat in compspec.get("compatibilities", []):
+        # Treat these flat for now. E.g., io.archspec instead of io -> archspec
+        resource_set = {}
+
+        # Here we are creating a set of attributes for a subsystem.
+        # E.g., "For io.archpsec I care about cpu.target"
+        name = compat.get("name")
+
+        # Skip those we don't care about for the level
+        if not name or (attributes and name not in attributes):
+            continue
+
+        for attrname, attrvalue in compat.get("attributes", {}).items():
+            if attributes and attrname not in attributes[name]:
+                continue
+            resource_set[attrname] = attrvalue
+
+        # Only add resource sets with at least one attribute
+        if resource_set:
+            resources[compat["name"]] = resource_set
+
+    if resources:
+        task["resources"] = resources
+
+    js["tasks"][0] = task
+    return js
+
+
+def new_simple_jobspec(command, nodes=1, name=None, tasks=1, jobspec_version=1):
+    """
+    Generate a new simple jobspec from basic parameters.
+    """
+    if isinstance(command, str):
+        command = shlex.split(command)
+
+    if not command:
+        raise ValueError("A command must be provided.")
+
+    # If we don't have a name, derive one
+    if name is None:
+        name = command[0]
+
+    if nodes < 1 or tasks < 1:
+        raise ValueError("Nodes and tasks for the job must be >= 1")
+
+    node_resource = {
+        "type": "node",
+        "count": nodes,
+    }
+    slot = {
+        "type": "slot",
+        "count": 1,
+        "label": name,
+    }
+    task_resource = {
+        "type": "core",
+        "count": tasks,
+    }
+    slot["with"] = [task_resource]
+
+    node_resource["with"] = [slot]
+    task_resources = [
+        {
+            "command": command,
+            "slot": name,
+            "count": {
+                "per_slot": 1,
+            },
+        }
+    ]
+    return {
+        "version": jobspec_version,
+        "resources": [node_resource],
+        "tasks": task_resources,
+        "attributes": {},
+    }