Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compilation fails when using an empty string as key #785

Closed
bastjan opened this issue Apr 26, 2023 · 2 comments
Closed

Compilation fails when using an empty string as key #785

bastjan opened this issue Apr 26, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@bastjan
Copy link
Contributor

bastjan commented Apr 26, 2023

Empty strings as key are valid in JSON and this in YAML.

The JSON syntax does not impose any restrictions on the strings used as names, does not require that name strings be unique, and does not assign any significance to the ordering of name/value pairs.

Steps to Reproduce the Problem

  1. Use an empty string as a key in the hierarchy. See Map BEs with empty country to Switzerland (the default) appuio/component-control-api#85
  2. Compile

Actual Behavior

/usr/local/bin/docker run --rm -u "$(id -u):$(id -g)" --userns="" -w /control-api -e HOME="/control-api"  -v "${PWD}:/control-api" docker.io/projectsyn/commodore:latest component compile . --search-paths ./dependencies --search-paths . -n control-api -f tests/defaults.yml
Agent pid 11
grep: /app/.ssh: No such file or directory
No SSH keys were added
Compile component control-api...
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/storage/memcache_proxy.py", line 51, in get_class
    return self._classes_cache[environment][name]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'defaults.control-api'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/commodore", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/commodore/cli/__init__.py", line 59, in main
    commodore.main(
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/commodore/cli/component.py", line 335, in component_compile
    compile_component(config, path, alias, values, search_paths, output, name)
  File "/usr/local/lib/python3.11/site-packages/commodore/component/compile.py", line 85, in compile_component
    nodes = inventory_reclass(inv.inventory_dir)["nodes"]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/resources.py", line 356, in inventory_reclass
    cached.inv = _reclass.inventory()
                 ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/core.py", line 259, in inventory
    inventory = self._get_inventory(True, '', None)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/core.py", line 195, in _get_inventory
    node = self._node_entity(nodename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/core.py", line 227, in _node_entity
    return self._recurse_entity(node_entity, merge_base=merge_base, seen=seen,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/core.py", line 137, in _recurse_entity
    class_entity = self._storage.get_class(klass, environment, self._settings)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/storage/memcache_proxy.py", line 55, in get_class
    ret = self._real_storage.get_class(name, environment, settings)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/storage/yaml_fs/__init__.py", line 113, in get_class
    entity = YamlData.from_file(path).get_entity(name, pathname, settings)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/storage/yamldata.py", line 98, in get_entity
    parameters = datatypes.Parameters(parameters, settings, self._uri)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 67, in __init__
    self.merge(mapping)
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 241, in merge
    self._base = self._merge_recurse(self._base, wrapped)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 210, in _merge_recurse
    return self._merge_dict(cur, new)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 185, in _merge_dict
    value = self._merge_recurse(cur.get(key), value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 210, in _merge_recurse
    return self._merge_dict(cur, new)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 185, in _merge_dict
    value = self._merge_recurse(cur.get(key), value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 210, in _merge_recurse
    return self._merge_dict(cur, new)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 185, in _merge_dict
    value = self._merge_recurse(cur.get(key), value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 210, in _merge_recurse
    return self._merge_dict(cur, new)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/kapitan/reclass/reclass/datatypes/parameters.py", line 174, in _merge_dict
    if key[0] in self._settings.dict_key_prefixes:
       ~~~^^^
IndexError: string index out of range

Expected Behavior

Successful compilation

@bastjan bastjan added the bug Something isn't working label Apr 26, 2023
@simu
Copy link
Member

simu commented Nov 24, 2023

This should be fixed in Commodore v1.19.1, since reclass-rs doesn't exhibit the same buggy behavior for empty keys as Python reclass.

See also appuio/component-control-api#110

@simu
Copy link
Member

simu commented Nov 27, 2023

Closing this since this bug isn't present in reclass-rs which is the reclass implementation used by Commodore since v1.18.0

@simu simu closed this as completed Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants