Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get the last iteration with iteration_encoding=variable_based #1611

Closed
costeastefan opened this issue Apr 17, 2024 · 1 comment
Closed

get the last iteration with iteration_encoding=variable_based #1611

costeastefan opened this issue Apr 17, 2024 · 1 comment
Labels

Comments

@costeastefan
Copy link

I wish to access the last iteration in a series saved to disk with "iteration_encoding": "variable_based" option and writeIterations().

To access the first iteration, I figured out the following example:

SeriesIterator it_it = series.readIterations().begin();
IndexedIteration it = *it_it;
  1. Is there a way to access the last iteration in a similar way?
  2. Is there a way to have a reverse iterator (e.g. std::vector::rbegin) for the cases when a simulation creates checkpoints using "iteration_encoding": "variable_based" and needs to restore from the latest useful iteration?

Software Environment:

  • version of openPMD-api: 0.15.1
  • installed openPMD-api via: from source
@franzpoeschel
Copy link
Contributor

Hello @stefurnic
You have found one of the main reasons why variable-based encoding is still considered experimental. The current implementation is not yet complete and focuses on streaming workflows where reverse iteration naturally does not work.
Closing the gap between streaming and random-access is currently WIP, I'm currently working on this in #1592, which prepares the API changes and adds other features like re-opening closed iterations, that are currently missing.
Since random-accessing IO steps in ADIOS2 requires a different way of interacting with the dataset, what you are asking for will be yet another PR after #1592.

So, the answer is: This is planned, but not there yet. Variable-based iteration encoding can be used today, but the workflows are restricted. If you want the full flexibility, the recommendation until then is file-based encoding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants