Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the partial model for formula evaluation in Princess #428

Open
daniel-raffler opened this issue Jan 11, 2025 · 2 comments
Open

Use the partial model for formula evaluation in Princess #428

daniel-raffler opened this issue Jan 11, 2025 · 2 comments

Comments

@daniel-raffler
Copy link
Contributor

Hello,
We stopped using the partial model in Princess here as there were several solver issues when using it with rational formulas. Most of these issues have now been resolved by the Princess developers and we should look into whether it's possible to return to using the partial model in JavaSMT.

@daniel-raffler
Copy link
Contributor Author

There is an issue in Ostrich/Princess that affects String evaluation and is blocking us from using the partial model for now. I've opened an issue here: uuverifiers/ostrich#94

@daniel-raffler
Copy link
Contributor Author

Other than that it seems to be working fine. I've updated the Princess backend and ran some benchmarks with CPAchecker:

Image

The timeout is 60 seconds and --option cpa.predicate.encodeFloatAs=RATIONAL was used to specifically test rational formulas as they used to cause crashes when being evaluated. The results on the left are with the current version, and the new version that uses the partial model can be found on the right. The old version produces a large number of incorrect false results that are avoided with the new code. However, these runs will still result in an Assertion error with the new version, and the message reads "Found imprecise counterexample with BMC".

The new version also returns fewer correct results, and performance seems to be slower, especially for more expensive runs:

Image

Here the blue/green line is the old version, and the brown line is with the new version that uses the partial model. I would suggest we still update as soon as the bug is fixed as this will allow us to remove the current workaround for the partial model from our code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant