[PRE REVIEW]: Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models

[editorialbot]

Submitting author: @AldoGl (Aldo Glielmo)
Repository: https://github.com/bancaditalia/black-it/
Branch with paper.md (empty if default branch): joss-paper
Version: v0.1.1
Editor: @drvinceknight
Reviewers: @Athene-ai, @aseyq
Managing EiC: Kevin M. Moerman

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/1d2ecc86d273ccbae84d6df98e396593"><img src="https://joss.theoj.org/papers/1d2ecc86d273ccbae84d6df98e396593/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/1d2ecc86d273ccbae84d6df98e396593/status.svg)](https://joss.theoj.org/papers/1d2ecc86d273ccbae84d6df98e396593)

Author instructions

Thanks for submitting your paper to JOSS @AldoGl. Currently, there isn't a JOSS editor assigned to your paper.

@AldoGl if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.17 s (724.5 files/s, 143407.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          78           1506           2894           4028
Markdown                        20            289              0            766
YAML                             5             34            121            544
Jupyter Notebook                 7              0          12284            528
TeX                              1             44              6            220
INI                              2             23              2            152
make                             1             51             13            149
Bourne Shell                     4             27             16            104
TOML                             1              5              0             86
Julia                            1             12             11             35
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                           121           1991          15347           6613
-------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1433

[editorialbot]

Failed to discover a Statement of need section in paper

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.11606/d.45.2021.tde-24022021-164949 may be a valid DOI for title: Community detection in graphs
- 10.2307/j.ctv1dv0tg9.21 may be a valid DOI for title: Dynamic models of segregation
- 10.2139/ssrn.2850414 may be a valid DOI for title: Macroprudential policy in an agent-based model of the UK housing market
- 10.2139/ssrn.3891583 may be a valid DOI for title: Macroprudential policy analysis via an agent based model of the real estate sector
- 10.1145/264029.264064 may be a valid DOI for title: Computational investigations of low-discrepancy sequences
- 10.1145/2739482.2768468 may be a valid DOI for title: Benchmarking Gaussian processes and random forests surrogate models on the BBOB noiseless testbed
- 10.1098/rsos.150703 may be a valid DOI for title: Dynamic calibration of agent-based models using data assimilation
- 10.2139/ssrn.2943297 may be a valid DOI for title: Agent-based model calibration using machine learning surrogates
- 10.1007/s10614-021-10095-9 may be a valid DOI for title: Bayesian estimation of economic simulation models using neural networks
- 10.2139/ssrn.3335397 may be a valid DOI for title: A comparison of economic agent-based model calibration methods
- 10.1016/j.jempfin.2009.06.006 may be a valid DOI for title: Applying the method of simulated moments to estimate a small agent-based asset pricing model
- 10.1016/j.ecosta.2017.01.006 may be a valid DOI for title: An information theoretic criterion for empirical validation of simulation models

INVALID DOIs

- None

[editorialbot]

⚠️ An error happened when generating the pdf.

[Kevin-Mattheus-Moerman]

@AldoGl

• The above paper compilation error says: Problem with ORCID (0000-0000-0000-0000) for Marco Benedetti. Invalid ORCID, can you please check your ORCID entries to make sure they are valid? You can remove the ORCID field if a person does not have an ORCID. Once you've made changes you can call the following here to update the draft paper: @editorialbot generate pdf
• Also, can you please check those potentially missing DOIs? ☝️ After you are done you can call: @editorialbot check references to check them again. Thanks
• Our system detects that you currently do not have the mandatory "Statement of need" section, please have a look at our requirements on what the paper should contain and update the paper accordingly.

[Kevin-Mattheus-Moerman]

@editorialbot invite @drvinceknight as editor.

@drvinceknight is this submission something you can help handle as editor? Thanks!

[editorialbot]

Invitation to edit this submission sent!

[drvinceknight]

@Kevin-Mattheus-Moerman yup this is up my street 👍 :)

[drvinceknight]

@editorialbot set me as editor

[editorialbot]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

[drvinceknight]

@editorialbot assign me as editor

[editorialbot]

Assigned! @drvinceknight is now the editor

[AldoGl]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.11606/d.45.2021.tde-24022021-164949 may be a valid DOI for title: Community detection in graphs
- 10.2307/j.ctv1dv0tg9.21 may be a valid DOI for title: Dynamic models of segregation
- 10.2139/ssrn.2850414 may be a valid DOI for title: Macroprudential policy in an agent-based model of the UK housing market
- 10.2139/ssrn.3891583 may be a valid DOI for title: Macroprudential policy analysis via an agent based model of the real estate sector
- 10.1145/264029.264064 may be a valid DOI for title: Computational investigations of low-discrepancy sequences
- 10.1145/2739482.2768468 may be a valid DOI for title: Benchmarking Gaussian processes and random forests surrogate models on the BBOB noiseless testbed
- 10.1098/rsos.150703 may be a valid DOI for title: Dynamic calibration of agent-based models using data assimilation
- 10.2139/ssrn.2943297 may be a valid DOI for title: Agent-based model calibration using machine learning surrogates
- 10.1007/s10614-021-10095-9 may be a valid DOI for title: Bayesian estimation of economic simulation models using neural networks
- 10.2139/ssrn.3335397 may be a valid DOI for title: A comparison of economic agent-based model calibration methods
- 10.1016/j.jempfin.2009.06.006 may be a valid DOI for title: Applying the method of simulated moments to estimate a small agent-based asset pricing model
- 10.1016/j.ecosta.2017.01.006 may be a valid DOI for title: An information theoretic criterion for empirical validation of simulation models

INVALID DOIs

- None

[AldoGl]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1080/0022250X.1971.9989794 is OK
- 10.1098/rsif.2007.1206 is OK
- 10.1371/journal.pcbi.1009146 is OK
- 10.2139/ssrn.2850414 is OK
- 10.2139/ssrn.3891583 is OK
- 10.1145/264029.264064 is OK
- 10.1145/2739482.2768468 is OK
- 10.1098/rsos.150703 is OK
- 10.48550/arXiv.1605.00998 is OK
- 10.1016/j.jedc.2017.01.014 is OK
- 10.1016/j.jedc.2018.03.011 is OK
- 10.1007/s10614-021-10095-9 is OK
- 10.1016/j.jedc.2020.103859 is OK
- 10.1016/j.jempfin.2009.06.006 is OK
- 10.1016/j.ecosta.2017.01.006 is OK
- 10.1016/S0165-1889(98)00011-6 is OK
- 10.1038/30918 is OK

MISSING DOIs

- None

INVALID DOIs

- https://doi.org/10.48550/arXiv.2202.00625 is INVALID because of 'https://doi.org/' prefix

[AldoGl]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1080/0022250X.1971.9989794 is OK
- 10.1098/rsif.2007.1206 is OK
- 10.1371/journal.pcbi.1009146 is OK
- 10.2139/ssrn.2850414 is OK
- 10.2139/ssrn.3891583 is OK
- 10.1145/264029.264064 is OK
- 10.1145/2739482.2768468 is OK
- 10.1098/rsos.150703 is OK
- 10.48550/arXiv.1605.00998 is OK
- 10.1016/j.jedc.2017.01.014 is OK
- 10.1016/j.jedc.2018.03.011 is OK
- 10.1007/s10614-021-10095-9 is OK
- 10.48550/arXiv.2202.00625 is OK
- 10.1016/j.jedc.2020.103859 is OK
- 10.1016/j.jempfin.2009.06.006 is OK
- 10.1016/j.ecosta.2017.01.006 is OK
- 10.1016/S0165-1889(98)00011-6 is OK
- 10.1038/30918 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[AldoGl]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[drvinceknight]

@AldoGl would you be able to take a look at this list of people and suggest a few that you feel would be a good fit to review this work? (Please don't @ them, just let me know their handles :))

[AldoGl]

@AldoGl would you be able to take a look at this list of people and suggest a few that you feel would be a good fit to review this work? (Please don't @ them, just let me know their handles :))

Dear @drvinceknight,

I had a look at the list you sent me. I found the following people who are experienced in Python and who work on computational economics and simulation methods. They could be particularly interested and suited to review our work, and the highlighted names seem the best candidates

• rickecon
• igarizio
• Athene-ai
• seyhunsaral

Alternatively, I also identified people experienced in Python and working on simulation methods and agent-based modelling in other social fields (not in economics). These might also be interested in reviewing our work as our tool is not intended solely for economic ABMs. Also in this case the highlighted names indicate the candidates who seem the most appropriate

• chrisvoncsefalvay
• Christovis
• dinacmistry; DMistry-IDM
• rusu24edward
• jamesdamillington
• RafalKucharskiPK

I hope this helps.

Best regards,
Aldo

[drvinceknight]

Very helpful, thank you @AldoGl.

[drvinceknight]

@rickecon @igarizio @Athene-ai @seyhunsaral I'm pinging you to ask if you would be able to assist with reviewing this submissions to JOSS (The Journal of Open Source Software).

JOSS publishes articles about open source research software. The submission I'd like you to review is titled: “Black-it: A Ready-to-Use and Easy-to-Extend Calibration Kit for Agent-based Models”. You can find more information at the top of this Github issue.

The review process at JOSS is unique: it takes place in a GitHub issue, is open, and author-reviewer-editor conversations are encouraged. If you have any questions please let me know.

[Athene-ai]

@drvinceknight I am very happy to serve as reviewer

[drvinceknight]

Thank you @Athene-ai!

Once I have a second reviewer I will start the review process.

[drvinceknight]

@editorialbot add @Athene-ai as reviewer

[editorialbot]

@Athene-ai added to the reviewers list!

[seyhunsaral]

It would be my pleasure to serve as well. Do you mind adding my active github account @aseyq instead of this one?

[drvinceknight]

Of course @aseyq and thank you.

[drvinceknight]

@editorialbot add @aseyq as reviewer

[editorialbot]

@aseyq added to the reviewers list!

[drvinceknight]

@editorialbot start review

[editorialbot]

OK, I've started the review over in #4622.

[drvinceknight]

The review takes place over on #4622, please let me know if there is anything I can assist with at any stage.