Dear Authors

Thank you for your responses to both reviewer reports. Based on these reports and your responses, I invite you to submit a revised version of you manuscript. Upon receipt of this I will seek another round of peer review from the first round of reviewers and, if required, new reviewers.

You manuscript struggles in places to sufficiently argue that copan:CORE represents an important contribution to science as such an argument is potentially strongly dependent on the specifics of the framework itself. One approach would be to publish the model, or framework, or platform with associated (and potentially quite extensive) details about assumptions, constraints, limitations and - crucially - implementation. This would represent sufficient information for a reviewer or independent researcher, to evaluate the framework. After the framework is published and so accessible, future manuscripts could focus on applications: what science can be progressed with this framework.

Your manuscript can be viewed as being motivated by making the case for the development of WEMs, of which your framework is an example. Indeed you conclude your manuscript with something of a call to arms: “We therefore call upon the interdisciplinary social-ecological modelling community to participate in further model and application development to facilitate “whole” Earth system analysis”.

In that respect copan:CORE is a demonstration or realisation of a new generation of model frameworks. This, as I think you argue in your R2 response, means that issues of computational cost, efficiency, and performance have not been discussed in this manuscript. Future iterations of coupon:CORE may be in response to performance.

The risk here is that you fall between the gap between framework specification and the academic motivation and application of any models and frameworks. I think this in part explains some comments from both reviewers. I can see that you understand these comments and have a plan in place to address them. e.g. potential confusion with “object-orientated”. I had one minor concern with the use of an ‘in prep’ manuscript to justify some aspects of framework implementation. e.g. Table 1 caption: “Possible classification of example model processes by owning process taxon (row) and affected process taxon (column) (following Donges et al. (in prep.)”

At 27 pages, your manuscript is not short. I currently do not see how sufficiently addressing all the issues raised in the first round of review can be achieved without increasing the number of pages further. I very much keep an open mind about overall manuscript length with future revisions.

This is a timely, ambitious, and important contribution to the scientific debate about the role of human actions on the Earth system. While the authors have made improvements to the manuscript, I have concluded that it is not currently acceptable for publication based on the latest round of reviews and a consideration of previous reviews and how the manuscript has developed. Consequently I recommend revisions with another round of peer review. I am confident that this review process could be completed quickly and that this manuscript can be published in ESD in a timely manner.

I see three main possible contributions of the manuscript:

C1 - Argue that coupling of human actions on the Earth systems is necessary or important for a class of scientific and policy-relevant questions.

C2 - Assume C1 (coupling is required) and then demonstrate how such coupling can be implemented into a scalable framework.

C3 - Provide worked examples of the output of a coupling framework that provides new scientific insights and findings that may be of policy relevance.

In attempting all three contributions, the manuscript is at risk of not comprehensively addressing any. In its current guise the manuscript attempts to justify the requirement for coupling, show how it could be achieved with a particularly modelling or implementation framework, and then demonstrate output from this implementation. This is ambitious. My questions on publication surrounds whether its feasible, or even desirable, to do all of that in a single publication. The temptation is to ask for more in order to cover C1-C3, but this may not be the best approach.

Given the authors do not present a single model, but a framework with which a potentially very large range of models could be developed and then integrated, the issues of reproducibility are quite complicated.

Documentation of the software implementation of the python copan:CORE model is available at the GitHub site. There is also documentation available at https://pycopancore.readthedocs.io/en/latest/index.html
This includes a set of tutorials, e.g. how does a user uses the package. It also describes some of the implementation of the assumptions behind the model. I take the job this (or similar) documentation also needs to do, is establish C2. Presentation of a model framework (i.e. code that ‘does something’ with instructions of ‘how to do something’ does not necessarily establish that C2 has been satisfied.

In these respects, what is absent is software verification and validation. This is what I take to be the substantive issue of journal scope. ESD does not require nor does it request the presentation of the verification and validation of software that is used to produce scientific output. ESD would rely on the this activity being done in another journal, platform, or forum. From an editorial perspective, this has been one of the main challenges of ensuring appropriate review of the manuscript.

To proceed, I suggest the following routes to revision. Consider a clear separation between the model implementation and the assumptions surrounding it and its output. Specifically:

R1 - Argue that there is a need - scientific and of policy relevance - for such coupling and then

R2 - that the overall design approach taken - in particular the use of agent based implementations - satisfies this need.

R3 - Demonstrate the utility of this approach with results that either could not be produced via ’traditional’ or established models and frameworks, or results that provide new insights into the class or perhaps example of the scientific (and potentially policy relevant) questions that the authors are motivated to address.

The existing manuscript does currently address R1-R3, but there are some gaps. For example, it may be argued that (potentially computationally expensive) agent based implementations are not required for some of the class of scientific questions that the authors argue their framework is designed for. There is a broader issue of to what extent human decision-making and behaviour needs to be disaggregated in social-ecological models. It is not the author’s job to conclude such arguments! But the authors should be mindful that some readers (and reviewers) may need to carried along with their argument that is there is utility in agent based implementations.

Currently, for R1 a good deal of the ‘intellectual heavy lifting - the 'motivation’ comes from just two papers Verburg et al., 2016; Donges et al., 2017a, b and five bullet points in section 1.2.2 Design principles for World-Earth models. Given the planetary-scale ambition of the authors, it is understandable that there cannot be a sufficient list of possible scenarios and scientific questions that the framework will seek to address. But the absence of a discrete set of questions, plus some opacity as to the central modelling assumptions means that R1 is not sufficiently addressed.

For R2, it should be sufficient to focus on the disaggregated social dynamics approach - that is to establish that at a planetary-scale, aggregating human behaviours is not sufficient as it means missing important dynamics. I would have assumed that a planetary-scale framework would have some sort of tunable parameter by which this level of disaggregation can be ’turned up/down’ as appropriate. Arguing that it’s important to include low(er) level dynamics in order to understand high(er) level behaviours/states/responses works inversely: that is, it is sometimes better (in terms not just of computational cost/efficiency but also in terms of mathematical tractability and conceptual understanding) to simplify lower level dynamics. Sometimes more is less.

For R3 you have already presented output from core:COPAN. This is ’toy’ model output as I take your main motivation is to demonstrate that the core:COPAN framework works - it is able to produce model output. I do not think it is necessary to implement a real world model (to use empirical data for the choice of model elements and parameter values), but I do think that there could be more convincing results presented. It is not surprising that a more complex model will produce quite different results. What new insights, what new knowledge can be gained here? This questions can be addressed using a toy or conceptual model. I would hope, that once the manuscript is freed from the requirement of establishing that the copan:CORE framework is verified and validated, you would be able to concentrate on demonstrating some of its power. This would not require large numbers of new simulations and analysis.

In this way, the manuscript is bookended by R1 and R3. R3 in an important sense helps demonstrate R1 - why do we need these potentially very complex models and approaches?

Separate (and perhaps in parallel) to this, I would recommend you expose the modelling framework to suitable review with verification and validation built into that process via perhaps a dedicated modelling journal. This more modular approach would allow you to produce a series of publications using a common and well established foundation for the implemented framework which could be referred and referenced to accordingly.

If you are able to revise your manuscript, then I would seek an additional round of independent peer review - but of a potentially limited nature. While I do not want to prejudice future outcomes, it may be sufficient for the review to progress with the response of one of the reviewers from the last round of peer review. I do not think it would be necessary to effectively treat the manuscript as a new submission with an entirely new round of peer review. However, I am open to your alternative suggestions.

Thank you for your continued contribution to Earth System Dynamics.

Dear Authors

Thank you for revising your manuscript. It is clear that there is still some work to be done in order for this to be published in Earth System Dynamics. The latest review demonstrates again that this reviewer (at least) is not convinced that there is sufficient scientific insight. The suggestion again is made to take the modelling elements of the paper and publish them in an appropriate journal.

In some respects it may be self-evident why your modelling approach is required. But if that is the case, then it would support the argument that this is a methodological paper that would be best suited in another journal. The latest review report continues the theme that what is missing is the exploration, discussion, and critical evaluation of why such a methodology is useful, or important, or even necessary. i.e. the justification for coupled human-earth system modelling. The manuscript falls between these two stalls and as such is not getting above the bar required for publication.

I appreciate that this is disappointing outcome and that this already has been a long and quite protracted review process that has involved multiple revisions. I do not think there is any utility in expanding the review process by inviting further reviewers. I think your task is to reflect upon the latest review and consider if you will be able to respond. I will very carefully consider your response and I would hope that if you are able to proceed you would be able to produce a revision in a timely manner and that this can be reviewed without delay.

Best wishes

James

Dear Authors

Thank you for your reply to the previous round of peer review. Previously, I provided the following three recommendations based on the July 2019 revision:

1 - An important motivation is to promote copan:CORE to potential users. Rather than a “towards” paper, a paper that is in some sense a manifesto that argues that “something should be done”, your manuscript alerts researchers to a new thing in the modelling world and asks them to use it. I would assume it would be critical that users are able to quickly and effectively get to grips with copan:CORE. I still get a 404 for the link http://pycopancore.readthedocs.io/ on the GitHub page. The only documentation I can see is very limited. This would also in part address some of the grumbles about reproducibility.

2 - There will always be some pointed skepticism about the role, place and utility of global socio-ecological models (e.g. World3). I wouldn’t shrink from addressing some of them quite directly. Be open about that fact that some people/communities think they area very poor science. I think the power of your approach is that you are agnostics about modules. The proof will be in the pudding in that individual model formations will sink or swim on their robustness. You are providing the platform for model development. For the motivation of copan:CORE I think it’s sufficient to point out that many of the key sustainability challenges we face intimately involve the sort of coupled dynamics and feedback loops that copan:CORE is designed to implement effectively.

3 - Beyond offering a platform, I think you are also potentially looking at managing a community of modellers or at least a repository of copan:CORE models. How that is managed will I think be crucial. It would be great to see enthusiastic take up of the platform and many models being produced. But it would a missed opportunity if these models were not curated in some respect. We need to ensure the sum of the modellers efforts are greater than the parts. We need ways to share best practice for these sorts of models. Perhaps some copan:CORE tailored practices from Software Carpentry (https://software-carpentry.org) perhaps? I appreciate that is potentially way beyond the scope of the paper, but if this paper is the first exposure someone has to the platform then you will want some things in place to ensure you “capture” them and their outputs.

I can see from your annotated revised manuscript that you recently submitted, that you have sufficiently addressed all three points. I am therefore happy to accept this revision for publication subject to the minor technical corrections of producing a version of the manuscript for final publication.

Best wishes
James Dyke