Dear authors,

Once you have submitted your revised version, I will ask Reviewer #2 to specifically comment on your response to his/her criticism relative to the likelihood. The other comments are mostly editorial. I appreciate that you have taken to respond in detail and consider the points of view of the reviewers. I believe we can deal with these more editorial comments at the level of a final iteration between author and editor if still needed.

Dear author,

I received one review communicated by the original Anonymous Reviewer #2, which I copy paste below along with my comments.

Comment 1: There is a reviewer/author disagreement about how pedagogic the paper should be and whether the section 2.1 should go or not in the appendix. Admittedly, the example with a
man dropping a weight is a bit lengthy, but I do not see a definitive reason to cut it. This said, a reference to an appropriate textbooks might be helpful.

Comment 2: Please respond to this comment.

Comment 3: It is correct, as you state, that Bayes Theory is a (perhaps not that trivial) consequence of the axioms of probability, however, whether the axioms of probability are the only reasonable way to handle uncertainty is perhaps not so uncontroversial. The text in its present form might leave the impression that an explicit Bayesian treatment is the only rational way to handle information about singular quantities, and the only rational way to make decisions which involve this quantity. I would indeed keep the door open to alternative treatments that might be as effective (if not more) for decision-making.

Comment 4: Please respond to this comment.

As we see it, there is a profound enough disagreement between the reviewer and the authors about the general message and the interpretation of the results, but the reviewer does not point to a specific technical flaw in the statistical processing. Therefore, I would tend to accept publication (pending further minor revisions) if you can provide a response to the reviewer's comments.

Verbatim copy of reviewer #2 comments:

---


Section 2. I commented on this as a major correction and the authors have not changed it and merely defended the choice to give an undergraduate statistics lesson in a peer reviewed journal article. If climate statistics is as bad as they claim in their defence then that is a sad state of affairs. However, it is
my experience that many climate journals publish proper statistical analyses using Bayes or even more advanced theory. Examples can be found in the works of Berliner, Guttorp, Wilkinson and many more. If this paper must be published the reader’s time should not be wasted in the main text with this and it should go in
the appendix. It is worth adding that the objection that everyone is misinterpreting things is quite funny given that the basic stats class everyone who uses confidence intervals must have taken at some point would have gone to great lengths to explain why they are not probabilities during that introduction.
The “natural” approach to assume DT = \mu +/- \sigma means that the true value is \mu measured with Gaussian random noise is not natural in general. I am not convinced by their handful of examples. I could spend a long time ranting about this, but to be brief: imposing infinitely many probability statements on a
researcher that has reported an error bar is not justifiable, even if you really really want it to be. You can of course decide to proceed in a world where we imagine for the sake of argument that this is what is implied and go from there. But to say this is normally what is meant can’t be justified. When a numerical
analyst reports an estimate with error, they very much understand that they are not invoking a measure. They might call it uncertainty, but they are talking about a bound on a numerical error (that can be derived due to the properties of the procedure etc). It is my experience that errors in observational products are
very often dominated not by measurement error (which lends itself to the measurement error model in the paper if an author should choose), but by errors in assumptions and the methods used to bring the product together. The 1Sv error in the Rapid array observations for MOC is like this. Tiny observation error, but
aggregating into a product requires assumptions which induce one and that error is not computed probabilistically. Global temperature must be similar. Sure, we might justify any observation having a Gaussian error, but a product?? Why? There was an opportunity to soften the paper’s tone instead of implicating anyone
who reported an error with the infinite uncertainty specification of a Normal distribution and the authors could only bring themselves to go as far as a confidence that they are right in most cases. I cannot say I share their confidence.
The investigation of assumed priors given assumed likelihoods and then looking at the impact of different priors is not uninformative for the community. The authors could go further and use different likelihood interpretations, as these are just as arbitrary and subjective. If the message is “ if you are Bayesian
then think about your priors” then great. But I guess the message is more “everyone is acting like Bayesians with this model and these priors and that seems silly to us. Now everyone should be Bayesian and then justify their priors” (page 16 line 12). I am as Bayesian as they come and I think if researchers can be
convinced that it is the only philosophically sound intellectual position, then they should be Bayesian and then they should think about their priors instead of using default priors. However, you have to first understand and accept the subjectivist view of uncertainty and to state that the measurement model represents
a full joint probability distribution over the data given the parameters (for you). Many and arguably most would choose not to do this. I’ve mentioned alternative views of the measurement model above and there are many other intellectually defensible and more mainstream views of the meaning of uncertainty statements.
The paper and conclusions could easily have been softened to say “under the subjective Bayesian view, this would mean this and that would mean that” without then adding “it is our experience that almost everyone does mean this”, and without even offering any of the alternative positions.
If we are being truly subjective, it seems hard to imagine that any researcher could truly hold a Cauchy prior. The huge tails actually make them less natural for things we can think about, such as a temperature change, as they assign much more probability to huge physically impossible changes as a distribution
like the normal. It would not be hard for most Cauchy priors to find implied subjective probability statements that no researcher could agree with just from the cdf of the prior predictive distribution. This would not be a problem for an objectivist, but of course it is against the message of this paper. Given the
strong message the authors wish to impart regarding careful consideration of the implications of their prior, the prior predictive should have been explored in more detail.


In summary, the authors have barely moved an inch from the first edition and my view of the paper has done similar.

---- end of verbatim copy.

Dear authors,

Thank you for your detailed response to the reviewer comments. Although we are unlikely to reach a total agreement with the reviewer #2, I believed you have provided enough evidence that you have taken these comments into account and considered them seriously. History will decide who has the final word on this, but at this point the paper should be published.