Dear Authors,
I have now received reports from three referees and thank you for your detailed responses to the first two referees’ comments. The third referee was unable to upload their report online, and so I have copied it below.
Based on these three reviews and my own reading of the manuscript, I find that the paper is of scientific interest to HESS readership, but would benefit from some improvements, including discussion of limitations and advantages of the method, description of the procedure used to define the value functions, and clarification of the other issues raised by the reviewers. In addition, it would be helpful if you could address the points made in the third review, which is copied below.
Two referees have requested to review the paper again after corrections have been made, and I consider that to be a fair request. I would therefore like to invite you to upload a revised manuscript for further review by the referees, incorporating the proposed changes and additions, and making any other modifications where you see fit.
I look forward to receiving the revised manuscript.
With best regards,
Louise Slater


Report #3
Submitted by email on September 16th

This is a stimulating perspective on applying participatory multi-criteria decision-making (MCDM) to vulnerability assessment. However, in the vulnerability assessment field, participatory approaches is commonly used for the general public involvement in the decision making. In the field, what the authors are doing is usually acknowledged as “expert knowledge” or “expert-derived assessment”. Previous studies have produced expert-derived vulnerability assessment using simpler methods, such as scorecard to select, rank and weight the multiple criteria and variables.

To achieve validation the authors might want to consider the three main issues arising from the literature they mention:

- acceptation by the experts involved in the process. Whilst the paper is doing that, it is surprising to collect the feedback of only 22 from the initial 101 experts; it might be seen as some form of retrofitting. Likewise, relying on only 9 expert to build the index and only 5 to calibrate the utility functions might not be seen as the best choice and the authors are right to question the sensitivity of their model. An even better evidence would be actual adoption, ie the experts having enough confidence in the resulting model to actually use it in their decision making.
See for example Beccari, B. (2016). A comparative analysis of disaster risk, vulnerability and resilience composite indicators. PLoS currents, 8, https://dx.doi.org/10.1371%2Fcurrents.dis.453df025e34b682e9737f95070f9b970

- better performance than simpler models. The more is not always the better, and simpler processes might be more cost effective, which is specially important in emerging countries, such as in this case study. The authors might want to confront their results against non-expert derived indexes (the more common are the SOVI and SVI) as well as against simpler expert-derived assessments such as scorecards in order to demonstrate their marginal improvement.
See for example Emrich, C. T. (2005). Social vulnerability in US metropolitan areas: improvements in hazard vulnerability assessment. PhD diss., University of South Carolina.

- actual disaster outcomes are usually the main target for validation in the vulnerability assessment field. The authors might want to confront their results against actual outcomes data for their case study or other nearby Brazilian municipalities. Otherwise, their assessment is not robust enough for decision making: it's just asserting and rewarding the judgments that the experts and decision makers already have in that area.
See for example Bakkensen, L. A., Fox‐Lent, C., Read, L. K., & Linkov, I. (2017). Validating resilience and vulnerability indices in the context of natural disasters. Risk analysis, 37(5), 982-1004; Burton CG (2010). Social Vulnerability and Hurricane Impact Modeling. Natural Hazards, 11 (2): 58-68.

Last but not least, the temptation to merge the multidimensionality of vulnerability into a single metric is prevalent but increasingly criticized in the field. There is a frustrating large gap between the contextual complexity collected among experts and decision makers and the resulting unidimensional correlated metrics.
See for example Barnett, J., S. Lambert, and I. Fry. (2008). The hazards of indicators: Insights from the environmental vulnerability index. Annals of the Association of American Geographers 98 (1), 102-19; Rufat, S., Tate, E., Burton, C. G., & Maroof, A. S. (2015). Social vulnerability to floods: review of case studies and implications for measurement, International Journal of Disaster Risk Reduction, 14, 470-486.

Dear Authors,

The manuscript “Participatory flood vulnerability assessment: a multi-criteria approach” was re-reviewed by another expert in the field and by myself. We both find that the authors have made considerable efforts to respond to the first round of reviews, with detailed discussion of the findings and limitations, as was requested.

Overall, I find that this paper describes an interesting experiment in combining participatory techniques and multi-criteria decision making (MCDM) tools to assess the vulnerability to flooding. The authors’ responses and revisions are particularly thorough and the resulting manuscript is both interesting and well-written. I therefore agree with the last reviewer and am pleased to recommend publication of the revised manuscript in HESS as-is.

With best regards,

Louise Slater