GENERAL COMMENTS

The study by Stoelzle and Stein is clearly written and carefully performed (e.g., elaborate and somewhat conservative classification scheme and cross-checks between reviewer team to reduce bias) and addresses an important topic from a new angle: This is the first time that a systematic review of unscientific rainbow colour maps is performed in an environmental journal covering almost a thousand papers in total and opens an important usage-over-time perspective with the previous review by Borland and Taylor (2007). This study is therefore useful not only to understand the current (mis)use of faulty colour maps and colour combinations, but also provides valuable information on the community-wide usage through time. Moreover, the study offers a valuable up-to-date overview over current resources needed for science-proof data visualisation.

Even though visualisation is a fundamental, and widely-used scientific method, the results found by the authors underline the lack of education in visualisation, and hence awareness, amongst scientists from an empirical standpoint. Even more dramatic is the finding that some editors and authors have disregarded expert review comments on figure accessibility and accuracy.

The authors have done a fine job to cover and refer to previous work on the topic and provide an insightful new angle. Given the widespread lack of understanding about the importance of colour in scientific data representation, and the relevance to the specific readership, I highly recommend this manuscript, after minor revision, for publication in Hydrology and Earth System Sciences.

SPECIFIC COMMENTS

Figure 6: The figure could be made more intuitive by clearly labelling the top row of panels too (which are currently not labelled). It should be clear (either by additional on-figure text or via the caption and added panel labels) that the ‚original‘ panel is the not suited version to improve, and the three panels to its right-hand side are options to make it suitable. So, I suggest to add (a), (b), (c), … labels to all panels or clarify the on-figure explanation, e.g., by „story-telling graph titles“.

lines 257-259: These points were also made in Crameri et al., 2020 (already in reference list of the current manuscript), to which the reader could be pointed to here as well.

lines 310-311: The reference for the actual Scientific colour maps is:

Crameri, F. (2018). Scientific colour maps. Zenodo. http://doi.org/10.5281/zenodo.1243862

lines 312-318: The authors might consider clarifying that there are two other potential data types/options (which however might be less common): multi-sequential (e.g., bathymetry + topography with a centric value but not diverging) and circular/cyclic (e.g., river orientation; with repeating colours for e.g., 0 and 360°) as outlined in Crameri et al., 2020.

line 320-321: This is a valid point to make.

line 322: Crameri et al. (2020) provides a handy flow chart to select a colour map based on the data to be visualised, which could be referred to here as well.

lines 343 and 391: I suggest to change: „Marie Curie“ to „Marie Skłodowska-Curie“

line 341: The reference for „Shephard et al., 2017“ should be:

Crameri, F. (2017), The Rainbow Colour Map (repeatedly) considered harmful, edited by G.E. Shephard, EGU-Geodynamics Blog, https://blogs.egu.eu/divisions/gd/2017/08/23/the-rainbow-colour-map/, last access: 13 January 2021 

or something like that, as it was written by myself and edited by Grace Shephard without contribution from others.

line 374: A good reference to back up this statement would be: 

Moreland, K. Why we use bad color maps and what you can do about it.

Electron. Imaging 2016, 1–6 (2016). 

who concludes that the widespread use of the rainbow is the main reason scientists (and others) propagate it further.

line 405: Not to leave out the other important aspect investigated here, the authors may rewrite to: „…to banish the rainbow color map, and simultaneous red and green usage, …“ or something along these lines.

TECHNICAL COMMENTS

lines 127-128: That sentence sounds unclear to me; consider clarifying. E.g., the term „vision deficiency scale“ sounds somewhat arbitrary.

line 132: If it is actually the case, consider clarifying that it means „all papers published in HESS“.

line 212: Consider informing the (potentially non-hydrologist) reader that the following suggestions are specific to/from the field of hydrology, as some of the used terms (e.g., ‚response surfaces’) are likely not familiar to readers from outside the discipline, and a potential source of confusion.

line 234: „point“ instead of „points“

line 318: Point D needs to be clarified grammatically.

lines 369-371: I do miss some critical commas throughout the manuscript. Here, for example, after „perspective“ and „With that“.

line 398: „alarming“ to „alarmingly“?

General comments

This paper tackles the question of color palettes, which is important for decodability of figures by individuals with color vision deficiency and more generally, for scientific interpretability.

The authors find that “36% of publications in HESS in 2020 had visualisations that were not scientifically correct, not perceptually uniform, and difficult to access for around 4% of the readership due to color vision deficiency”. I particularly like the way the authors evaluate reviewer comments, and highlight the lack of awareness among both reviewers and editors.

Overall, I fully support this paper. It raises an important issue; the analyses are novel and technically sound; the paper is well written, and it provides helpful solutions. Figure 7 is particularly useful as it highlights examples of “poor” and “good” color palettes clearly.  

Specific comments

• The paper focusses mainly on color vision deficiency (CVD) and people with low/reduced vision, however one might also argue that good visualisation and labelling is equally important for people with other cognitive differences, such as (I’m guessing) dyslexia. Has there been research on this? If so, this aspect might be worth including in your literature review.
• The discussion of color palette type (negative-to-positive, strictly negative, or strictly positive) comes a little late in the manuscript (Figure 7c-d). It might be worth describing the type of color gradient that is most suited for negative-to-zero, negative-to-positive, and zero-to-positive scales sooner; e.g. a red-white-blue palette, which is currently missing from Figure 1.
• It might be helpful to provide the readers with a “checklist” of items to verify when creating a readable scientific figure (e.g. “the data-ink ratio”; “a white mid-point at zero for negative-to-positive palettes”).
• Some repetition could be avoided, e.g. section 3.4 also contains some repetitions about CVD etc; perhaps it could be condensed a little.
• It was useful to read about the colorblind options in R packages. Are there similar options for Python users?
• I wondered if the paragraph about preprints (l.89-95) was really useful. It seemed to me this was a small sample compared with the analyses in subsequent paragraphs, so the utility wasn’t entirely clear.
• Lines 212-218 and elsewhere mention various types of visualisation (e.g. heatmaps at l.252), but it might be helpful to see examples (especially examples of good hydro-climatological visualisations).
• Finally, the title focusses on the hydrologic community but there were large parts of the text that were not specifically hydrological. Perhaps this could be strengthened a little. For example, Figure 1 could provide examples of hydroclimatic variables (i.e. highlighting which types of palettes are particularly suitable for specific variables).

Technical corrections

1.14 raise awareness how -> raise awareness of how

1.14 the rainbow color maps still is -> the rainbow color map still is

1.23 we sketch a way to improve the communication of rainbow flaws -> we outline an approach to ?? (unclear what is meant by ‘improve the communication of rainbow flaws’)

l.31 10 millions of unique -> 10 million unique

l.35 In terms of correct encoding, (comma needed for meaning)

l.36 “we are stronger in encoding..”: meaning could be clarified

l.42 “uses” -> “used”

l.47 the word “shares” is used instead of “percentages” (here and elsewhere); perhaps consider replacing for clarity

l.61 the term “perceptual uniform” needs to be explained. I would recommend replacing “perceptual uniform” with “perceptually uniform” throughout the paper. It is explained at line 309, but this comes too late.

l.87 notable -> notably

l.114 “a graph with two lines encoding continuous variables over time without any annotations… is classified as rainbow-related”: worth providing examples alongside A-D for clarity?

l.127 “a vision deficiency scale” – terminology could be clearer.

l.139 “two cross checks… led (not lead) to minor deviations”: if this information is included, then it might be worth specifying what “minor deviations” means and how many people are in the cross-checking and original reviewer teams.

l.149 It might be worth justifying the choice of journals – why were Sci Rep and NComms selected?

l.168 “a current redistribution of disappearing black and white papers into papers with and without color issues”. I think this means something like ‘coincidence between the decline of black and white papers and the emergence of papers with color issues’

l.186 less -> fewer

l.190 73-92% of how many? A little unclear why two numbers here.

l.193 four “suggestions” perhaps

l.233 ScientistS

l.237 “a pointedly use of” is unclear

l.246 “luminance” is unclear (also used elsewhere in manuscript). Does it mean transparency? Shading?

l.84 rise awareness -> raise

l.289 parts of science -> areas of science

Figures

Figure 1. “The same delta changes in values”: this could be rephrased for clarity; it is not entirely clear what the “+1” on the figure or in the caption refer to. Also, is “perceptual uniform” “perceptually uniform”?  

By this point in the manuscript (line 70), I think if would be helpful to distinguish the colors used for scales that range from negative to positive (e.g. “red white blue”) and those that are “strictly positive” or “strictly negative”.

Figure 2. I wonder if it might make more sense to show the % of red-green or rainbow color maps as a fraction of the total number of papers (instead of just the papers with color issues).

Figure 4 seems clear to me.

Figure 5 is a little unclear. I wonder if examples (of alternatives, properties, tools etc.) could be provided for clarity?

Figure 6.

Dark background – is this supposed to be easier or harder to read?

Panel c. is it brightness or shades?

Panel i. beyond the graph title, good labelling can also be helpful. Historically, many journals have discouraged the use of labels on figures; but for some people, clear panel/facet labelling can help greatly. Perhaps this is worth a mention.

Also worth making sure that all panels are referred to in the main text.

Figure 7.

OrangeRed and Batlow are almost too small to read; would recommend deletion.

Do you mean “white strokes decrease the data-ink-ratio” (rather than increase)?

P.S. Is the correct technical term “color map”, “color scale”, “color palette”, or “color gradient”?

This is a nice study by Stölzle and Stein which add to the growing guidance literature on how to best visualize data. Given that the earlier reviews already stressed several aspects, I’d like to suggest three aspects for the authors to consider. They are in no particular order:

[1] Maybe I missed this in the paper, but how many software packages (Matlab etc.) offer a rainbow colour scheme as the default? Do the authors simply use the default and not think about it? This would be my personal hypothesis based on my own past mistakes. If many software packages offer this as default scheme, then is there just a straight mapping of default schemes and schemes used? If so, then would the best strategy to approach the software producers to change their schemes (rather than focus on the users)? How much does the use of rainbow colour schemes correlate with the default colour scheme in the software used (do the authors have the data to calculate this)?

[2] The authors discuss in section 3.3. that tools like colorbrewer2.0 and others can be used to avoid issues for colour blind people. Tools like these offer a much wider help to avoid a wide range of colour issues discussed in this paper. Do the authors not think that a general use of such tools would avoid most errors they discuss? Basic use of such tools for all colour choices would solve most of the problem, why not suggest this as a standard? Would this be easier than a list of things that the scientist has to check separately?

[3] Point two leads me to my third point. The authors state at the end of their paper that “As a guide we presented manifold visual techniques…”. This is great for those highly motivated to do the right thing in terms of publishing visualizations, but there is a risk that this will be too much for many scientists. Is there a simpler step-wise guide the authors could propose? My personal strategy is to require all my students to use colorbrewer 2.0 to ensure that major errors are avoided, but maybe the authors could summarize their suggestions into a few key points?

Overall, this is a very nice and valuable paper and my suggestions do not require more than minor revisions to the actual manuscript. I personally think (and the authors do not have to share this opinion of course) that much is to do with laziness. Hence thinking about how we have a better starting point (change software defaults?) and what a simple strategy might look like that scientists can easily adopt (e.g. use existing tools as a standard) would be my suggestions to fight this.

Thorsten Wagener