GRAINet: mapping grain size distributions in river beds  from UAV images with convolutional neural networks

Lang, Nico; Irniger, Andrea; Rozniak, Agnieszka; Hunziker, Roni; Wegner, Jan Dirk; Schindler, Konrad

doi:https://doi.org/10.5194/hess-25-2567-2021

Articles | Volume 25, issue 5

https://doi.org/10.5194/hess-25-2567-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-25-2567-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 25, issue 5

Research article

|

19 May 2021

Research article |

| 19 May 2021

GRAINet: mapping grain size distributions in river beds from UAV images with convolutional neural networks

Nico Lang, Andrea Irniger, Agnieszka Rozniak, Roni Hunziker, Jan Dirk Wegner, and Konrad Schindler

Download

Final revised paper (published on 19 May 2021)
Preprint (discussion started on 25 May 2020)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Referee comment', Anonymous Referee #1, 23 Jun 2020
- AC1: 'Answer to RC1', Nico Lang, 26 Aug 2020
RC2: 'GRAINet review', Patrice Carbonneau, 19 Jul 2020
- AC2: 'Answer to RC2', Nico Lang, 26 Aug 2020
RC3: 'Review of the paper by Lang et al.', Anonymous Referee #3, 01 Aug 2020
- RC4: 'Ground Truth in grain size measurement from imagery', Patrice Carbonneau, 03 Aug 2020
  - AC3: 'Answer to RC3', Nico Lang, 26 Aug 2020
- AC3: 'Answer to RC3', Nico Lang, 26 Aug 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (further review by editor and referees) (22 Oct 2020) by Matjaz Mikos

AR by Nico Lang on behalf of the Authors (25 Nov 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (27 Nov 2020) by Matjaz Mikos

RR by Patrice Carbonneau (20 Dec 2020)

Suggestions for revision or reasons for rejection

Review of GrainNet.

This revised manuscript has many improvements on issues like model architecture and training. Class acitivation maps also provide a very useful insight into how the model works. Overall, this method is very innovative and it produces results that are high quality and very difficult to achieve with other methods. However, the authors still do not clearly acknowledge the limitations of their method and this rests on 2 points: an unclear understanding of the logistical costs of acquiring sufficient data for GrainNet with a UAV and a false result for their so-called geographic cross validation.

First, the authors begin their response letter by stating that they do not see an issue with the acquisition of drone data at 0.25 cm of spatial resolution and characterise it as a ‘minor technical detail’. I will therefore clarify my comment with a worked example. Start with a 1 hectare (100x100 metre) bar a a unit sampling area. The project uses a DJI P4 pro, let’s simplify the problem by asuming a 90 degree FOV meaning that the image footprint is twice the flying altitude. The images were acquired at 16:9 aspect ratio with 5472x3648 pixels. From this we can derive that the drone was approximately 6.8 m above ground. Given that the method needs an orthomosaic, I will assume that the images are flown at 80% forward overlap with a 50% sidelap. The image height is 9.1 m. Between images the drone must move 20% of the image to get the 80% overlap. This is 1.82 meters. On the P4 pro and with the fastest SD card on the market, you need to leave abut 2s for the mage to write to disk, anything less and the drone will start missing images during the mission. So the optimal flight method is to get a slow continuous motion of the drone. 1.82 meters in 2s is 0.9 m/s. It will therefore take roughly 0.9 minutes to complete 1 line of 100m. With the images being 12m wide and a 50% sidelap, We need about 17 flight lines to cover 1 hectare. For a total flight time of about 15 minutes/hectare.

Now consider an alternative setup that is used to get grain size data for alternative texture mapping methods. In this case imagery acquired at 2-3 cm of spatial resolution is suitable. In this case, flying a P4 pro at 50 m altitude will deliver suitable imagery at about 2cm. At 50m, the footprint of 1 image is 100m x 56m. At 80 overlap and the same 2s interval between images, the drone flies at 5.6 m/s. Given the image width, we only need 2 lines to cover the hectare. Meaning that the total operation needs only 34 seconds/hectare.

Therefore, data acquisition for GrainNet requires drone operations that are 30x longer than for older methods. That is not trivial and readers deserve to know this fact.

The authors suggestion that magnification can solve the problem is incorrect. When you magnify you increase the focal length and thus reduce the image footprint, flight velocity for SfM acquisition remains the same. Whilst it is true that a higher resolution camera could indeed improve things, that trend is very slow. The current UAV market for science is now dominated by consumer, non-scientific, drones made by DJI. The simple reason is cost. The P4 pro resolution of 20 Mpix is already on the high side. The only way to improve the performance would be to use top of the line cameras that have high speed writing buses. For example, mounting a Canon EOS on a big drone like a Matrice 600 would indeed be much faster, but then you are talking of a 1 order magnitude increase in cost for drone equipment. Either way, the acquisition of appropriate data for GrainNet is a significant barrier to access.

The second issue is the geographic cross validation. My view is still that the authors approach is mistaken and unjustified in geomorphology literature. The authors state on line 564 that there is no strong correlation between grain sizes on the same rivers. This statement is not evidenced and it flies in the face of decades of fluvial geomorphology. It has long been known that grain size decreases exponentially with distance downstream with periodic discontinuities (Rice, 1999; Rice and Church, 1998, 2001). This was again observed in recent remote sensing studies(Carbonneau et al., 2005). So barring the incidence of a source of coarse grained material, two successive bars on the same river can be expected to have a similar grain size composed of similar material of the same source. So unless the authors can show that between each and every one of their sampling bars there is an new input of sediment, then we must expect that the majority of neighbouring bars in the dataset are similar and LOOCV is not an appropriate method. I again make the request that the authors revise this process to hold-out entire rivers.

This is critical because as it stands, this method does provide unprecedented data over a gravel surface, but as I show above, the logistic costs of data acquisition are an order of magnitude more in time or cost when compatred to older methods. If it turns out that the method does not generalise to new rivers, then local calibration will be needed at each acquisition thus increasing the total cost of the method. I do not doubt that in certain applications, such a large field effort will be justified in order to produce such high quality outputs, but the reader deserves to get a clear indication of these costs upfront.

Patrice Carbonneau
December 2020

Hide

RR by Anonymous Referee #3 (30 Dec 2020)

Suggestions for revision or reasons for rejection

Overall, I can see several improvements in this revised version of the manuscript. That said, I still have some concerns about this manuscript:

1. Introduction. I see small improvements in this Section. I think that the main points (suggests) in my previous review were not well addressed. For simplicity such points are reported below (lines refer to the previous version of the manuscript):

L 15-28. This part is not very useful. It would be more useful to focus on why grain size data are crucial (e.g. process understanding, modelling).
L 38-42. Reference to traditional approaches is very poor. I would avoid the reference to Fehr (1987), maybe a good reference in the German-speaking countries but not worldwide (and in an international journal). I would suggest to look and refer to classical works by Church, Bunte, and many others. For instance, a look to Bunte and Abt (2001, USDA) would be very useful to put this work in the general context of sediment sampling in gravel-bed rivers.
L 56. “….is more efficient than traditional field measurements…”: I would say that automatic grain size is much less time consuming but it is also, commonly, less accurate. This should be pointed out since it is probably not obvious for readers who are not familiar with sediment sampling.

2. Testing of the approach. L 606-607. “…Our CNN-based approach makes it possible to robustly estimate grain size distributions and characteristic mean diameters from raw images…”. Comparison with field data (real data) is weak in this work. Overall, the authors do not fully recognize that field sampling is more accurate that sampling from images. It seems to me that the final message of this work is as follows: sediment characterization from images is more reliable and accurate than characterization by field measurements. I do not think this is the case: it would be useful to clarify better advantages and limitations of both approaches (this would be very useful in the “Introduction”). Finally, I think that a sound test to assess the performance of the model was not carried out in this work: it should be relevant to point out this for future research development.

SPECIFIC COMMENTS

L 480-481. “…In order to calibrate numerical bedload transport models, a single representative dm-value of a gravel bar or a cross-section is essential…”: I am not so sure about this, could you better support and justify this statement?

Hide

ED: Reconsider after major revisions (further review by editor and referees) (31 Dec 2020) by Matjaz Mikos

After receiving two reviews, there is a clear need for further revisions to take into account the suggestions of the both reviewers. Comments and suggestions of Referee #3 is a bit easier to answer - but please, follow the suggestions to add (American) literature - I am also aware of the Fehr method (Anastasi), but Bunte, Abt, ... should be used and mention.
I would also agree with his remark on mean grain diameter dm-values and its significance for the calibration of numerical sediment transport models. I would rather think into the direction of fraction models using several sediment fractions to be able to incorporate building of an armour layer and the influence of selective transport mechanism. An advanced method for grain size determination should overcome the limitations by a mean grain size d-m models.
Why to apply a sophisticated method for d-m determination rather than to get a full GSD?
The comments from Referee#2 are more important. Please, give readers more details about the procedure and the equipment (UAV - we have applied Phantom DJI drones for rock fall applications and surface displacements on scree slopes), so that the reader would be aware about the possibilities. This is important, if such field campaigns are performed by non-experts (we ask geodetical engineers that have pilot certificates). A further clarification is needed in this regard.
Please, follow the suggestion and redo the geographical cross validation. This is a critical issue to be able to generalise the results. Are you sure that grain size distributions at a series of gravel bars of the same river are not inter-related? You should elaborate more on this issue (using classic papers from fluvial geomorphology on sediment sources and sediment links in a fluvial system). Please, compare results of the geographical cross validation using all data (this is already done) and only data from different rivers (skipping potentially cross-related data from neighbouring gravel bars on the same river). We should clarify this issue before going on with the publication process.

Hide

AR by Nico Lang on behalf of the Authors (05 Feb 2021) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (08 Feb 2021) by Matjaz Mikos

RR by Patrice Carbonneau (10 Mar 2021)

ED: Publish as is (25 Mar 2021) by Matjaz Mikos

AR by Nico Lang on behalf of the Authors (29 Mar 2021) Author's response Manuscript

Short summary

Grain size analysis is the key to understanding the sediment dynamics of river systems and is an important indicator for mitigating flood risk and preserving biodiversity in aquatic habitats. We propose GRAINet, a data-driven approach based on deep learning, to regress grain size distributions from georeferenced UAV images. This allows for a holistic analysis of entire gravel bars, resulting in robust grading curves and high-resolution maps of spatial grain size distribution at large scale.