the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 22 Sep 2021
A comparison of tools and techniques for stabilising unmanned aerial system (UAS) imagery for surface flow observations
Robert Ljubičić
Dariia Strelnikova
Matthew T. Perks
Anette Eltner
Salvador Peña-Haro
Alonso Pizarro
Silvano Fortunato Dal Sasso
Ulf Scherling
Pietro Vuono
Salvatore Manfreda
Download
- Final revised paper (published on 22 Sep 2021)
- Supplement to the final revised paper
- Preprint (discussion started on 12 Apr 2021)
Interactive discussion
Status: closed
-
RC1: 'Comment on hess-2021-112', Anonymous Referee #1, 05 May 2021
I have read and reviewed the manuscript titled “A comparison of tools and techniques for stabilising UAS imagery for surface flow observations” submitted to Hydrology and Earth System Sciences. I have written a long soliloquy on the debate about what term should be used for the U in UAS. It boils down to this – the journal has a right to set its standards, and if the authors disagree with those standards (which they could have known before submission) they should not have chosen to submit their paper to a journal they so vehemently disagree with. I also think there are several flaws in the logic of the linguist who defends the original term, and I think examples and evidence they use to support their argument are unsound in some ways. Read below for more context.
The text is well-written and I have only minor comments, which are included in the edited .pdf. In general I think the paper is very important and desperately needed given the proliferation of UAS-based hydraulic measurements. I recommend the paper be published basically as is, but I would like the authors to consider the comments I have made. I don’t need to see the paper returned with my comments formally addressed, but I think they could help the authors. The authors have produced quality work here, and I hate to see a dispute about a single word prevent the publication of this work.
On the “U” in UAS
On the topic of “unmanned” vs “uncrewed” or some similar work such as unoccupied (or indeed a term like “Remotely Piloted Aircraft System”: the authors agree that this would be a minor change but feel uncomfortable with it and think that keeping “unmanned” would lead to better consistency and visibility. There are many publications that refer to uncrewed, unoccupied, RPAS, or something different. I think the argument of visibility/recognizability and consistency is therefore not all that sound. Either the particular name/term does not matter – because many people have published with many names, and we can figure out what they are talking about regardless of the first term – or it has not been decided yet. For example, “Remotely Piloted Aircraft” has been published in titles going back to the 1980s (Tomlins, 1983). That being said, “unmanned” is probably the dominant term – and something I have published with. I understand and respect the opinion of the linguist on the author list, but I have heard dissenting opinions from other people I respect. After hearing that perspective from these individuals (who have more knowledge and experience about the issue than I), I would have changed my past use of the term.
While the language of the linguist is passionate, I have a few problems with their logic. I am not sure I follow the idea that changing the term is a “very minor change” (a statement signed by all authors), yet the linguist can also claim “I am convinced though that this path has only one possible end - the complete destruction of quality science.”. Is it a minor change or is it the destruction of science as we know it? I also find this statement troubling and confusing: “Tomorrow we may be demanded to declare that the Sun rotates around the Earth to avoid certain people feeling excluded or discouraged by the fact that they are not the centre of everything. I may be very upset with the results of Milgram experiment but my emotional personal reactions to these results do not make them untrue or less valuable”. First, there are many issues with the Milgram Experiment, and Milgram himself is very controversial. The experiment was not sound science. It is not the place to get into it here but using the Milgram Experiment as an example of something “true” that people might begin to disagree with is ridiculous as that experiment has very little to do with reproducible, “quality science”. Second – do the authors really think changing a few letters in a term has anything to do with people thinking they are the center of everything? The linguist is equating using (or preferring, or mandating) gender-neutral terms as akin to denial of basic science. I do not clearly see the connection, and this seems over-dramatic. Words are meant to be fluid and change – something the linguist should certainly understand better than I. Have changing words led to people becoming flat-earthers or science deniers? If the linguist is passionately arguing that this could happen, I think they need to provide evidence – stronger evidence than is provided.
I am more than willing to be proven wrong (and am not an expert on this in any way), but I am not aware of a scientific consensus (or indeed, individual studies) that show preference or mandating of gender-neutral terms has led to rejection of science. In fact, the linguist states “…2018 Brown University removed their paper on rapid-onset gender dysphoria when they heard “concerns that the conclusions of the study could … invalidate the perspectives of members of the transgender community.”” On the website they included, however, there are numerous explanations that the real concern about the Brown University paper was just as much about data and methodology. For example: “Given the concerns about research design and methods — not the controversial nature of the subject — the University decided to stop featuring this news story on its news site.” On the PLOS website with the formal notice of correction (https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0214157), it is stated that “The Materials and methods section was updated to include new information and more detailed descriptions about recruitment sites and to remove two figures due to copyright restrictions. Other than the addition of a few missing values in Table 13, the Results section is unchanged in the updated version of the article.”. The results were not changed. There are over 3000 words that detail the changes made to the article, which range from changing the quantitative analysis to addresses limitations and biases. I think reducing the cause for the paper’s change (not removal, as the linguist claims) to the idea that the study could invalidate the perspectives of the transgender community is dubious at best and is disingenuous at worst. My interpretation could be incorrect, but I hardly think that is an example of how the slippery slope of changing to a gender-neutral term will lead to the destruction or science and to people believing the Sun rotates around the Earth.
Again, I do not think the linguist is fundamentally “wrong”. I just think that their “slippery slope” argument is too dramatic and not backed up sufficiently with evidence. Do the authors honestly think that we will reach a dystopian point where papers are rejected due to lack of transgender citations? This is patently ridiculous – and I know it was meant to be an example of a ridiculous endpoint, but it reads as fearmongering. I understand that my point – that it does not seem like a big deal to change it to something more inclusive – can also be used to argue that leaving it as is will not hurt anyone. But ultimately, I do not see the “destruction of quality science” as a potential endpoint of inclusivity. No, I do not want to ban the word “man” and I understand the linguist’s arguments.
At the end of the day, if HESS has a policy that they do not want to use unmanned – and they do, as the linguist pointed out with a link to the EGU website (https://blogs.egu.eu/geolog/files/2021/01/Promoting-inclusive-language-v2.pdf) – and the authors feels so strongly and passionately that this policy will erode the very foundation of science…why did they submit a paper to HESS? Why would they publish in and support a journal they so passionately disagree with?
Science is never completely free of judgement, opinion, biases, politics…and that includes words. Words have baked-in meaning, and the idea that science is (or can/should be) left “pure” and untainted by other parts of society is not realistic. People decide what is funded, what is published, what is acceptable, what is not. I agree that whatever term is used is very minor. Would using “uncrewed” potentially make a tiny, minor increase in inclusivity? Sure, just as pouring a bottle of water into a lake increases its volume. Does removing the term “unmanned” reduce the quality of science? I do not think it does – at the very least the authors have not demonstrated that it does. If we all pour water into a lake for many years, we can raise the level. So why not do our part and try to increase inclusivity? If we change the meaning of numbers so that 2+2 = 5 because people are uncomfortable with the number 4, the result is the destruction of quality science. If we change a term that some perceive to limit inclusivity…I think science will go on. I believe it is possible to be inclusive without causing the end of science as we know it.
Reference
Tomlins, G. F. (1983). Some considerations in the design of low cost remotely-piloted aircraft for civil remote sensing applications. The Canadian Surveyor, 37(3), 157-167.
-
AC1: 'Reply on RC1', Robert Ljubicic, 03 Jun 2021
Answers to the comments from referee #1
We would like to sincerely thank the reviewer for their valuable suggestions and comments. In the following sections we have tried to address all of them and have made appropriate changes to the manuscript itself.
On the “U” in UAS
As the authors of the manuscript, we wholeheartedly support the motion for increased inclusivity in science. At the moment of the submission we were not all aware of the brochure that EGU had published (https://blogs.egu.eu/geolog/2021/01/13/accessibility-at-egu-promoting-inclusive-language-an-incomplete-guide-2/) to which the handling Editor had pointed us to. This brochure could not be found on the HESS website, and no mention of it was made in the HESS Guidelines for Authors (https://www.hydrology-and-earth-system-sciences.net/submission.html#english). The brochure also does not appear to mandate the use of such terms but rather encourages and suggests it – something that we, as a team, recognize and support. Hence, it could be argued that HESS, as a journal, does not mandate the use of such terms either and that they are not part of the journal’s standard. After the publication of the incomplete guide (v1) on February 2020 different papers have been published in HESS containing the term “unnamed”. Here are some examples:
- https://doi.org/10.5194/hess-2020-271
- https://doi.org/10.5194/hess-24-3643-2020
- https://doi.org/10.5194/hess-24-5173-2020
Decision to submit our work to HESS was made exclusively based on the basis of the exceptional scientific reputation of the journal. For all these reasons we have decided to focus primarily on the scientific content of the reviewer’s comments, leaving the decision on the linguistic issues in the hands of the Editor. However, we have included several alternative terms for “unmanned” in the Introduction section when it first appears, in order to suggest that there are other terms indicating the same technology.
Specific comments from the supplementary PDF
Responses to the comments made by the reviewer are prefixed with the term Authors in a new line. References to specific lines are with regards to the unrevised version of the manuscript. Text in [square parentheses and in orange] are inserted by the Authors for clarification where needed.
Line 40: “See my comment/response to the authors' preference for this term.”
Authors: We have written a substantial answer in the previous section of this document. In the revised manuscript we have added additional/alternative terms for “unmanned” in the parentheses.
---
Line 63: “You might also mention that generally one wants to maximize the viewing area, so often there is only the water surface in view (or just barely some non-moving parts). this complicates many things.”
Authors: We agree with the reviewer, and we have added the following sentence: “In order to maximize the amount of available information in the ROI (i.e., pixels per cm), the size of static areas in the image is often kept as low as possible (giving more space to the water surface) which also limits the applicability of 3D stabilisation methods.”
---
Line 64: “Can this be quantified? i.e. it would be great to provide an estimate, perhaps based on literature, how much horizontal movement there is in comparison to vertical movement or rotation. is it 10-1? 5-1? 100-1?”
Authors: As the ratio of horizontal to vertical movement of the UAV depends both on the technical specification of the aircraft, as well as the local conditions, such estimate would be difficult to substantiate. Looking at the manufacturer’s brochures, the horizontal positioning accuracy in hover mode appears to be around 3 times as much as the vertical positioning accuracy. Also, the amount of rotation (pitch, roll) is far more difficult to predict, and cannot be directly compared to the amount of translation due to different nature of the motion. However, additional sentence is added to clarify this issue: “For the aircrafts used in this investigation, manufacturer specifies that the positioning accuracy in hover mode is three times higher in vertical direction, when compared to the horizontal.”
---
Line 104: “Any comment on lens or perspective distortions? It might not be in the scope of the paper, but perhaps add a sentence that mentions how these are issues that will not be addressed in the current paper.”
Authors: We agree with the reviewer that this is worth mentioning. Two additional sentences are added to the paragraph: “The issues of camera calibration – estimation of internal camera parameters such as the focal length, optical centre position, radial and tangential distortion parameters – were not addressed in this research, but can be found in literature (Mathworks, 2021a). Images used in this investigation have had the camera distortion removed prior to the analyses.”
---
Line 171: “Might be confusing to use this term [interrogation area, IE], as the same term and abbreviation is used for pattern-matching regions using PIV. In fact this whole sentence is confusing - are you referencing looking for particle displacement to measure velocity? You refer to static-feature displacement to measure frame movement (which is very similar to measuring flow velocity) but this is potentially confusing.”
Authors: Indeed, most of the tools presented in this manuscript are mechanically similar to some of the image velocimetry methods, especially PIV and PTV. We intentionally used the same terms when describing the delineation of images into subregions containing static features (IA, SA), hoping that the readers with knowledge on velocimetry methods would be able to understand the mechanics of these stabilisation methods as well. However, as the reviewer pointed out, this may be confusing for some readers, so the following sentence was added later in the paragraph: “This approach to the estimation of the static feature displacements is very similar to some image velocimetry approaches (such as the PIV) which pattern-match the interrogation areas (IAs) from one frame to the broader search areas (SAs) in the following frame to estimate the displacement of tracer particles. Due to such mechanical similarity, we have used the same terms (interrogation and search area) when describing some stabilisation tools in this paper.”
---
Line 205: “This is probably not the place to mention it [SA update limitations of FFT-DCH tool], but what about when the apparent movement is different in each location? This has happened to me when the movement is 3D, which actually does occur frequently.”
Authors: This sentence was added by the author of the tool in question in order to immediately and truthfully recognise some of its current limitations, especially with regards to the statement in lines 174-175: “Once the new feature positions are estimated, the positions of the search areas are usually updated for the following image…”. This does mean that in cases with significant camera motion (such as in the Basento river case), this tool would not be able to consistently follow the motion of the static features across the entire video and to stabilise accordingly. The sentence in question is now expanded to properly explain our intentions: “At the time of the analysis, the available version of the tool did not allow the search areas to be updated with the positions of tracked features from the subsequent frames, which limits its applicability to those cases where the camera frames do not deviate significantly from the initial frame of the video.”
---
Line 313: “Why? yes the gimbals help, but careful – in windy conditions, for example, you can get a fair amount of pitch and roll.”
Authors: We acknowledge the comment of the reviewer and have hence added a sentence to the end of the paragraph: “In cases where pitch- and roll-type rotation of the camera can be identified (e.g., strong wind conditions), the use of perspective transformation method is necessary in order to ensure proper stabilisation.”
---
Line 319: “Any recommended tools for this?” [removal of the camera distortion]
Authors: Two appropriate references are added: (1) Matlab’s Camera Calibration Toolbox, and (2) Python’s OpenCV library.
---
Line 457: “It should say in the caption that this is for the Kolubara case.”
Authors: This is indeed the case, and the reference to Kolubara river case study is now added to the caption of the Table 2.
---
Line 578: “What does this indicate to you? something about the area, or something about the raw video? Might want to expand on this, if you haven't in the discussion.”
Authors: The exact cause of such results was not properly identified in the investigation. One of the explanations could correlate the lower contrast in those areas, compared to the other parts of the image, to the difficulty of identification of feature displacements inside of them. Other explanations, related to the flight conditions or camera itself would be unsubstantiated, and are probably better left out. An additional sentence is now added to the paragraph: “The exact cause of higher displacements of static features in these regions (compared to the features in other sections of the images) was not definitively identified in this study, and it could be related to both flight and/or ground conditions.”
---
Other minor changes have been made to improve readability as per reviewer’s suggestions.
-
AC1: 'Reply on RC1', Robert Ljubicic, 03 Jun 2021
-
RC2: 'Comment on hess-2021-112', Anonymous Referee #2, 14 May 2021
This manuscript summarizes an in-depth analysis of the impacts of image stabilization on particle image velocimetry, a tool that is increasingly used in the hydrologic sciences. The authors correctly point out that the effects of stabilization errors have not been examined systematically, so this study fills an important void in the literature. Several open source tools are evaluated in an objective manner and some useful new approaches for characterizing stabilization errors are presented, so I think this study provides a uesful contribution. I am generally supportive of the manuscript's eventual publication but would first like to see the authors address a handful of issues. Although the paper is well written overall, I have made a large number of in-line text edits to improve readability and English usage. These changes should not be difficult to incorporate. In addition, I have just a few more substantive comments:
- Line 215: By "structural" are you referring to shape, size, and orientation of features? Please clarify.
- Tables 2-4: It would be helpful to display these vectors on a background image to help orient the reader.
- Equation 3: What about an intercept term of the form + c at the end of this equation?
- Figures 11-14: Are the distributions summarized in these boxed plots over the frames in the sequence (i.e., over time)? Please clarify.
With attention to these issues, the paper could proceed toward publication.
-
AC2: 'Reply on RC2', Robert Ljubicic, 03 Jun 2021
Answers to the comments from referee #2
We would like to sincerely thank the reviewer for their valuable suggestions and comments. In the following sections we have tried to address all of them and have made appropriate changes to the manuscript itself.
Responses to the comments made by the reviewer are prefixed with the term Authors in a new line. References to specific lines are with regards to the unrevised version of the manuscript.
---
Line 90: “This reference is incomplete.”
Authors: We acknowledge the remark of the reviewer and have added three more papers of recent date which use IMU/gyro/control action data for the stabilisation of UAV video, or UAV itself:
- Auysakul et al., 2018 – which describes a hybrid KLT + IMU data approach,
- Hanning et al., 2011 – which describes approach for IMU data for stabilisation of videos with rolling shutter effects, and
- Stegagno et al., 2014 – which (while not aimed specifically at videos) describes a method of UAV stabilisation using IMU, visual, and control action information.
---
Line 215: “By "structural" are you referring to shape, size, and orientation of features? Please clarify.”
Authors: We agree with the reviewer that this term could be explained in more depth. When comparing two images (of the same size) shape, size and orientation will all have impact on the SSIM score. However, other effects such as image noise will have far lower impact on the SSIM score than with metrics such as MSE. We have added a brief explanation to the existing sentence: “… as it implicitly relies on the information on shape, size and orientation of features – structural information.”
---
Figure 3: “Scaling, not scalling.”
Authors: We thank the reviewer for pointing out this mistake. Text in the figure was changed accordingly.
---
Tables 2-4: “It would be helpful to display these vectors on a background image to help orient the reader.”
Authors: Presented vectors could indeed benefit from background image, and this is something that we discussed during manuscript preparation. However, readability issues quickly arose where vectors of lower intensities could not be perceived correctly with their directions. For those reasons we opted to omit the background. However, for the revised version we have prepared supplementary material – unaggregated velocity fields, which would allow for deeper insights into the results of our work.
---
Equation 3: “What about an intercept term of the form + c at the end of this equation?”
Authors: The intercept term in the polynomial expression was deliberately omitted by the authors in order to retain as much physical meaning as possible. Keeping the intercept term would indicate non-zero displacement even for RMSD=0. Furthermore, the difference in R2 values between the two relationships (with and without the intercept term) have been found to be negligent (up to order of 0.001). Intercept term values have been found between 0.1 and 0.2 for all verification points. However, the reviewer’s point is valid, and it is worth explaining in the text why the intercept term was not used. We have added the following sentence to the paragraph after Fig. 10: “The intercept term in the polynomial relationship in Eq. (3) was intentionally omitted in order to retain some physical meaning in the relationship and prevent Eq. (3) from indicating small displacements even when RMSD=0.”
---
Figures 11-14: “Are the distributions summarized in these boxed plots over the frames in the sequence (i.e., over time)? Please clarify.”
Authors: We thank the reviewer for pointing out this issue. Indeed, a small clarification may be required regarding these results. Boxplots presented in Figs. 11-14 contain information about all estimated displacements of verification points, either relative to their position in the first frame of the sequence (Figs. 11 and 13), or relative to their position in the previous frame (Figs. 12 and 14). Total number of points summarised by one column in the boxplot is equal to the number of frames in the sequence minus one. Lines 557 and 579 (in the previous version of the manuscript) have now been changed to read:
- “Figures 11 and 13 summarize all displacements of validation points across the entire frame sequence relative to their location in the initial frame ...”
- “Figures 12 and 14 summarise all displacements of validation points across the frame sequence relative to their position in the previous frame.”
We hope that this small correction will clarify the origin of the values presented in Figs. 11-14.
---
Line 652: “Missing a number here.”
Authors: The value in question was indeed accidentally omitted. This is rectified in the revised version of the manuscript.
---
Line 758: “Should this be table 6? I don't see a table 7.”
Authors: We thank the reviewer for pointing this out. The text should indeed point towards the Table 6, and this was rectified in the revised version of the manuscript.
---
Many minor linguistic corrections have been made as per reviewer’s suggestions.
Peer review completion
