Deep Learning-based marine species detection and classification framework for biomonitoring in the Arctic fjords, Svalbard
Vol.13,No.2(2023)
The effects of ongoing climate change have caused a poleward shift in the distribution of species due to the rapidly rising water temperatures. This calls for an immediate need to assess and document the extent of climate change-driven animal migrations occurring in the Arctic waters. However, the extreme climatic conditions and the remoteness of the region makes biomonitoring tedious in the Arctic ecosystem. The present study puts forward a deep learning-based analysis of a large underwater video dataset that was captured from the Arctic region. The dataset was acquired using underwater cameras mounted on custom-made stainless-steel frames. The video footages were collected over a period of 26 days from the Kongsfjorden- Krossfjorden twin Arctic fjords in Svalbard, Norway. The collected data sets were used to train YOLO-based object detection framework (You Only Look Once) for an automated detection of the organisms. The YOLO model employed for the study was found to be very efficient in classifying the underwater images captured from the region. The object detection framework could detect images of Comb jelly, Echinoderm, Sea Anemone and Ulke (Shorthorn sculpin) from the underwater images. The model attained a superior value of Mean Average Precision (mAP), precision, and recall of 99.5%, 99.2%, and 97.4%, respectively.
Arctic; biodiversity; biomonitoring; deep learning; YOLO; climate change; Artificial Intelligence
Ardyna, M., Babin, M., Gosselin, M., Devred, E., Rainville, L. and Tremblay, J.-É. (2014): Recent Arctic Ocean sea ice loss triggers novel fall phytoplankton blooms. Geophysical Research Letters, 41: 6207-6212. doi: 10.1002/2014GL061047
Arrigo, K. R., Van Dijken, G. and Pabi, S. (2008): Impact of a shrinking Arctic ice cover on marine primary production. Geophysical Research Letters, 35: L19603. doi: 10.1029/2008GL035028
Banan, A., Nasiri, A. and Taheri-Garavand, A. (2020): Deep learning-based appearance features extraction for automated carp species identification. Aquacultural Engineering, 89: 102053.
Bicknell, A. W. J., Godley, B. J., Sheehan, E. V., Votier, S. C. and Witt, M. J. (2016): Camera technology for monitoring marine biodiversity and human impact. Frontiers in Ecology and the Environment, 14: 424-432.
Brand, M., Fischer, P. (2015): Species composition and abundance of the shallow water fish community of Kongsfjorden, Svalbard. Polar Biology, 39: 2155-2167. doi: 10.1007/s00300-016-2022-y
Boussarie, G., Bakker, J., Wangensteen, O. S., Mariani, S., Bonnin, L., Juhel, J. B., Kiszka, J. J., Kulbicki, M., Manel, S., Robbins, W. D., Vigliola, L. and Mouillot, D. (2018): Environmental DNA illuminates the dark diversity of sharks. Science Advances, 4(5): eaap9661. doi: 10.1126/sciadv.aap9661
Burton, A. C., Neilson, E., Moreira, D., Ladle, A., Steenweg, R., Fisher, J. T., Bayne, E. and Boutin, S. (2015): Wildlife camera trapping: A review and recommendations for linking surveys to ecological processes. Journal of Applied Ecology, 52: 675-685.
Deiner, K., Bik, H. M., Mächler, E., Seymour, M., Lacoursière-Roussel, A., Altermatt, F., Creer, S., Bista, I., Lodge, D. M., De Vere, N., Pfrender, M. E. and Bernatchez, L. (2017): Environmental DNA metabarcoding: Transforming how we survey animal and plant communities. Molecular Ecology, 26(21): 5872-5895. doi: 10.1111/mec.14350
Fossheim, M., Primicerio, R., Johannesen, E., Ingvaldsen, R. B., Aschan, M. M. and Dolgov, A.V. (2015): Recent warming leads to a rapid borealization of fish communities in the Arctic. Nature Climate Change, 5: 673-677. doi: 10.1038/nclimate2647
Frainer, A., Primicerio, R., Kortsch, S., Aune, M., Dolgov, A.V., Fossheim, M. and Aschan, M. M. (2017): Climate driven changes in functional biogeography of Arctic marine fish communities. Proceedings of the National Academy of Sciences of the United States of America, 114(46): 12202-12207. doi: 10.1073/pnas.1706080114
Fu, R., Li, B., Gao, Y. and Wang, P. (2018): Visualizing and analyzing convolution neural networks with gradient information. Neurocomputing, 293: 12-17.
Gordó-Vilaseca, C., Stephenson, F., Coll, M., Lavin, C. and Costello, M. J. (2023) Three decades of increasing fish biodiversity across the northeast Atlantic and the Arctic Ocean. Proceedings of the National Academy of Sciences of the United States of America, 120(4): e2120869120. doi: 10.1073/pnas.2120869120
Hentati-Sundberg, J., Olin, B. A., Reddy, S., Berglund, P., Svensson, E., Reddy, M., Kasarareni, S., Carlsen, A. A., Hanes, M., Kad, S. and Olsson, O. (2023): Seabird surveillance: combining CCTV and artificial intelligence for monitoring and research. Remote sensing in Ecology and Conservation, 9(4): 568-581. doi: 10.1002/rse2.329
Hoegh-Guldberg, O., Bruno, J. F. (2010) The impact of climate change on the world’s marine ecosystems. Science, 328: 1523-1528. doi: 10.1126/science.1189930
Isaksen, K., Nordli, Ø., Ivanov, B., Køltzow, M. A. Ø., Aaboe, S., Gjelten, H. M., Mezghani, A., Eastwood, S., Førland, E., Benestad, R. E., Hanssen-Bauer, I., Brækkan, R., Sviashchennikov, P., Demin, V., Revina, A. and Karandasheva, T. (2022): Exceptional warming over the Barents area. Scientific Reports, 12: 9371. doi: 10.1038/s41598-022-13568-5
Jalal, A., Salman, A., Mian, A., Shortis, M. and Shafait, F. (2020): Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics, 57: 101088.
Jiang, H., Learned-Miller, E. (2017): Face detection with the faster R-CNN. In: 12th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2017). IEEE, pp. 650–657.
Johson, L. (2010): The fearful symmetry of Arctic climate change: Accumulation by degradation. Environment and Planning D. Society and Space, 28(5): 828-847. doi: 10.1068/d9308
Ju, Z., Xue, Y. (2020): Fish species recognition using an improved AlexNet model. Optik, 223: 165499.
Kahru, M., Lee Z., Mitchell, B. G. and Nevison, C. D. (2016): Effects of sea ice cover on satellite-detected primary production in the Arctic Ocean. Biology Letters, 12(11): 20160223. doi: 10.1098/rsbl.2016.0223
Keith, D., Akçakaya, H. R., Butchart, S. H. M., Collen, B., Dulvy, N. K., Holmes, E. E., Hutchings, J. A., Keinath, D., Schwartz, M. K., Shelton, A. O. and Waples, R. S. (2015): Temporal correlations in population trends: conservation implications from time-series analysis of diverse animal taxa. Biological Conservation, 192: 247-257. doi: 10.1016/j.biocon.2015.09.021
Khan, S., Islam, N., Jan, Z., Din, I. U., Joel, J. and Rodrigues, P. C. (2019): A novel deep learning based framework for the detection and classification of breast cancer using transfer learning. Pattern Recognition Letters, 125: 1-6.
Knausgard, M. K., Wiklund, A., Sørdalen, T. K. and Halvorsen, K. T. (2022): Temperate fish detection and classification: a deep learning based approach. Applied Intelligence, 52: 6988-7001. doi: 10.1007/s10489-020-02154-9
Kurvits, T., Alfthan, B. and Mork, E. (2010): Conservation of Arctic flora and fauna. In: Arctic biodiversity trends 2010: Selected indicators of change. https://wedocs.unep.org/20.500.11822/8670.
Lacoursière-Roussel, A., Howland, K., Normandeau, E., Grey, K. E. and Archambault, P. (2018): eDNA metabarcoding as a new surveillance approach for coastal Arctic biodiversity. Ecology and Evolution, 8(16): 7763-7777. doi: 10.1002/ece3.4213
Lathifah, H. M, Novamizanti, L. and Rizal, S. (2020): Fast and accurate fish classification from underwater video using You Only Look Once. IOP Conference Series: Materials Science and Engineering, 982: 012003. doi: 10.1088/1757-899X/982/1/012003
Layton, K. K. S., Snelgrove, P. V. R., Dempson, J. B., Kess, T., Lehnert, S. J., Bentzen, P., Duffy, S. J., Messmer, A. M., Stanley, R. R. E., Dibacco, C. and Salisbury, S. J. (2021): Genomic evidence of past and future climate-linked loss in a migratory Arctic fish. Nature Climate Change, 11(2): 158-165.
Laidre, K. L., Stirling, I., Lowry, L. F., Wiig, Ø., Heide-Jørgensen, M. P. and Ferguson, S. H. (2008): Quantifying the sensitivity of Arctic marine mammals to climate – induced habitat change. Ecological Applications, 18(sp2): S97-S125.
Liu, S., Qi, L., Qin, H., Shi, J. and Jia, J. (2018): Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8759–8768. https://ieeexplore.ieee.org/document/8579011
Lu, S., Lu, Z. and Zhang, Y. (2019): Pathological brain detection based on AlexNet and transfer learning. Journal of Computer Science, 30: 41-47.
Mahmood, A., Bennamoun, M. A. S., Sohel, F. and Boussaid, F. (2020): ResFeats: Residual network based features for underwater image classification. Image and Vision Computing, 93: 225-236.
Mclaren, B. W., Langlois, T. J., Harvey, E. S., Shortland-Jones, H. and Stevens, R. (2015): A small no-take marine sanctuary provides consistent protection for small-bodied by-catch species, but not for large-bodied, high-risk species. Journal of Experimental Marine Biology and Ecology, 471: 153-163.
Mueter F. J., Reist, J. D., Majewski, A. R., Sawatzky, C. D., Christiansen, J. S. and Hedges, K. J. (2013): Marine fishes of the Arctic, In: Arctic Report Card: Update for 2013–Tracking Recent Environmental Changes. http://www.arctic.noaa.gov/reportcard
Pedersen, M., Haurum, J. B., Gade, R. and Moeslund, T. B. (2019): Detection of marine animals in a new underwater dataset with varying visibility. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 18–26).
Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016): You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 779–788, doi: 10.1109/CVPR.2016.91
Shortis, M., Abdo, E. H. D. (2016): A review of underwater stereo-image measurement for marine biology and ecology applications. In: Oceanography and Marine Biology. CRC Press, pp. 269–304.
Simonyan, K., Zisserman, A. (2014): Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409-1556.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D. and Erhan, D. (2015): Going deeper with convolutions. Proceedings with the IEEE Conference on Computer vision and Pattern recognition, pp. 1–9. doi: 10.1109/CVPR.2015.7298594
Tian, Q., Arbel, T. and Clark, J. J. (2018): Structured deep Fisher pruning for efficient facial trait classification. Image and Vision Computing, 77: 45-59.
Varpe, Ø., Daase, M. and Kristiansen,T. (2015): A fish-eye view on the new Arctic lightscape. ICES Journal of Marine Science, 72: 2532-2538. doi: 10.1093/icesjms/fsv129
Vinnikov, K. Y., Robock, A., Stouffer, R. J. Walsch, J. E., Parkinson, C. L., Cavalieri, D. J., Mitchell, J. F., Garrett, D. and Zakharov, V. F. (1999): Global warming and Northern hemisphere sea ice extent. Science, 286(5446): 1934-1937. doi: 10.1126/science.286.5446.193
Xu, W., Matzner, S. (2018): Underwater fish detection using deep learning for water power applications. In: 2018 International conference on computational science and computational intelligence (CSCI), pp. 313–318.
Web sources / Other sources
[1] Arctic Council (2002) Inari Declaration - on the occasion of the Third Ministerial Meeting of the Arctic Council, Inari, Finland, 2002. http://hdl.handle.net/11374/88 (Accessed on 03-Oct-2023).
[2] OpenCV Python library, https://github.com/opencv/opencv-python (Accessed on 21-Aug-2023).
[3] LabelImg, https://github.com/HumanSignal/labelImg (Accessed on 11-Aug-2023).
[4] TensorFlow, https://www.tensorflow.org (Accessed on 23-Aug-2023).
[5] Świeżewski, J. (2022): Counting nests of shags with YOLO to assess the wellbeing of the Antarctic ecosystem, https://appsilon.com/yolo-counting-nests-antarctic-birds/ (Accessed on 05-Oct-2023).
Copyright © 2024 Felix Mattathil Philip, Gipson Edappazham, Anupama Jims, Lakshmi Devi Prabhullachandran