Skip to main content
SpringerLink
Log in

Generation and application of hyperspectral 3D plant models: methods and challenges

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Hyperspectral imaging sensors have been introduced for measuring the health status of plants. Recently, they also have been used for close-range sensing of plant canopies with a highly complex architecture. However, the complex geometry of plants and their interaction with the illumination setting severely affect the spectral information obtained. Furthermore, the spatial component of analysis results gain in importance as higher plants are represented by multiple plant organs as leaves, stems and seed pods. The combination of hyperspectral images and 3D point clouds is a promising approach to face these problems. We present the generation and application of hyperspectral 3D plant models as a new, interesting application field for computer vision with a variety of challenging tasks. We sum up a geometric calibration method for hyperspectral pushbroom cameras using a reference object for the combination of spectral and spatial information. Furthermore, we show exemplarily new calibration and analysis methods enabled by the hyperspectral 3D models in an experiment with sugar beet plants. An improved normalization, a comparison of image and 3D analysis and the density estimation of infected surface points underline some of the new capabilities gained using this new data type. Based on such hyperspectral 3D models the effects of plant geometry and sensor configuration can be quantified and modeled. In future, reflectance models can be used to remove or weaken the geometry-related effects in hyperspectral images and, therefore, have the potential to improve automated plant phenotyping significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bannehr, L., Luhmann, T., Piechel, J., Roelfs, T., Schmidt, A.: Extracting roof parameters and heat bridges over the city of Oldenburg from hyperspectral, thermal, and airborne laser scanning data. ISPRS Int Arch Photogramm Remote Sens Spat Inf Sci 3819, 17–22 (2011)

    Google Scholar 

  2. Bareth, G., Aasen, H., Bendig, J., Gnyp, M.L., Bolten, A., Jung, A., Michels, R., Soukkamäki, J.: Low-weight and UAV-based hyperspectral full-frame cameras for monitoring crops: spectral comparison with portable spectroradiometer measurements. Photogrammetrie-Fernerkundung-Geoinformation 2015(1), 69–79 (2015)

  3. Behmann, J., Mahlein, A.K., Paulus, S., Kuhlmann, H., Oerke, E.C., Plümer, L.: Generation and application of hyperspectral 3d plant models. In: Computer Vision-ECCV 2014 Workshops, pp. 117–130. Springer, Berlin (2014)

  4. Behmann, J., Mahlein, A.K., Paulus, S., Kuhlmann, H., Oerke, E.C., Plümer, L.: Calibration of hyperspectral close-range pushbroom cameras for plant phenotyping. ISPRS J Photogramm Remote Sens 106, 172–182 (2015)

    Article  Google Scholar 

  5. Behmann, J., Steinrücken, J., Plümer, L.: Detection of early plant stress responses in hyperspectral images. ISPRS J Photogramm Remote Sens 93, 98–111 (2014)

    Article  Google Scholar 

  6. Bellasio, C., Olejníčková, J., Tesa, R., Sebela, D., Nedbal, L.: Computer reconstruction of plant growth and chlorophyll fluorescence emission in three spatial dimensions. Sensors 12(1), 1052–71 (2012)

    Article  Google Scholar 

  7. Bergsträsser, S., Fanourakis, D., Schmittgen, S., Cendrero-Mateo, M.P., Jansen, M., Scharr, H., Rascher, U.: HyperART: non-invasive quantification of leaf traits using hyperspectral absorption–reflectance–transmittance imaging. Plant Methods 11, 1 (2015)

    Article  Google Scholar 

  8. Biskup, B., Scharr, H., Schurr, U., Rascher, U.: A stereo imaging system for measuring structural parameters of plant canopies. Plant Cell Environ. 30(10), 1299–1308 (2007)

    Article  Google Scholar 

  9. Bousquet, L., Lachérade, S., Jacquemoud, S., Moya, I.: Leaf BRDF measurements and model for specular and diffuse components differentiation. Remote Sens. Environ. 98(2–3), 201–211 (2005)

    Article  Google Scholar 

  10. Burkart, A., Aasen, H., Alonso, L., Menz, G., Bareth, G., Rascher, U.: Angular dependency of hyperspectral measurements over wheat characterized by a novel UAV based goniometer. Remote Sens. 7(1), 725–746 (2015)

    Article  Google Scholar 

  11. Comar, A., Baret, F., Viénot, F., Yan, L., de Solan, B.: Wheat leaf bidirectional reflectance measurements: description and quantification of the volume, specular and hot-spot scattering features. Remote Sens. Environ. 121, 26–35 (2012)

    Article  Google Scholar 

  12. Dana, K.J., Van Ginneken, B., Nayar, S.K., Koenderink, J.J.: Reflectance and texture of real-world surfaces. ACM Trans. Graph. (TOG) 18(1), 1–34 (1999)

    Article  Google Scholar 

  13. Dupuis, J., Kuhlmann, H.: High-precision surface inspection: uncertainty evaluation within an accuracy range of 15\(\mu \)m with triangulation-based laser line scanners. J. Appl. Geod. 8(2), 109–118 (2014)

    Google Scholar 

  14. Fiorani, F., Rascher, U., Jahnke, S., Schurr, U.: Imaging plants dynamics in heterogenic environments. Curr. Opin. Biotechnol. 23, 227–235 (2012)

    Article  Google Scholar 

  15. Grahn, H., Geladi, P.: Techniques and Applications of Hyperspectral Image Analysis. Wiley, New York (2007)

  16. Gupta, R., Hartley, R.I.: Linear pushbroom cameras. IEEE Trans. Pattern Anal. Mach. Intell. 19(9), 963–975 (1997)

    Article  Google Scholar 

  17. Haralick, B.M., Lee, C.N., Ottenberg, K., Nölle, M.: Review and analysis of solutions of the three point perspective pose estimation problem. Int. J. Comput. Vis. 13(3), 331–356 (1994)

    Article  Google Scholar 

  18. Hosoi, F., Nakabayashi, K., Omasa, K.: 3-D modeling of tomato canopies using a high-resolution portable scanning lidar for extracting structural information. Sensors 11(2), 2166–2174 (2011)

    Article  Google Scholar 

  19. Jacquemoud, S., Verhoef, W., Baret, F., Bacour, C., Zarco-Tejada, P.J., Asner, G.P., François, C., Ustin, S.L.: PROSPECT+SAIL models: a review of use for vegetation characterization. Remote Sens. Environ. 113(Supplement 1), S56–S66 (2009)

    Article  Google Scholar 

  20. Kim, Y., Glenn, D.M., Park, J., Ngugi, H.K., Lehman, B.L.: Hyperspectral image analysis for water stress detection of apple trees. Comput. Electron. Agric. 77(2), 155–160 (2011)

    Article  Google Scholar 

  21. Kuester, T., Spengler, D., Barczi, J.F., Segl, K., Hostert, P., Kaufmann, H.: Simulation of multitemporal and hyperspectral vegetation canopy bidirectional reflectance using detailed virtual 3-D canopy models. Geosci. Remote Sens. 52(4), 2096–2108 (2013)

    Article  Google Scholar 

  22. Liang, J., Zia, A., Zhou, J., Sirault, X.: 3D plant modelling via hyperspectral imaging. In: 2013 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 172–177 (2013)

  23. Mahlein, A.K., Oerke, E.C., Steiner, U., Dehne, H.W.: Recent advances in sensing plant diseases for precision crop protection. Eur. J. Plant Pathol. 133(1), 197–209 (2012)

    Article  Google Scholar 

  24. Mahlein, A.K., Steiner, U., Dehne, H.W., Oerke, E.C.: Spectral signatures of sugar beet leaves for the detection and differentiation of diseases. Precis. Agric. 11(4), 413–431 (2010)

    Article  Google Scholar 

  25. Mason, J.C., Handscomb, D.C.: Chebyshev Polynomials. CRC Press, Boca Raton (2002)

  26. Omasa, K., Hosoi, F., Konishi, A.: 3D Lidar imaging for detecting and understanding plant responses and canopy structure. J. Exp. Botany 58(4), 881–898 (2007)

    Article  Google Scholar 

  27. Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33(3), 1065–1076 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  28. Paulus, S., Behmann, J., Mahlein, A.K., Plümer, L., Kuhlmann, H.: Low-cost 3D systems—well suited tools for plant phenotyping. Sensors 14, 3001–3018 (2014)

    Article  Google Scholar 

  29. Paulus, S., Dupuis, J., Mahlein, A., Kuhlmann, H.: Surface feature based classification of plant organs from 3D laserscanned point clouds for plant phenotyping. BMC Bioinf. 14, 238–251 (2013)

    Article  Google Scholar 

  30. Paulus, S., Dupuis, J., Mahlein, A.K., Kuhlmann, H.: Surface feature based classification of plant organs from 3D laserscanned point clouds for plant phenotyping. BMC Bioinf. 14(1), 238–251 (2013)

    Article  Google Scholar 

  31. Paulus, S., Eichert, T., Goldbach, H.E., Kuhlmann, H.: Limits of active laser triangulation as an instrument for high precision plant imaging. Sensors 14(2), 2489–2509 (2014)

    Article  Google Scholar 

  32. Paulus, S., Schumann, H., Leon, J., Kuhlmann, H.: A high precision laser scanning system for capturing 3D plant architecture and analysing growth of cereal plants. Biosyst. Eng. 121, 1–11 (2014)

    Article  Google Scholar 

  33. Schöler, F., Steinhage, V.: Towards an automated 3D reconstruction of plant architecture. In: Proceedings of the 4th International Conference on Applications of Graph Transformations with Industrial Relevance, pp. 51–64. Springer, Berlin (2012)

  34. Tilly, N., Hoffmeister, D., Liang, H., Cao, Q., Liu, Y., Miao, Y., Bareth, G.: Evaluation of terrestrial laser scanning for rice growth monitoring. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, ISPRS Congress, Melbourne, Australia XXXIX, pp. 351–356 (2012)

  35. Vos, J., Evers, J., Buck-Sorlin, G., Andrieu, B., Chelle, M., De Visser, P.: Functional-structural plant modelling: a new versatile tool in crop science. J. Exp. Botany 61(8), 2101–2115 (2010)

    Article  Google Scholar 

  36. Wagner, B., Santini, S., Ingensand, H., Gärtner, H.: A tool to model 3D coarse-root development with annual resolution. Plant Soil 346(1–2), 79–96 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

The authors acknowledge the funding of the CROP.SENSe.net project in the context of Ziel 2-Programms NRW 2007–2013 “Regionale Wettbewerbsfähigkeit und Beschäftigung (EFRE)” by the Ministry for Innovation, Science and Research (MIWF) of the state North Rhine Westphalia (NRW) and European Union Funds for regional development (EFRE) (005-1103-0018) during the preparation of the manuscript.

Author information

Authors and Affiliations

  1. Institute of Geodesy and Geoinformation (IGG) - Geoinformation, University of Bonn, Bonn, Germany

    Jan Behmann & Lutz Plümer

  2. Institute for Crop Science and Resource Conservation (INRES)-Phytomedicine, University of Bonn, Bonn, Germany

    Anne-Katrin Mahlein & Erich-Christian Oerke

  3. Institute of Geodesy and Geoinformation (IGG) - Geodesy, University of Bonn, Bonn, Germany

    Stefan Paulus, Jan Dupuis & Heiner Kuhlmann

Authors
  1. Jan Behmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne-Katrin Mahlein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Paulus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan Dupuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heiner Kuhlmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Erich-Christian Oerke
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lutz Plümer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Behmann.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Behmann, J., Mahlein, AK., Paulus, S. et al. Generation and application of hyperspectral 3D plant models: methods and challenges. Machine Vision and Applications 27, 611–624 (2016). https://doi.org/10.1007/s00138-015-0716-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0716-8

Keywords

Search

Navigation