Multimodal Remote Sensing Classification via Hierarchical Fusion of Hyperspectral Bands and LiDAR-Derived Features
Keywords:
multimodal remote sensing; hierarchical fusion; hyperspectral imaging; LiDAR; land cover classification; system architecture; robustness; fairness; policy governanceAbstract
The fusion of hyperspectral imagery (HSI) and Light Detection and Ranging (LiDAR) data has emerged as a critical paradigm for high-resolution land cover classification, yet existing approaches often treat multimodal integration as a straightforward concatenation of features, neglecting the structural and semantic hierarchies inherent in both modalities. This paper proposes a hierarchical fusion framework that systematically integrates spectral bands from hyperspectral sensors with geometric and elevation features derived from LiDAR point clouds, explicitly modeling the multi-scale dependencies between spatial, spectral, and structural information. The architecture comprises three fusion levels: early band-level alignment, intermediate feature-level aggregation, and late decision-level refinement, each governed by a context-aware gating mechanism that adaptively weights contributions from each modality based on local scene complexity. We analyze system-level trade-offs including computational load, sensor calibration requirements, transferability across geographic domains, and robustness to missing or noisy channels. Deployment considerations are discussed in the context of operational remote sensing platforms, emphasizing the need for scalable infrastructure that can handle terabytes of high-dimensional data while maintaining real-time classification latency. Furthermore, we examine fairness and policy implications, particularly the risks of biased classification in heterogeneous urban–rural landscapes and the governance challenges associated with open-access versus proprietary data sources. Extensive evaluation on benchmark datasets demonstrates that the hierarchical approach outperforms conventional late fusion and early fusion baselines by up to 7% in overall accuracy while reducing sensitivity to spectral misregistration. The findings underscore that hierarchical fusion not only improves classification fidelity but also provides a more interpretable and auditable decision pipeline, aligning with emerging standards for trustworthy autonomous earth observation systems.
References
1. Ghamisi, P., Höfle, B., & Zhu, X. X. (2017). Hyperspectral and LiDAR data fusion: A review. IEEE Geoscience and Remote Sensing Magazine, 5(4), 29–55.
2. Liao, W., Bellens, R., Pizurica, A., Philips, W., & Pižurica, V. (2015). Classification of hyperspectral and LiDAR data with coupled convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 53(8), 4549–4564.
3. Rasti, B., Hong, D., Hang, R., Ghamisi, P., Kang, X., Chanussot, J., & Benediktsson, J. A. (2020). Feature extraction for hyperspectral imagery: The evolution from shallow to deep. IEEE Geoscience and Remote Sensing Magazine, 8(4), 60–88.
4. Huang, X., & Zhang, L. (2013). An SVM ensemble approach combining spectral, structural, and semantic features for the classification of high-resolution remotely sensed imagery. IEEE Transactions on Geoscience and Remote Sensing, 51(1), 257–272.
5. Pedergnana, M., Marpu, P. R., Dalla Mura, M., Benediktsson, J. A., & Bruzzone, L. (2012). Classification of hyperspectral and LiDAR data using attribute profiles and support vector machines. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(5), 1362–1373.
6. Hong, D., Gao, L., Yokoya, N., Yao, J., Chanussot, J., Du, Q., & Zhang, B. (2020). More diverse means better: Multimodal deep learning meets remote sensing imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 59(5), 4340–4354.
7. Yang, J. X., Wang, J., Li, Z., Sui, C., Long, Z., & Zhou, J. (2025). HSLiNets: Evaluating Band Ordering Strategies in Hyperspectral and LiDAR Fusion. IEEE Geoscience and Remote Sensing Letters.
8. Li, J., Huang, X., & Gong, J. (2019). A multi-level fusion network for hyperspectral and LiDAR data classification. IEEE Transactions on Geoscience and Remote Sensing, 58(4), 2493–2507.
9. Audebert, N., Le Saux, B., & Lefèvre, S. (2018). Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks. ISPRS Journal of Photogrammetry and Remote Sensing, 140, 20–32.
10. Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017). Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4), 8–36.
11. Stumpf, A., Lachiche, N., Malet, J.-P., Kerle, N., & Puissant, A. (2014). Active learning in the spatial domain for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 52(5), 2492–2507.
12. Kempenaars, M., & van der Meer, F. (2016). Co-registration of hyperspectral and LiDAR data using mutual information. International Journal of Applied Earth Observation and Geoinformation, 52, 123–133.
13. Cheng, G., Han, J., & Lu, X. (2017). Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10), 1865–1883.
14. Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016). XNOR-Net: ImageNet classification using binary convolutional neural networks. European Conference on Computer Vision, 525–542.
15. European Commission. (2020). Data governance act. Official Journal of the European Union, L 220/1.
16. Debats, S. R., Luo, D., Estes, L. D., Fuchs, T. J., & Caylor, K. K. (2017). A generalized segmentation and classification framework for very high resolution satellite imagery of smallholder agriculture. Remote Sensing of Environment, 198, 1–14.
17. Cohen, J., & Vondrick, C. (2023). Uncertainty-aware flood mapping using multimodal remote sensing. Proceedings of the AAAI Conference on Artificial Intelligence, 37, 14102–14110.
18. Lobell, D. B., Di Tommaso, S., & Burney, J. A. (2020). A remote sensing perspective on fairness in agricultural technology adoption. Nature Food, 1(9), 530–532.
19. Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68.
20. Foody, G. M. (2002). Status of land cover classification accuracy assessment. Remote Sensing of Environment, 80(1), 185–201.
21. Tuia, D., Persello, C., & Bruzzone, L. (2016). Domain adaptation for the classification of remote sensing data: An overview of recent advances. IEEE Geoscience and Remote Sensing Magazine, 4(2), 41–57.
22. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645–3650.
23. Tollefson, J. (2022). Earth observation satellite proliferation raises concerns about data access and equity. Nature, 605(7910), 420–421.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Computer Science and Engineering Transactions
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
