Determination of Crop Yield Detection and Prediction With Multimodal Data Using Fusion Techniques

Abstract

Most often, soil, environmental, and agronomic data are used in pieces, hence limiting the performance of crop selection and yield forecasting. This study develops an integrated, fusion-based framework that combines soil image classification, environmental feature analysis, and input-driven yield prediction to support precise agricultural decision-making. The system consists of three major modules: (i) a CNN for the classification of soil images into their respective classes, such as Alluvial, Black, Red and Clay; (ii) a soft-voting ensemble model integrating SVM, Random Forest and XGBoost to generate crop suitability rankings based on agro-environmental attributes; and (iii) a Conv1D-LSTM hybrid model for predicting crop yield based on rainfall, fertilizer usage, cultivated area and crop type. These predictions are refined by the proposed fusion algorithm, which integrates environmental compatibility with soil-specific crop suitability. The proposed system performed well overall, with the classifier achieving 95.5% accuracy on the soil images. The results showed an overall accuracy of 99% across 22 crop classes, and the prediction errors were very low, with low MAE and RMSE. The integrated framework offers a practical data-driven tool for localized crop planning that enables farmers and other agricultural stakeholders to make informed decisions that optimize productivity and resource utilization. The proposed framework achieved a practical, scalable fusion strategy for integrating heterogeneous agricultural data sources into a unified decision support system. Despite the strong predictive performance, the study acknowledges limitations related to dataset size, particularly for soil image classification, and the lack of large-scale field validation. Additional statistical validation and ablation experiments were conducted to assess robustness and generalizability. The proposed framework demonstrates promising applicability for decision support. However, further validation of larger and geographically diverse datasets is required for broader deployment.