Performance Evaluation of Classifiers Based on Rainfall-Related Data Collected by an Automatic Weather Station

Abstract

Precipitation forecasting is essential for the planning and management of water resources, affecting areas such as agriculture, energy generation, urban planning, and water conservation. In the case of agriculture, this forecasting enables the collection of accurate data on rainfall distribution and quantity, optimising water usage and increasing productivity. This study explores the application of different classical machine learning approaches to precipitation forecasting, using a pluviometric dataset from Tianguá (CE), encompassing stages such as pre-processing, data balancing, evaluation of nine different models, and local explanation of a model. The objective is to conduct a comparative analysis of the performance of algorithms such as Logistic Regression (LR), Naive Bayes (NB), Extra Tree (ET), Extreme Learning Machine (ELM), k-nearest neighbours (k-NN), Multilayer Perceptron (MLP), Random Forest (RF), Support Vector Machines (SVM), and an ensemble model. Additionally, the Local Interpretable Model-Agnostic Explanations (LIME) method was applied, aiming to identify the most suitable model and understand the attributes that most influence the classifications. This approach provides valuable insights for improving performance in future studies, such as the application of feature selection techniques. Among the tested models, the ensemble based on the majority vote of four algorithms demonstrated the best overall performance, with an accuracy of 85% and excellent balance in the F1-Score, making it a robust choice for applications requiring reliable predictions. Although the NB model showed lower accuracy (76%), it stood out for its high precision (91%), indicating that when it predicted rain, it was almost always correct, as well as demonstrating excellent sensitivity in detecting the absence of precipitation (recall of 88%). Other models recorded accuracy rates between 83% and 84%, making them viable alternatives depending on the specific needs of each application. When explaining the ET model using the LIME method, the analysis of the first ten predictions from the test set revealed that the most influential features for predicting rain included maximum and instantaneous humidity, as well as minimum and instantaneous temperature, while factors such as maximum temperature and minimum humidity were more relevant for predicting the absence of rain.