Improving total phosphorus prediction and monitoring in data scarce streams using a hybrid machine learning framework

Phosphorus concentration largely (approximately 80 %) contributes to eutrophication, and its monitoring and accurate prediction are crucial for a healthy aquatic environment. However, data scarcity and inconsistent water quality parameters in time or space limit the prediction of phosphorus concentration. This study developed a hybrid explainable machine learning framework to predict total phosphorus (TP) concentration in data-scarce streams using different water quality parameters. We coupled the probabilistic principal component analysis (P2CA) and several machine learning (ML) models, including Gaussian Process Regression, Boosting Ensemble Learning, Support Vector Regression, and Bagging Ensemble Learning, to enhance the prediction accuracy of TP. We assessed the input feature importance and the global sensitivity of each input feature to identify their ability and functional relationship in the prediction of TP. Based on their importance score, we considered different scenarios of the feature set to estimate the TP concentration. We further conducted a residual and error analysis and compared their predictive capabilities using nine different statistical metrics. The findings indicate that the performance of the hybrid model P2CA-Boosting Ensemble Learning with correlation coefficient R=0.91 outperformed the other applied models, such as P2CA-SVR (R=0.86), P2CA-Bagging (R=0.81), and P2CA-GPR (R=0.84). We further validated this best-performing model setup on three different river catchments, using the same input features to assess its transferability. Results indicate that the combined approach of P2CA and Boosting Ensemble Learning is capable of accurately predicting TP (R≥0.81[jls-end-space/]) in data-scarce streams with constrained conventional water quality data, which enables more effective utilization of sparse water quality data and more informed and effective decisions on water resources management.