Methods and Approaches for Mathematical Optimization of Hyperparameters of Machine Learning Models

Abstract

This paper investigates the mathematical optimization of machine learning hyperparameters in the context of early chronic kidney disease (CKD) detection. This study compares two high -performing ensemble classifiers - Random Forest and XGBoost - under five tuning strategies: default scikit-learn settings, Grid Search, Random Search, a Genetic Algorithm (GA), and Ant Colony Optimization (ACO). Logistic regression, k-nearest neighbors, support vector machines, decision trees, and Naive Bayes were evaluated as baseline comparators under default settings only; their tuning results are reported in the supplementary material. The main experimental comparison focuses on RF and XGBoost, for which full five-strategy benchmarks, sensitivity analysis, and runtime profiling are available. Grid Search and Random Search are shown to be impractical at scale: for Random Forest, Grid Search evaluated 432 configurations in 4.2 hours and Random Search evaluated 200 in 1.8 hours, neither reliably reaching high-sensitivity solutions. GA-optimized Random Forest and XGBoost reached AUC-ROC values of 0.963 and 0.968, respectively, while ACO-optimized variants achieved 0.969 and 0.975. Critically, sensitivity for the CKD class the clinically most important metric rose from 0.864 to 0.924 (ACO) for Random Forest. Paired Wilcoxon signed-rank tests confirmed that all GA/ACO improvements over Grid Search are statistically significant (p < 0.01). The results establish that evolutionary optimization offers a practically favorable accuracy -to-runtime trade-off for medical classification tasks where the cost of false negatives is high.