Estimating the Link Function in a Partial Linear Single Index Model for Longitudinal Data using LSTM Neural Networks

Abstract

This paper investigates and compares the performance of two estimation approaches - Long Short-Term Memory (LSTM) networks and the semi-parametric Minimum Average Variance Estimation (SMAVE) method - for the Partial Linear Single Index Model (PLSIM) in the context of longitudinal data. The PLSIM combines linear and nonlinear components, offering modeling flexibility for complex data structures often encountered in repeated measurements. We conduct extensive simulations with varying sample sizes (N = 50, 100, 150) to evaluate the prediction accuracy of both methods in estimating the unknown link function. Evaluation metrics such as Mean Squared Error (MSE), bias, and coefficient of determination (𝑅²) are used to assess estimation quality. Results show that LSTM significantly outperforms SMAVE in estimating both the linear parameters and the nonlinear link function. The LSTM method consistently achieves lower MSE and bias values, as well as higher 𝑅² scores for both the model and the nonlinear function, highlighting its superior ability to capture temporal dependencies and complex nonlinear relationships in longitudinal data. In contrast, SMAVE's performance is more sensitive to bandwidth selection and sample size. These findings suggest that deep learning models like LSTM offer a powerful alternative to traditional semi-parametric methods in longitudinal data analysis.