Parameter Estimation Methods in Multiple Linear Regression Analysis with  Autocorrelation and Heavy-Tailed Distributed Data

Katechan Jampachaisri; Khanittha Tinochai

doi:10.14456/nujst.2019.35

Authors

Katechan Jampachaisri katechanj@nu.ac.th
Khanittha Tinochai Department of Mathematics, Faculty of Science, Naresuan University, Phitsanulok

DOI:

https://doi.org/10.14456/nujst.2019.35

Keywords:

Autocorrelation, Heavy-tailed distribution, Maximum likelihood, Hierarchical Bayes

Abstract

Regression analysis is extensively used in a wide variety of fields, especially for predictive purpose. Its assumptions play a crucial role in parameter estimation. This paper focuses on parameter estimation in multiple linear regression when the assumptions are violated with simultaneous presence of autocorrelated random errors of AR(1) structure and heavy-tailed distribution, using hierarchical Bayes approach, which prior information about parameters, both noninformative and informative priors, is incorporated into the model. The result is also compared with frequently used method, maximum likelihood estimation, using the mean square error (MSE) as a criterion for comparison. The result reveals that hierarchical Bayes with informative priors outperform the maximum likelihood method, yielding the smallest MSEs for all sample sizes and correlation coefficients.

References

Beach, C. M., & MacKinnon, J. G. (1978). A Maxim Procedure for Regression with Autocorrelated Errors. Econometrical, 46(1), 51-58.

Farrell, S., & Ludwig, C. G. S. (2008). Bayesian and Maximum Likelihood Estimation of Hierarchical Response Time Models. The Psychonomic Society, 15(6), 1209-1217.

Fernandez, C., & Steel, M. F. J. (1999). Multivariate Student-t Regression Model: Pitfalls and Inference. Biometrika, 86(1), 53-167.

Gill, J. (2008). Bayesian Methods: A Social and Behavioral Sciences Approach(2nd ed.). USA: CRC.

Lange, K. L., Little, R. J. A. & Taylor, J. M. G. (1989). Robust Statistical Modeling Using the t Distribution. Journal of the American Statistical Association, 84, 881-896.

Li, Z., & Zhao, W. (2015). Robust Bayesian Regularized Estimation based on t Regression Model. Retrieved from http://dx.doi.org/10.1155/2015/989412.

Liu, J., & Dey, D. K. (2007). Hierarchical Overdispersed Poisson Model with Macrolevel Autocorrelation. Statistical Methodology, 4, 354–370.

Nadarajah, S., & Kotz, S. (2008). Estimations for the Multivariate t Distribution. Acta Applicandae Mathematicae, 102, 99-118.

Pires, A. M., & Rodrigues, I. M. (2007). Multiple Linear Regression with Some Correlated Errors: Classical and Robust Methods. Statistics in Medicine, 26, 2901–2918.

Press, S. J. (2005). Applied Multivariate Analysis: Using Bayesian and Frequentist Methods of Inference

(2nd ed.). Malabar, Florida, USA: Robert E. Krieger Publishing Company.

Rahman, A., & Khan, S. (2007). Prediction Distribution for Linear Regression Model with Multivariate

Student-t Errors under the Bayesian Approach. Proceedings in the 3rd International Conferences on Research and Education in Mathematics (ICREM3). 10-12 Apr 2007 (pp. 188-193). Kuala Lumpur: Malaysia.

Ravi, A., & Butar, F. (2010). An Insight into Heavy-Tailed Distribution. Journal of Mathematical Sciences & Education, 5(1), 1-15.

Roy, V., & Hobert, J. P. (2010). On Monte Carlo Methods for Bayesian Multivariate Regression Models with Heavy-Tailed Errors. Journal of Multivariate Analysis, 101, 1190–1202.

Tanizaki, H. (2003). On Regression Models with Autocorrelated Error: Small Sample Properties. International journal of pure and applied mathematics, 5(3), 247-252.

Zellner, A., & Ando, T. (2010). Bayesian and Non-Bayesian Analysis of the Seemingly Unrelated Regression Model with Student-t Errors and Its Application for Forecasting. International Journal of Forecasting, 26, 413–434.