Analyzing multivariate time series has been a very important topic in economics, finance, engineering, social and natural sciences. Compared to univariate models, the multivariate models better represent the dynamics and correlations of the component series. Many popular univariate models such as autoregressive conditional heteroscedasticity (ARCH), generalized ARCH (GARCH), and capital asset pricing model (CAPM), are all under investigations for the extension to their multivariate counterparts. However, when increasing the data dimension, the number of parameters in the multivariate model easily explodes. This brings in various issues such as unsatisfactory estimation efficiency, heavy computational burden, and poor model interpretability, and becomes the bottleneck of high-dimensional time series analysis. In an attempt to address the problem, this dissertation studies a regularization technique for high-dimensional time series by penalty, which simultaneously performs variable selection and parameter estimation.

The idea of regularization, including the shrinkage type of estimators, has a long history in statistics. Recent emergence of a large amount of high-dimensional data from various resources has given the old technique renewed attention. Several statisticians in the past decade have made significant contributions to the study of regularization technique in the new context. However, their works are mainly under the framework of independent observations and the extension to the time series settings had remained an unexplored area. This dissertation takes a step forward in filling the gap, and reconstructs several major theorems with regard to the regularization technique in the dependent settings. The established new procedure for analyzing high-dimensional time series data is general in the sense that it readily applies to a large class of stationary multivariate time series models. To demonstrate it, two chapters of the dissertation are dedicated to providing two examples, the first one being the sparse loading full-factor multivariate GARCH model and the second one being the sparse autoregressive model. The second example is extended in a following chapter, to a study of long-order AR approximation to autoregressive fractionally integrated moving average (ARFIMA) models.