This dissertation contributes to two areas of statistics: volatility matrix estimation and high dimensional classification, in methodology and theory. Time- and state-domain methods are two common approaches for nonparametric estimation. Combining information in these two domains is an interesting challenge in statistics. This problem is surmounted in this thesis by dynamic integration of these two pieces of information from time and state domains. A data driven weighting strategy is proposed to optimally combine the time- and state-domain estimators, for both scalar and multivariate volatility estimation. In the multivariate case, a factor modeling strategy is proposed to attenuate the curse of dimensionality. By comparing their effciencies, it is demonstrated both theoretically and numerically that the newly proposed integrated estimator uniformly dominates the time- and state-domain estimators alone. For the second topic, determined efforts are devoted to understanding the impacts of dimensionality on classifications. In the presence of high dimensionality, classical methods such as Fisher discriminant do not perform well or even break down, and the independence rule has been proposed in the literature to overcome the difficulty. Studies in this thesis demonstrate that even for the independence rule, classification using all features can be as bad as random guessing due to noise accumulation in estimating parameters, and moreover that almost all linear discriminants can perform as bad as random guessing. It is therefore paramountly important to select a subset of important features, resulting in a new procedure, the Feature Annealed Independence Rules. The conditions under which all the important features can be selected by the two-sample t-statistic with probability one are established. The choice of the optimal number of features is given.