In this dissertation, two different types of noisy matrix completion models are studied. The first one is a linear model in which the expected response is the summation of two matrices: a partially observed one and a hidden one. The partially observed matrix represents the effect from fully observed covariates and can be decomposed into the product of the covariate matrix and a coefficient matrix. The hidden matrix is assumed to be low-rank and thus can be considered as the product of latent factors and loadings in high-dimensional columns with a small rank. With random sub-Gaussian noise adding to the model, we proposed an iterative least square (LS) method to estimate the coefficients, factors, loadings in the context and thus the expected responses. This approach enjoys the low computational cost comparing to the iterative PCA method, and it is easy to implement. However, the statistical inference for the estimators becomes challenging because the orthogonal property does not hold during the iterations. We show that the estimators from the iterative LS method hold asymptotic properties in any finite number of iterations. The entry-wise estimators of the hidden matrix and the coefficients are guarantee to asymptotically follow normal distributions. Therefore, point-wise hypothesis testing can be conducted and confidence intervals can be constructed. In addition, a simultaneous testing procedure for the high-dimensional coefficient matrix is provided through Gaussian multiplier bootstrap. With this inferential procedure, we can further investigate the effects from the auxiliary covariates on the estimation and prediction of the missing values. The second model studied is a generalized linear model with binary responses. In this model, we also consider the same parameterization with an additional link function. The estimator is then acquired by minimizing the negative-log-likelihood with Frobenius norm regularization on the factor and loading matrices. We use iterative rerweighted least square (IRLS) algorthm to get the solution. Some propositions about the error bounds of the estimation under different metrics are stated and are tested through simulation.