Optimization of Penalty Parameter in Penalized Nonlinear Canonical Correlation Analysis by using Cross-Validation

Abstract

There is Canonical Correlation Analysis (CCA) as a way to find a linear relationship between a pair of random vectors. However, CCA cannot find a nonlinear relationship between them since the method maximizes the correlation between linear combinations of the vectors. In order to find the nonlinear relationship, we convert the vectors through some known conversion functions like a kernel function. Then we find the nonlinear relationship in the original vectors through the conversion function. However, this method has a critical issue in that the maximized correlation sometimes becomes 1 even if there is no relationship between the random vectors. Some author proposed a penalized method with a penalty parameter that avoids this issue when the kernel functions are used for conversion. In this method, however, methods have not been proposed for optimizing the penalty and other hyper parameters in the conversion function, even though the results heavily depend on these parameters. In this study, we propose an optimization method for the penalty and other parameters, based on the simple cross-validation method.