When does Dual Dimensionality Reduction perform better than Single Dimensionality Reduction?

Current experiments in many fields often generate large-dimensional datasets with multiple modalities (e.g., neural activity and animal behavior). Often the first step in understanding these experiments is finding correlations between different modalities, which requires dimensionality reduction (DR). We previously introduced the concept of Dual Dimensionality Reduction (DDR) approaches, which simultaneously compress both data modalities to maximize the covariation between their reduced descriptions. We argued that DDR requires significantly fewer data points to detect correlations than performing DR on each modality independently and then identifying relations between the reduced descriptions. Here we use a generative model of multivariate correlated data and linear dimensionality reduction approaches to carefully explore under which conditions DDR methods outperform independent approaches. We extend the argument to nonlinear reduction methods as well, using Deep Canonical Correlation Analysis as a nonlinear DDR and autoencoders for independent reduction of individual modalities. We believe that our analysis points to a general principle that DDR methods are often more data efficient in detecting weak correlations than their independent DR equivalents.