The covariance matrix tells you how data is spread. The eigenvectors of that matrix tell you the directions the data actually points. They are the same idea, two ways. Drag the points below and watch the green/pink axes track the data in real time.
Variance measures spread along an axis. Covariance generalizes that — it measures how two variables move together. Stack the pairwise covariances into a matrix and you have, for any cloud of points, a complete description of how that cloud is shaped.
Drop your own points, or use the presets to morph the distribution. The green arrow is the largest eigenvector (principal component). The pink arrow is the second. The dashed grey ellipse is the 1σ contour of the fitted Gaussian.
Here's the punchline: any covariance matrix can be written as Σ = R · S · S · R⁻¹, i.e. a rotation times a scale. That is exactly what eigendecomposition gives you, and it is exactly what an SVD of the data computes.
If you transform the data by V, you rotate it onto its own eigen-axes — i.e. you axis-align it. That is PCA.
Inside a point cloud — say from a stereo camera or a LiDAR scan — covariance over a local neighborhood gives you the surface's orientation. The smallest eigenvector is the surface normal. The two larger eigenvectors lie in the tangent plane.
This is the same trick used in plane fitting, ICP normals, primitive detection, and even DNN feature pooling. Once you see covariance as rotation × scale, every "fit a shape" algorithm starts looking the same.