Abstract: When manifolds are crossed by each other, many existing manifold clustering methods perform poorly because samples near manifold intersections are usually difficult to distinguish. To solve this problem, a distance metric that can simultaneously consider spatial distance between samples and local manifold difference of samples is proposed. Based on the above metric, the proposed algorithm can adaptively learn the weight of sample spatial distance and local manifold difference during learning, and assign larger weight when calculating distance for samples near manifold intersections to better distinguish samples near intersections. Experimental results on real datasets and object tracking datasets demonstrate that the proposed algorithm can achieve better clustering accuracy compared with existing multi-manifold clustering algorithms.