Abstract: Masonry buildings are prone to cracks, which seriously threaten the life of buildings and the safety of people's lives and property. Therefore, crack detection is an important basis for building maintenance. In order to improve the detection accuracy of masonry building cracks, the improved method of YOLOv5s is applied in this paper. The SPD-Conv was introduced into the backbone network to improve the detection ability of fine-grained features. The BiFPN combined Coordinate Attention module was used to replace the feature fusion network of YOLOv5, which improved the detection accuracy. SIoU Loss is used to replace the original loss function to improve poor detection in complex environments. Experimental results on the crack dataset of masonry buildings show that the proposed method has an average mean accuracy of 96.0% (mAP@0.5), which is 4.0 percentage points higher than that of the original YOLOv5s and 2.0 percentage points higher than that of the YOLOv8s proposed in 2023, and can effectively detect cracks in masonry buildings.