Abstract: Cracks are one of the most serious diseases of concrete bridges, which affect the safety of the whole bridge structure. In this paper, a novel Ghost-YOLO network is proposed to detect cracks of concrete structures in different environments. This network effectively combined the advantages of GhostNet and YOLOv4 networks, which could greatly reduce the network model parameters and improve the detection accuracy. In order to comprehensively evaluate the detection performance of the network, this paper constructed a large-scale dataset of concrete structure diseases in different environments, and successfully transferred the above water crack detection model to the underwater environment and outdoor practical engineering environment by using the transfer learning method. Through ablation experiments, it is found that the Ghost-YOLO network shows strong detection ability in different complex environments. Compared with state-of-the-art object detection networks, such as YOLOv4, Faster R-CNN, VFNet, YOLOF, etc.KG-*3, the results show that the Ghost-YOLO network has obvious advantages in accuracy and speed of crack detection.