Abstract: When driving on a rainy day, the images taken by the on-board camera will be obscured by raindrops in front of the lens or rain streaks in the air, which affects the accuracy of vehicle detection. In order to solve the problem, the idea of rain removal first and then detection is adopted and an image rain removal method based on improved generative adversarial networks (GAN) is proposed. In this method, the attention module was added to the generative network and one layer of convolution was added to the patch-GAN discriminative network to extract attention mask for local discrimination. While the rain removal effect was improved, the image details were also preserved. After the rain was removed from the image, the YOLOv4 algorithm was used for vehicle detection. Multiple data sets were used to comparison experiments of this method with others. The experiments show that this method was effective in rain removal and can effectively improve the accuracy of vehicle detection in rainy days.