Abstract: Deformation measurement is an important method in the monitoring of structural engineering safety problems. Most of the existing computer vision-based building displacement measurement methods are prone to large errors in scenes with variable lighting conditions. In view of this problem, by using the color channel information and local light information of the target, based on the traditional image matching methods, this paper proposes an adaptive correlation matching-based deformation and displacement detection method, for multiple light conditions in outdoor scenes. A deformation and displacement monitoring system was developed based on C++. Through fixed-point monitoring technology, the system acquired real-time images of bridges/buildings. By target detection and positioning technologies, the coordinates and displacement of targets were dynamically obtained, and converted into deformation data according to the hardware parameters, thereby realizing high-precision non-contact real-time measurement of bridges/buildings, which achieved all-weather monitoring for buildings and structures.