Designing and developement a laser imaging system with depth measurement and image defogging capabilities
Subject Areas : Journal of Radar and Optical Remote Sensing and GISAli Faraji 1 * , Abbas Bashiri 2 , Mehdi Nasiri 3
1 - Master's degree student in Electrical Engineering, Electronic Warfare, Imam Hossein University, Tehran, Iran
2 - Instructor of electronic, department of Information and communication technology, Imam Hossein University, Tehran, Iran
3 - Associate Professor, Faculty of Microelectronics, Imam Hossein University, Tehran, Iran
Keywords: distance measurement in fog, stereo imaging in fog, removal of fog effects, 3D imaging,
Abstract :
Most smart and unmanned aerial vehicles use optical imagers for imaging and distance measurement. But in the foggy environment, the quality of the images taken by these systems is not good enough and there is even a possibility of destroying the images. Because the light gets scattered in contact with water vapors and fog and destroys the image recorded in the imager. Therefore, image processing is very important in these systems, but in heavy fog, distance measurement faces a serious problem. Other alternative methods are generally not economical or efficient. In this article, a new method is introduced for distance measurement and imaging on the sea surface. This method scans the environment by using two stereo imagers whose optical axis is parallel and a linear laser located on one of the cameras, and using trigonometric relations, the difference of light lines recorded in the imagers is calculated and the image and sample 3D is created from the environment. The analysis of the obtained results shows that the system is able to measure the distance of the environment with an error of less than one centimeter, and due to the type of arrangement of imagers and laser, it overcomes the effects of fog in the images with a much lower cost than other hardware.
Anwar, M. I., & Khosla, A. (2017). Vision enhancement through single image fog removal. Engineering Science and Technology, an International Journal, 20(3), 1075–1083. https://doi.org/10.1016/j.jestch.2016.11.015
John, J., & Wilscy, M. (2008). Enhancement of weather degraded video sequences using wavelet fusion. 2008 7th IEEE International Conference on Cybernetic Intelligent Systems, 1–6. https://doi.org/10.1109/UKRICIS.2008.4798926
Kokul, T., & Anparasy, S. (2020). Single Image Defogging using Depth Estimation and Scene-Specific Dark Channel Prior. 2020 20th International Conference on Advances in ICT for Emerging Regions (ICTer), 190–195. https://doi.org/10.1109/ICTer51097.2020.9325450
Li, Z., Tan, P., Tan, R. T., Zou, D., Zhou, S. Z., & Cheong, L.-F. (2015). Simultaneous video defogging and stereo reconstruction. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4988–4997. https://doi.org/10.1109/CVPR.2015.7299133
Liu, N., Fu, Q., Guo, H., Wang, L., Tai, Y., Liu, Y., Liu, Z., Shi, H., Zhan, J., Zhang, S., & Liu, J. (2023). Multi-band polarization imaging and image processing in sea fog environment. Frontiers in Physics, 11. https://doi.org/10.3389/fphy.2023.1221472
Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J. B., & Zuiderveld, K. (1987). Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 39(3), 355–368. https://doi.org/10.1016/S0734-189X(87)80186-X
Salazar-Colores, S., Moya-Sanchez, E. U., Ramos-Arreguin, J.-M., Cabal-Yepez, E., Flores, G., & Cortes, U. (2020). Fast Single Image Defogging With Robust Sky Detection. IEEE Access, 8, 149176–149189. https://doi.org/10.1109/ACCESS.2020.3015724
Xu, Y., Wen, J., Fei, L., & Zhang, Z. (2016). Review of Video and Image Defogging Algorithms and Related Studies on Image Restoration and Enhancement. IEEE Access, 4, 165–188. https://doi.org/10.1109/ACCESS.2015.2511558
Yang, Y., Bai, S., Guo, Y., & Tang, J.-B. (2013). Video Fogging Hiding Algorithm Based on Fog Theory. 2013 Ninth International Conference on Computational Intelligence and Security, 503–507. https://doi.org/10.1109/CIS.2013.112
Yoon, I., Kim, S., Kim, D., Hayes, M., & Paik, J. (2012). Adaptive defogging with color correction in the HSV color space for consumer surveillance system. IEEE Transactions on Consumer Electronics, 58(1), 111–116. https://doi.org/10.1109/TCE.2012.6170062
Designing and Development a Laser Imaging System with Depth Measurement and Image Defogging Capabilities
Ali Faraji1 , Abbas Bashiri2 , Mahdi Nasiri3
1. Master's degree student in Electrical Engineering, Electronic Warfare, Imam Hossein University, Tehran, Iran
2. Instructor of electronic, department of Information and communication technology, Imam Hossein University, Tehran, Iran
3. Associate Professor, Faculty of Microelectronics, Imam Hossein University, Tehran, Iran
Article Info | ABSTRACT |
Article type: Research Article | Most smart and unmanned aerial vehicles use optical imagers for imaging and distance measurement. But in the foggy environment, the quality of the images taken by these systems is not good enough and there is even a possibility of destroying the images. Since the light gets scattered in contact with water vapors and fog, it destroys the image recorded in the image. Therefore, image processing is essential in these systems, but in heavy fog, distance measurement faces a serious problem. Other alternative methods are generally not economical or efficient. In this article, a new technique is introduced for distance measurement and imaging on the sea surface. This method scans the environment by using two stereo images. The imager’s optical axis is parallel and a linear laser is located on one of the cameras. The difference of light lines recorded in the imagers, using trigonometric relations, is calculated and the image and sample 3D are created from the environment. The analysis of the obtained results shows that the system can measure the distance of the environment with an error of less than one centimeter. Due to the type of arrangement of images and laser, the system overcomes the effects of fog at a much lower cost than other hardware. |
Article history: Received March 03, 2023 Accepted October 18, 2023 Published online December 06, 2023
| |
Keywords: Distance measurement, Stereo imaging in fog, Removal of fog effects, 3D imaging
| |
Cite this article: Ali Faraji, Abbas Bashiri, Mahdi Nasiri. (2023). Designing and Development a Laser Imaging System with Depth Measurement and Image Defogging Capabilities. Journal of Radar and Optical Remote Sensing and GIS, 6(4), pages 46.
Publisher: Yazd Campus (Ya.C.), Islamic Azad University | |
Introduction
Foggy weather often makes it difficult to capture clear images because water particles of significant size dominate the air. These particles not only scatter and absorb scene light but also scatter some atmospheric light toward the imager. Therefore, the images captured by the imager are degraded and usually have low contrast and poor visibility (Xu et al., 2016).
Fog occurs when there is a high concentration of water droplets in the air due to unfavorable weather conditions. These water droplets scatter sunlight and light reflected from other objects. Due to scattering, the contrast of an outdoor scene fades, and a white cast forms on the viewer's or photographer's side, which can result in low-quality images. The fog in an image mainly depends on the distance between the scene and the photographer. Therefore, estimating the depth map of a foggy image is critical to recovering the fog-free image (Anwar & Khosla, 2017). Defogging an image is challenging because the amount of fog depends on the depth, the details of which are unknown. If the input is only one image, depth map prediction is a challenging issue. Therefore, many deblurring approaches have used multiple images and additional information to recover details. However, without any prior information, real-time single-image fog removal is in demand for various real-world applications such as automated driving systems and surveillance systems (Kokul & Anparasy, 2020). Pizer and his colleagues proposed the contrast-limited adaptive graph equalization method for defuzzification. CLAHE limits the noise gain by establishing a maximum value. However, there are only a few works on color images using CLAHE (Pizer et al., 1987). John et al proposed a de-fogging algorithm based on video background and foreground separation. First, they used a physical model-based single-image dehazing algorithm to improve the background image and simultaneously obtain the global illumination parameter. Then, the estimated brightness parameter was used to provide the foreground image of each frame. Finally, the enhanced video was obtained by merging the background image and the enhanced foreground (John & Wilscy, 2008). Yun et al proposed an improved previous dark channel deblurring algorithm that uses a multi-phase surface set formulation method to replace the soft matte algorithm to recover each frame from the video and then proposed a color correction method to solve the color jump problem (Yoon et al., 2012). Yang et al. considered background image transmission as the global component for dehazing. They first extracted the background through the differential frame method. Then, they estimated the transition of the background image using the MSR algorithm and used the bilateral filter to optimize the transition. Finally, the global transition was applied to improve the next frame (Yang et al., 2013). Li et al. presented a method to estimate the scene depth using the stereo method and recover the blurred image. They jointly studied stereo vision and image deblurring problems and designed an algorithm that simultaneously estimates scene depth and deblurs input images. Its method is based on the fact that depth cues from stereo and fog thickness are complementary. Their method works best in scenes with dense fog when both near and far-depth cues are strong (Li et al., 2015). Salazar et al. have proposed a new method based on depth approximation via dark channel prior, local Shannon entropy, and fast-guided filter to reduce artifacts and improve image recovery in sky regions with low computation time. The focus is improving the processing time without losing recovery quality and avoiding image artifacts during image de-fogging. By analyzing images with different resolutions, the proposed method in this work shows the lowest processing time in similar software and hardware conditions (Salazar-Colores et al., 2020). Liu et al. have proposed multi-band polarization imaging to overcome single-band defects. In their study, sea fog and non-sea fog polarization imaging experiments are conducted in an indoor simulated environment and compared and analyzed by creating a synthetic simulation system to describe sea fog concentrations with different concentrations. Then, the polarization information of each waveband transformed by the Stokes component is fed to the two-dimensional discrete wavelet algorithm for image fusion processing. Compared to images by a conventional camera, polarized images of foggy scenes have a purer background, less noise, less distortion, and more distinct edge features (Liu et al., 2023). A foggy environment poses a significant challenge to the regular and effective operation of many imaging systems. Existing imaging equipment is subject to interference from the external environment. Also, the obtained images are often highly degraded, and mainly the information of the scene features is blurred, with low contrast and color distortion, which is inappropriate for imaging systems. Image extraction affects its analysis, perception, recognition, and other subsequent processing, which significantly reduces the performance of the visual system and limits the practical value of the image. Also, the main feature that fog has against rain and snow is its stability, which destroys images. This issue is challenging in sea surface imaging, and due to the low contrast of the sea surface, sky, ships, and ships, fewer systems can work properly. The purpose of overcoming the effects of water vapor and fog in the image is to remove the interference of weather factors of the degraded image and to increase the clarity and color saturation. In this way, the valuable features of the image can be recovered to the maximum extent so that it can be better used in many vision systems, such as remote sensing observation and automated driving. Therefore, it is of great practical importance to study how to effectively reconstruct the original clear image from the image captured in the foggy environment and improve the robustness of the vision system.
Materials and Methods
The main aim of the present article is to design an active 3D scanner with acceptable accuracy for measuring relatively far distances using tools consisting of ordinary digital imagers and an inexpensive linear laser along with appropriate algorithms. In this study, we used a simple and accurate algorithm for matching, which is based on comparing laser line displacement in two imagers, since matching is the most fundamental part of accuracy in the imaging process with two cameras. The accuracy obtained by this method is independent of the shape and color of the image and is within the pixel limit.
Also, the advantage of this method over stereoscopic scanners is its narrow-band imaging capability, which makes it possible for the scanner to take images and depth measurements with minimal processing and without using image processing algorithms in fog and water vapor conditions. In this article, a simple yet accurate 3D laser scanner is introduced using a computer, linear laser, imager, and turntable. Figure 1 shows the final system.
Figure 1. The laser scanner system built in this article
Figure 2 illustrates the stages of construction and analysis of this system.
Figure 2. Diagram of the stages of construction and commissioning of the proposed system
In this system, we used two similar CCD imagers with a resolution of 2 megapixels and an imaging rate of 24 frames per second, lenses with a focal length of 16 mm, and a green laser with a wavelength of 532 nm with a linear lens. The reason for not using a laser with a shorter wavelength and less light scattering was the high cost of lasers with a shorter wavelength. The study used a simple DC motor with a gearbox to make the rotating axis. Additionally, the study used Labview software for processing and calculations due to its high processing speed and fast adaptability to imaging systems. To image with a 3D scanner, the motor rotates at a constant speed in a completely dark environment. As the motor rotates, the system moves. We used the same methods as stereo imagers to adjust the imagers. To use the maximum depth measurement range, we set the imagers parallel to the optical axis. At this stage, we adjusted the images of two imagers with a subject at a distance of one kilometer or more, until both displayed a single image. In the next step, we put the laser on one of the imagers and adjust its light line perpendicular to the image. Then, by observing the laser line in the second imager and the trigonometric relations, we performed the distance measurement operation. We tested the system in an environment with very thick fog and at the end we checked the obtained results.
Results and Discussion
The first camera, on which the laser is located, always observes the laser line at a fixed point and records one line in each frame. By putting these lines together, a relatively clear image of the scene is recorded. Figure 3 shows the image created by the first camera by placing the recorded lines in each image frame.
Figure 3. Monospectral image made of the environment by the first camera
The second camera, located away from the laser is responsible for measuring depth and generating a 3D image. Since both cameras shoot simultaneously, every point captured by the first camera is recorded by the second. Figure 4 shows the image of a second camera. The line breaks when the laser light hits distant or nearer objects, which causes the depth measurement process.
Figure 4. The image recorded in one frame from the second camera
We calculated all three dimensions of the desired points by using The camera components and the relations governing the geometric pattern of the images and formed the matrix of super points.
Figure 5. The 3D image made by the second camera, whose scale is in centimeters
As illustrated in Figure 5, as the objects approach the system, their spectrum becomes closer to pink and as they move away, they become darker. In some parts of the scene, due to the reflection of light on transparent and shiny objects, the system has an error and imagines those areas close to the system. We used the obtained point cloud matrix and converted the environment into a 3D model. The points that are ahead of the wall and the table are system errors and are caused by the collision of laser light with objects with a shiny surface. In addition, the corner points and holes that cause the return light of the laser to remain unrecorded in the second camera, cause errors. Nevertheless, the number of errors is not considerable enough to prevent the recognition of the target dimensions. Figure 6 shows the 3D model.
Figure 6. 3D pattern made by the matrix of obtained points
After developing the 3D scanner system and designing the processing algorithm, fog formed in the laboratory environment during the imaging process. In the first camera, where the laser is located, due to very thick fog, the image is almost destroyed, making it impossible to see any details in the foggy areas. Figure 7 shows that the sampling line of the first camera is saturated in fog, and that line is destroyed.
Figure 7. The image of the first camera in conditions with fog (left) and without fog (right)
However, in the second camera, the image was hardly affected by fog, because the main issue with foggy imaging is Rayleigh scattering, which reflects a large part of the light in the same direction when the light hits the water droplets. However, since the second camera is not in line with the laser light, its image is not disrupted up to the range the system can measure. Furthermore, there is no need for complex image processing algorithms and heavy processing loads. This issue makes this method highly practical and economical compared to other methods. Figure 8 shows the image of the first imager, the presence of thick and heavy fog destroys the image and the image becomes completely saturated in areas where the fog is very intense.
Figure 8. Image captured by the first camera in heavy fog conditions. As it is known, thick fog has destroyed many parts of the image, which cannot be reconstructed with image processing algorithms.
Figure 9, shows the image made by the second camera, the details of the scene are clearly recorded and the halo effects and fog saturation are not visible.
Figure 9. The image recorded by the second camera in heavy fog conditions, which clearly records the details of the scene and has the least distortion and noise.
The test results revealed that this method has an advantage over the passive stereo scanner since we used inexpensive equipment and light processing algorithms. Additionally, it provides a good image and depth measurement in heavy fog and water vapor. This study used the distance matrix and presented a 3D model environment model.
Conclusion
In this article, we developed a system with two cameras and a linear laser for monospectral imaging and passing through fog and water vapor. We tested the system in an environment with very thick fog. Finally, we checked the results obtained in the proposed system. The results revealed that the objects in the recorded images are clearly defined in terms of size and shape in the proposed system. The fog has the least effect on the image. The results revealed that the presence of water vapor and fog caused the saturation of the laser light in the first camera. However, in the second camera, due to the angle it had with the light line, the scattering effects were less effective. This system has a high processing speed compared to image processing algorithms for de-fogging, which have a large processing load and are slow. Compared to other 3D laser scanning systems, this system is faster due to linear rather than point sampling, and the most important thing is the much lower price of the proposed system compared to other laser systems, which increases its usability.
References
Anwar, M. I., & Khosla, A. (2017). Vision enhancement through single image fog removal. Engineering Science and Technology, an International Journal, 20(3), 1075–1083. https://doi.org/10.1016/j.jestch.2016.11.015
John, J., & Wilscy, M. (2008). Enhancement of weather degraded video sequences using wavelet fusion. 2008 7th IEEE International Conference on Cybernetic Intelligent Systems, 1–6. https://doi.org/10.1109/UKRICIS.2008.4798926
Kokul, T., & Anparasy, S. (2020). Single Image Defogging using Depth Estimation and Scene-Specific Dark Channel Prior. 2020 20th International Conference on Advances in ICT for Emerging Regions (ICTer), 190–195. https://doi.org/10.1109/ICTer51097.2020.9325450
Li, Z., Tan, P., Tan, R. T., Zou, D., Zhou, S. Z., & Cheong, L.-F. (2015). Simultaneous video defogging and stereo reconstruction. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4988–4997. https://doi.org/10.1109/CVPR.2015.7299133
Liu, N., Fu, Q., Guo, H., Wang, L., Tai, Y., Liu, Y., Liu, Z., Shi, H., Zhan, J., Zhang, S., & Liu, J. (2023). Multi-band polarization imaging and image processing in sea fog environment. Frontiers in Physics, 11. https://doi.org/10.3389/fphy.2023.1221472
Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J. B., & Zuiderveld, K. (1987). Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 39(3), 355–368. https://doi.org/10.1016/S0734-189X(87)80186-X
Salazar-Colores, S., Moya-Sanchez, E. U., Ramos-Arreguin, J.-M., Cabal-Yepez, E., Flores, G., & Cortes, U. (2020). Fast Single Image Defogging With Robust Sky Detection. IEEE Access, 8, 149176–149189. https://doi.org/10.1109/ACCESS.2020.3015724
Xu, Y., Wen, J., Fei, L., & Zhang, Z. (2016). Review of Video and Image Defogging Algorithms and Related Studies on Image Restoration and Enhancement. IEEE Access, 4, 165–188. https://doi.org/10.1109/ACCESS.2015.2511558
Yang, Y., Bai, S., Guo, Y., & Tang, J.-B. (2013). Video Fogging Hiding Algorithm Based on Fog Theory. 2013 Ninth International Conference on Computational Intelligence and Security, 503–507. https://doi.org/10.1109/CIS.2013.112
Yoon, I., Kim, S., Kim, D., Hayes, M., & Paik, J. (2012). Adaptive defogging with color correction in the HSV color space for consumer surveillance system. IEEE Transactions on Consumer Electronics, 58(1), 111–116. https://doi.org/10.1109/TCE.2012.6170062