Identification and Detection of Surface Defects of Outer Package Printed Matter Based on Machine Vision

Shuo Wang; Jiangfeng Xu; Fangzhou Wang; Shenshen Ruan

doi:10.7584/JKTAPPI.2020.04.52.2.3

Preview

Journal of Korea TAPPI. 30 April 2020. 3-11
https://doi.org/10.7584/JKTAPPI.2020.04.52.2.3

Identification and Detection of Surface Defects of Outer Package Printed Matter Based on Machine Vision

Shuo Wang¹^†

Jiangfeng Xu²

Fangzhou Wang¹

Shenshen Ruan¹

¹School of Information Engineering, Zhengzhou University, Zhengzhou, Henan, 450001, China, Student

²School of Information Engineering, Zhengzhou University, Zhengzhou, Henan, 450001, China, Professor

^†Corresponding author: E-Mail: shuow_ws@126.com (Address: School of Information Engineering, Zhengzhou University, No. 100, Kexue Avenue, Zhengzhou, Henan, 450001, China)

ABSTRACT

In the printing process, the surface of the outer package printed matter is prone to defects such as chromatic aberration and misting, leading to a defective product. It is necessary to take an effective method to identify and detect surface defects in printed matter. In this paper, the application of machine vision technology in the detection of defects in outer package was studied. Firstly, a machine vision image acquisition device based on charge coupled device (CCD) camera was designed; secondly, the obtained image was processed by graying the average method, then OTSU binarization was performed after denoising by median filtering, and finally an improved differential matching method was designed for defect detection. The example analysis showed that the clear defect image of the outer package could be obtained by using the method proposed in this paper. The detection rate of the method in detecting 2,000 images reached 99.4%, and the average detection time was 103 ms. In the detection of 5,000 images, the detection rate of the manual detection method was 81%, but the detection rate of the method proposed in this paper was 99.1%. The experimental results proved the reliability of the method and provided some theoretical basis for the further application of defect detection technology based on machine vision in the print industry, which was conducive to the good development of the print industry.

Keywords

Machine vision

defect detection

image processing

MAIN

1. Introduction

The outer package of various products is a very common item in people’s lives. Consumers’ demands for quality tend to increase over time. However, in the printing process, the outer package printed matters may have defects such as chromatic aberration, misting, scratches, stains, etc. due to insufficiency in terms of technology, equipment, environment or other aspects.¹⁾ If these defects are not detected in time, defective products will be produced, resulting in waste of production resources and damaging the image and reputation of the company in case of circulation to the market. Therefore, in order to control the appearance of defective products, it is necessary to detect timely and effectively surface defects during the printing process to avoid the occurrence of unqualified products. With the development of automation technology, the emergence of defect detection methods based on machine vision has greatly improved the efficiency of defect detection,²⁾ which can acquire image of objects through cameras, optical lenses and other equipment and then achieve accurate identification of defects by processing the images. Defect detection technology based on machine vision has been widely applied and studied. Meng et al.³⁾ processed the image of the collected car hose through HALCON algorithm according to machine vision and then identified the hose. It was found through experiments that the method could accurately predict the surface defects of the hose. Wang et al.⁴⁾ combined with machine vision and sensor coupling, collected images by charge coupled device (CCD) camera and processed the images by VS2010 and Halcon library. The results showed that the method could successfully identify gears with surface defects and achieve gear automation defect detection, which had a high application value. Min et al.⁵⁾ designed a railway track surface defect detection method based on machine vision. Firstly, the image acquisition device equipped with light emitting diode (LED) light source was used to acquire the orbit image, then the image was optimized by the morphological process, and finally the defect feature was obtained by tracking the direction chain code. It was found through experiments that the detection speed of the method could reach 2 m/s, which effectively realized the real-time automatic detection of railway tracks. Wang et al.⁶⁾ studied the detection of raw silk defects. Firstly, an image acquisition system based on machine vision was developed, then defect segmentation and feature extraction were performed through threshold processing and morphological operations such as opening, hole fill up, and image subtraction, and finally defect types were identified by using back propagation (BP) neural network. The experiment proved that the method could accurately detect and identify raw silk defects. In this paper, a defect identification detection method for outer package printed matter based on machine vision was designed. The results of this study confirmed the possibility of applying machine vision technology for defect detection in the printing industry.

2. Materials and Methods

2.1 Detection of defects on the surface of outer package print

The traditional defect detection method mainly relies on manual detection, i.e., directly comparing printed matter and standard manuscript through human eyes. Manual detection has many disadvantages:

(1) The rate of missed detection is high. The long-term repetitive work of the human eye is prone to fatigue and missed inspection.

(2) The quality standards are different. Manual detection relies mainly on the subjective judgment of individuals.⁷⁾ Different people has different standards for defects, so the standards for qualified products are also different.

(3) Efficiency is low. The manual detection speed is slow, the resource consumption is large, and the detection efficiency is low. Due to the fast printing speed, manual inspection requires off-line inspection instead of performing defect detection directly on the machine, reducing production efficiency.

With the development of technology, the emergence of defect detection technology based on machine vision makes up for the various deficiencies of manual detection. It replaces human eyes with machines, collects the target image through computer, and then identifies the image defects by digitizing the image with high speed, precision and degree of automation. The technology has strong objectivity and lower cost compared to manual detection, which has been widely used in medical testing,^8,9) and parts testing,¹⁰⁾ etc.

2.2 Defect detection method based on machine vision

Machine vision defect detection first needs to obtain an image of the object to be inspected through some devices. The image acquisition device used in this paper was shown in Fig. 1.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f001.jpg

Fig. 1.

Image acquisition device.

CCD camera has been widely used in image acquisition of industrial inspection. In this paper, area array CCD camera (TXG12c, Baumer Company, Germany) was used, and its parameters were shown in Table 1. It had high sensitivity and could suppress overexposure. Its lens (M3514-MP2, Computar Company, Japan) were used for focusing the detected object on the sensor device of the camera, and its parameters were shown in Table 2. The lighting tool was LED (LDR-75SW2-LA1, CCS Company, Japan). The illumination mode was coaxial light sources illumination.

Table 1.

The parameters of TXG12c area array CCD camera

Sensor type	Sony ICX445
Resolution	1296×960 px
Color process filter	RGB Bayer Mosaic
Pixel size	3.75×3.75 μm
Transmission rate	1,000 Mbits/s

Table 2.

The parameters of Computar M3514-MP2 optical lens

Maximum imaging size	8.8×6.6 mm
Aperture range	F1.6-F16C
Minimum object distance	0.3 m
Control (aperture and focus)	Manual

2.3 Image processing and defect detection

2.3.1 Image graying

In the case of external packaging printing, most of them are multicolored, and it is difficult to analyze them in this state. In order to make it easier to detect defects on the printing area, it is necessary to first convert the multicolor image to gray. In this paper, the average value method was used to perform image graying. Firstly, the average values of three components R, G, and B of the image was calculated and then the obtained average was used to replace the original pixel, that is:

[1]

R = G = B = \frac{R + G + B}{3}

2.3.2 Median filtering denoising

The median filtering method is used to remove noise from the image of outer package printed matter. The principle of the method is: first, select a template consisting of several pixels from the image to be processed, measure the gray value of each pixel in order to get the media value, and then replace the original pixel gray value. It is assumed there is a noisy image, f(i,j), its pixel gray value is $\{x_{i j}, (i, j) \in I^{2}\}$ , and the selected template is S. The process of median filter denoising is:

[2]

g (i, j) \begin{array}{l} = M e d \{x_{i j}\} \\ = M e d (x_{(i + r), (j + s)}, (r, s) \in S, (i, j) \in I^{2}) \end{array}

where g(i,j) represents the image after denoising.

2.3.3 Image binarization

Image binarization can make an image of black and white, reduce the amount of data in the image, and highlight the target outline. In this paper, the maximum inter-class variance (OTSU) method¹¹⁾ was used to achieve binarization. OTSU method which was put forward by Nobuyuki Otsu in 1979 can realize image segmentation based on the idea that the variance between the target area and background area was maximum. It is assumed that there is an image I with a size of M×N, and the gray value of the pixel point (i,j) is expressed as f(i,j). The binarization process can be expressed as:

[3]

h (i, j) = \{\begin{array}{l} 0, f (i, j) \geq t \\ 255, f (i, j) < t \end{array}

where h(i,j) represents the gray value of point (i,j) after binarization, 0 for black and 255 for white, and t represents the segmentation threshold, which can divide the image into background and target. The optimal threshold is obtained by OTSU.

The gray level of image I is expressed as L, C={0,1,...,L-1}, the occurrence probability of gray level l is expressed as $ρ (l), ρ (l) = \frac{1}{M \times N} c o u n t (f (i, j) = l), f (i, j) \in I, l \in C$ , the target is C₀=(0,1,...,t), and the background is C₁=(t+1,t+2,...,L-1). The proportion of C₀ in the image is expressed as $w_{0} = ρ (C_{0}) = \sum_{l = 0}^{t} ρ_{t} = w_{t}$ , the average grayscale is $u_{0} = \sum_{l = 0}^{t} \frac{l ρ_{t}}{w_{0}} = \frac{u (t)}{w (t)}$ , where , $u (t) = \sum_{l = 0}^{t} l ρ_{l}$ and the variance is $σ_{0}^{2} = \sum_{l = 0}^{t} \frac{{(l - u_{0})}^{2} ρ_{l}}{w_{0}}$ . The proportion of C₁ in the image is $w_{1} = ρ (C_{1}) = \sum_{l = t + 1}^{L - 1} ρ_{t} = 1 - w_{t}$ , the average grayscale is $u_{1} = \sum_{l = t + 1}^{L - 1} \frac{l ρ_{l}}{w_{1}} = \frac{u (T) - u (t)}{1 - w (t)}$ , where $u_{T} = u (L - 1) = \sum_{l = 0}^{L - 1} l ρ_{l}$ , and the variance is $σ_{1}^{2} = \sum_{l = t + 1}^{L - 1} \frac{{(l - u_{1})}^{2} ρ_{l}}{w_{1}}$ .

The variance between classes can be expressed as $σ_{B}^{2} = w_{0} w_{1} {(u_{1} - u_{0})}^{2}$ , and the overall variance can be expressed as $σ_{T}^{2} = σ_{B}^{2} + w_{0} σ_{0}^{2} + w_{1} σ_{1}^{2}$ . The computational formula of the optimal threshold is $l = ζ \arg \max λ (t), t \in [0,1, \dots, L - 1]$ , where ϛ indicates the correction factor and $λ (t) = \frac{σ_{B}^{2}}{σ_{T}^{2}}$ indicates the judgment condition.

2.4 Defect detection

In this paper, the method of difference matching was used to detect the defect of the processed image, that was, the two images were subtracted to obtain the shadow image to find the difference between the images. It is assumed that the standard image is s(x,y), the image of the printed product to be inspected is w(x,y), and difference image is d(x,y). The calculation process of the difference matching is: d(x,y)=|w(x,y)-s(x,y)|.

The hard threshold and the soft threshold are introduced to solve the artifact problem in the shadow matching. It is assumed that the image after binarization is f(x,y), the hard threshold is f_thred(x,y), and the soft threshold is f_soft(x,y). The new threshold for the image can be expressed as:

[4]

f^{'} (x, y) = \{\begin{array}{l} g_{t h r e d} (x, y), g_{t h r e d} (x, y) \geq g_{s o f t} (x, y) \\ g_{s o f t} (x, y), g_{t h r e d} (x, y) < g_{s o f t} (x, y) \end{array}

Effective removal of image artifacts can be achieved by using new f' (x,y).

3. Results and Discussion

In order to verify the effectiveness, the method proposed in this paper needed to be verified by experiments. First, the defect detection was performed by taking an outer package printed matter as an example. The images were shown in Fig. 2, where the left side of the figure was the standard image, and the right side was the defect image.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f002.jpg

Fig. 2.

Standard and defect images.

It was found that the outer package printed matter was a color image. In order to achieve accurate detection of the defect, the average value of R, G, B values in the image was calculated by the average method firstly, and then the image was processed by graying, as shown in Fig. 3.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f003.jpg

Fig. 3.

The grayscale image.

Since the image collected by the CCD camera contained a certain amount of noise, which might affect the result of the defect detection, the image needs to be denoised, and the image processed by the median filtering method was shown in Fig. 4.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f004.jpg

Fig. 4.

The denoising image.

After obtaining the denoised image, in order to further reduce the amount of data in the image and highlight the defect information, the optimal threshold of binarization was obtained by using the OTSU method proposed in this paper, and then the binarization operation was performed, as shown in Fig. 5.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f005.jpg

Fig. 5.

The binarized image.

Finally, the improved difference matching method was used, and a clear defect image was obtained.

It could be found from Fig. 6 that the method proposed in this paper could accurately identify and detect the defects in the printed matter of the outer package. The image defect could be clearly found through the difference matching, thereby realizing the detection of the defect. To further analyze the effectiveness of the method, 2,000 sheets of printed matter were tested. The outer package printed matter was coated white board paper with a size of 787 mm×1,092 mm, printing gloss no lower than 88% and 15-28% ink absorption; its technical requirement satisfied GB-T 10335.4-2004. There were 500 standard images and 1,500 defective images. The test results were shown in Table 3.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f006.jpg

Fig. 6.

The defect image.

Table 3.

The test results of outer package printed matter

Number of samples	Number of false positives	Detection rate	Average detection time
2,000	12	99.4%	103 ms

From Table 3, it could be found that in the detection of 2,000 printed matter images using the method proposed in this study, the number of false positives was 12, the overall detection rate was 99.4%, and the average detection time was also short. It took only 103 ms to complete defect detection for the images, which could not be realized by traditional manual detection methods. From Fig. 7, i.e., the partial defect image detected, it could be found that the method can effectively identify defects, no matter how large or small, or thick or light, on the outer package printed matter, which proved the reliability of the method.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f007.jpg

Fig. 7.

Partial defects detected.

The difference between the traditional manual detection method and the method proposed in this paper was compared using samples of different sizes. The sample size was 10, 50, 100, 200, 500, 1,000, 2,000 and 5,000 respectively, and the proportion of standard images in each group of samples was 10%. The comparison of the detection rates of the two methods was shown in Fig. 8.

https://cdn.apub.kr/journalsite/sites/ktappi/2020-052-02/N0460520201/images/JKTAPPI_2020_v52n2_3_f008.jpg

Fig. 8.

Comparison of the detection rate.

It could be found from Fig. 8 that when the number of samples was small, the detection rates of manual detection and machine vision detection both reached 100%. As the number of samples increased, the detection rate of manual detection began to decrease. The larger the number of samples is, the lower the detection rate is. However, the detection rate is high in machine vision inspection. When the sample size is 5,000, the detection rate of manual detection is 81%, while the detection rate of machine vision detection is 99.1%, which proved the reliability of the detection method based on machine vision in this paper.

Machine vision technology which combines electronics, automatic control, artificial intelligence, image processing and other technologies¹²⁾ can solve the shortcomings of manual detection. It has a wide range of applications in industrial defect detection¹³⁾ for its high detection quality and high speed. In the production process of outer package printed matter, the appearance of defects is inevitable, therefore the detection of defects of printed matter is crucial to prevent it become defective products.

Clear defect images could be obtained through this method (Fig. 6). The average detection time of the method was very short, 103 ms, and the detection rate was high, reaching 99.4%. Compared with the traditional manual detection method, it was found that the advantage of the machine vision based detection method proposed in this paper was more obvious in the defects detection of large sample size. In the detection process of large sample, the detection rate of manual detection gradually declined due to fatigue and other reasons. In the detection of 5,000 samples, the detection rate of the manual detection dropped to 81%, but the detection rate of machine vision method changed little, and the accuracy of 99.1% was maintained, which proved the reliability of the method proposed in this paper.

4. Conclusions

This study analyzed the application of machine vision in the recognition and detection of defects of outer package printed matter and found that the designed method had short detection time and high detection accuracy in the defect detection of outer package printed matter. Compared with the traditional manual detection method, it had a higher detection rate and good performance in the detection of large samples. This study provided some theoretical support for the application of machine vision technology in the defect detection of outer package printed matter, which was conducive to the further development of detection technology of defects of printed matter.

Literature Cited

Xing, S. Y., Liu, Y. Q., and Zheng, Y. S., Research on the inspection and mark on the defects of printed matter based on template matching algorithm, Appl. Mech. Mater. 713- 715:377-380 (2015).

10.4028/www.scientific.net/AMM.713-715.377

Yuan, X. C., Wu, L. S., and Peng, Q., An improved Otsu method using the weighted object variance for defect detection, Appl. Surface Sci. 349:472-484 (2015).

10.1016/j.apsusc.2015.05.033

Meng, F., Ren, J., Wang, Q., and Zhang, T., Rubber hose surface defect detection system based on machine vision, IOP Conf. Ser.: Earth Environ. Sci. 108:022057 (2018).

10.1088/1755-1315/108/2/022057

Wang, Y., Wu, Z., Duan, X., Tong, J., Li, P., Chen, M., and Lin, Q., Design of gear defect detection system based on machine vision, IOP Conf. Ser.: Earth Environ. Sci. 108:022025 (2018).

10.1088/1755-1315/108/2/022025

Min, Y., Xiao, B., Dang, J., Yue, B., and Cheng, T., Real time detection system for rail surface defects based on machine vision, EURASIP J. Image and Video Process. 2018(1): 3 (2018).

10.1186/s13640-017-0241-y

Wang, C., Li, J., Chen, M., He, Z., and Zuo, B., The obtainment and recognition of raw silk defects based on machine vision and image analysis, J. Text. I. 107(3):316-326 (2016).

10.1080/00405000.2015.1031517

Yang, Y., Zhao, L., Wang, S., Cao, P., Liu, D., Li, L., Yan, L., Li, C., Xie, S., Li, Y., and Chen, Y., A high-resolution detecting system based on machine vision for defects on large aperture and super-smooth surface, Proc. SPIE-Int. Soc. Opt. Eng. 9446:94462N (2015).

10.1117/12.2181182

Wu, C., Wang, Y., He, Z., Zhang, H., and Zhou, X., Research on foreign insoluble particulate detection method for medicinal solution based on machine vision, Chin. J. Sci. Instrum. 36(7):1451-1461 (2015).

Li, X., Qiao, T., Pang, Y., Zhang, H., and Yan, G., A new machine vision real-time detection system for liquid impurities based on dynamic morphological characteristic analysis and machine learning, Measurement 124:130-137 (2018).

10.1016/j.measurement.2018.04.015

Hu, F., He, Z., Zhao, X., and Zhang, S., A novel vision-based mold monitoring system in an environment of intense vibration, Meas. Sci. Technol. 28(10):105906 (2017).

10.1088/1361-6501/aa8537

Ren, Y., He, P., Wang, H. L., Cen, Z. J., Feng, P., and Wei, B., Compressed sensing and Otsu’s method based binary CT image reconstruction technique in non-destructive detection, Nucl. Sci. Tech. 26(5):63-68 (2015).

Vodrahalli, K. and Bhowmik, A. K., 3D computer vision based on machine learning with deep neural networks: A review, J. Soc. Inf. Display 25(11):676-694 (2017).

10.1002/jsid.617

Zhang, J., Zhang, C., Guo, S., and Zhou, L., Research on tool wear detection based on machine vision in end milling process, Prod. Eng. 6(4-5):431-437 (2012).

10.1007/s11740-012-0395-5

Journal of Korea TAPPI ISSN:0253-3200(Print) 펄프종이기술

Preview

Identification and Detection of Surface Defects of Outer Package Printed Matter Based on Machine Vision

ABSTRACT

MAIN

Fig. 1.

Image acquisition device.

Table 1.

The parameters of TXG12c area array CCD camera

Table 2.

The parameters of Computar M3514-MP2 optical lens

[1]

[2]

[3]

[4]

Fig. 2.

Standard and defect images.

Fig. 3.

The grayscale image.

Fig. 4.

The denoising image.

Fig. 5.

The binarized image.

Fig. 6.

The defect image.

Table 3.

The test results of outer package printed matter

Fig. 7.

Partial defects detected.

Fig. 8.

Comparison of the detection rate.

Literature Cited