Defect detection and quantification on printed circuit board (PCB) assemblies are crucial for ensuring the quality of electronic products. X-ray inspection is commonly used to detect internal defects, in which high-resolution gray-scale images are captured to reveal the internal structure, particularly the defect features, of an assembled PCB. To detect defect features, conventional computer vision techniques require hand-crafted feature filters and usually have a narrow application scope. This study introduces an automated method for detecting voids using supervised machine learning. The method employs end-to-end segmentation models to identify pixels that belong to the void and integrated circuit (IC) regions in X-ray PCB images. The segmentation results are then used to compute the percentage of voids within each IC, which is a key measure of the quality of the inspected PCB. Accurate predictions with an intersection over union of over 0.95 have achieved. A challenge in this development is the large size of the images, which can reach to millions of pixels, and the small scale of defect features. Training a deep segmentation model for large-scale images using conventional methods requires a lot of memory. Downscaling high-resolution images result in the loss of defect features. In this study, a unique training method utilizing the local nature of the defects is proposed, which reduces memory consumption and speeds up the training process. Additionally, to overcome the limited training data, various augmentation methods are applied and transfer learning is employed to construct segmentation models for different types of PCBs.
PCB inspection is crucial to assuring that the product meets quality standards and operates as expected. During the production of assembled PCBs, various defects such as short/defected circuits, solder defects, bubbles and misplaced components may present. Among them, two defect features are of particular interest due to their great influence on the performance of the assembled PCBs. One of the features is the defect in conductive components. Defects in conductive components such as traces, pads and soldering, could degrade the performance of the electronic product or, worse still, lead to the malfunction of the product. Inspection methods such as in-circuit testing, flying probe testing and automated optical inspection can be used to detect defects in conductive components. The other default feature that deteriorates the quality of an electronic product is the void feature between the bonding pad of a PCB and its electronic components such as ICs. Since air has a lower thermal conductivity than that of solid structures, air trapped in voids will impair the cooling performance of the device, resulting in a high operation temperature, which degrades the performance of ICs and shortens their lifespan. Another deteriorating effect caused by voids is the reduced fatigue life of the solder layers. During thermal cycle/shock tests, cracks tend to initiate at the boundaries of voids and propagate inwards, which dramatically reduces the fatigue life of the solder layer. A metric to evaluate the quality of an assembled PCB is the percentage of void areas over an IC area. In general, the void percentage should be well controlled to be less than 30% in a regular reflow process, and 10% is desirable by using specific solder paste/flux with optimized process recipes. In the power electronics field, the industry standard is less than 2%. Hence, the quality of an electronic product is adversely related to the amount of air trapped in voids, and it is critical to have an accurate estimation about the area of voids.
A common inspection approach to detect voids in the field is the automatic X-ray inspection (AXI), in which high-resolution gray-scale images are captured to reveal the internal structure and the defect features of a PCB. The high resolution is necessary because the defect features are usually very small compared to the size of the entire PCB board. Fig. 1 shows an IC region in two X-ray PCB images of different resolutions. Fig. 1(a) is a portion of the high-resolution PCB image (400×400) while Fig. 1(b) is the corresponding low-resolution image (100×100), downscaled from the high-resolution image. Comparing with the high-resolution X-ray PCB image, the features in the low-resolution image are blurry, making it difficult to correctly identify the boundary of voids and thus to accurately predict the void percentage. With a high-resolution X-ray PCB image, the void percentage can be calculated by counting the number of pixels of the voids and the number of pixels of an IC. However, it would be time-consuming and labour-intensive to perform such a task manually. Automated inspection methods such as those based on computer vision should be used.
Computer vision (CV) aims to extract useful information from images using computers. There are different tasks in CV, for example, image classification determines the class of a given image, object detection detects and locates objects in a given image, and image segmentation performs pixel-wise classification on a given image. The task of void inspection in ICs on an X-ray PCB image belongs to object detection and image segmentation. First, ICs need to be detected and localized from an X-ray PCB image, which is an object detection task. Second, the void features need to be segmented at the pixel level, which is an image segmentation task. In traditional computer vision methods, object detection is typically achieved by processing the image with some hand-crafted feature filters and image segmentation is performed using computer vision algorithms such as k-mean clustering and superpixels. However, finding a set of hand-crafted feature filters that work well for a certain type of PCB design requires expert knowledge and is not trivial, let alone doing it for several different types of PCBs.
To address the limitations of traditional computer vision (CV) methods, this paper presents a study focusing on the development of deep learning-based approaches for automated inspection of assembled PCB quality. Specifically in this work, machine learning models are constructed to detect void percentages in each IC. To do so, both the void pixels and IC pixels need to be located. However, using two models to segment voids and detect ICs from an X-ray PCB image is inefficient because training multiple models requires more resources and the input image needs to be processed multiple times. In the proposed method, the task of detecting ICs is incorporated into a segmentation model so that both tasks can be completed with one model.
In the 2012 ImageNet Large Scale Visual Recognition Challenge, AlexNet, a machine learning model for image classification, outperformed the traditional computer vision methods based on hand-crafted features. This victory demonstrated the potential of deep learning approaches in solving computer vision problems and marked a turning point in the field. Since then, deep learning has become the state-of-the-art method in the field. One of the
Machine learning-based defect detection has been widely applied in various industries such as manufacturing, transportation, and construction. In the area of manufacturing of electronic products, such type of approaches has been used to monitor the quality of these products. Zhang et al. (2018) used CNN to classify conductive defects in PCB such as mouse bite, short circuit and open circuit. In their work, they used a small window of 128×128 pixels to slide over the
In this work, we propose to use a supervised machine learning approach to segment both void features and ICs from an X-ray PCB image. A segmentation model is first constructed and trained with a dataset containing PCBs of a specific design. A non-traditional training pipeline is devised to reduce the memory usage and the amount of training data in training a deep segmentation model with high-resolution image. To further reduce the amount of the training data and increase the robustness of the
To find out the best encoder-decoder pair, all 30 combinations of encoders and decoders are trained using dataset1. In this training, 20 PCB images are randomly selected from 30 annotated images in dataset1 and used to generate the training data. Among the rest of the annotated images, half images are used as the validation data and the other half annotated images are used for testing. To ensure the fairness of the training, all combinations are trained with the same training and validation
In this work, the training data only includes void features labelled within IC regions, which means the model is specifically trained to identify void features inside IC regions. Consequently, similar void features outside the IC regions may not be effectively recognized by the trained model. In an attempt to make the model more general and improve learning efficiency, we re-train the model by including void features outside the IC regions. Table 6 shows the best average validation IOU of
In this paper, an automated deep learning-based detection approach for voids in PCB assembly is presented. Various segmentation models are constructed by combining different encoders and decoders for detecting void features within ICs directly from X-ray PCB images. Among different segmentation models, the Resnet34- UnetPlusPlus pair performs the best on the two datasets corresponding to two different PCB designs considered in this work. A non-traditional training approach that allows the
Ho Yeung Ma: Formal analysis, Investigation, Methodology, Validation, Writing – original draft, Conceptualization. Minglu Xia: Data curation, Writing – review & editing. Ziyang Gao: Data curation, Funding acquisition, Writing – review & editing. Wenjing Ye: Conceptualization, Funding acquisition, Project administration, Supervision, Writing – review & editing.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
This work is supported in part by the Hong Kong Research Grants under Competitive Earmarked Research Grant No. 16206320 and the Project of Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone (HZQB-KCZYB-2020083). We would like to thank Professor Dan Xu for the discussion on this work and providing some suggestions on data annotation software.
