Deep-Learning-Based Real-Time Monitoring of Laser Keyhole Welding Processes

In Chapter 1, the background of laser keyhole welding and the introduction of deep learning were introduced. In Chapter 2, a data-driven deep learning model was first proposed for the prediction of laser beam absorption in full-penetration keyhole laser welding.

Introduction

Laser Keyhole Welding

Introduction of Laser Keyhole Welding
Predictions of Laser-Beam Absorptance
Observations of Keyhole Dynamics

Laser beam absorption inside a keyhole affects other physical processes that occurred during the laser keyhole welding processes. These simulation methods have well implemented the multiple reflections inside the keyhole to calculate the laser beam absorption.

Figure 1.1. The OM images of weld bead on top surface and cross-section of bead welded by laser (a) conduction and (b) keyhole mode welding

Deep Learning in Computer Vision

Introduction of Deep Learning in Computer Vision
Image Classification
Object Detection

Since object detection is a multi-task problem with classification and localization, the deep learning architecture in object detection consists of backbone network for extracting feature maps from input images and region of interest (RoI) for classification and bounding box regression [52] . Average precision (AP) is used to evaluate the accuracy of object detection models, and both classification and detection performance can be expressed as a single variable.

Figure 1.7. Representative examples of deep learning in computer vision fields. (a) Image recognition [37] and (b) Image generation [38]

Application of Deep-Learning-Based Monitoring in Laser Keyhole Welding

Deep-Learning-Based Predicting of Laser-Beam Absorptance in Full-penetration

Data Preparation for Deep Learning Model

First, the shape of a 3D keyhole for full penetration laser welding was constructed linearly from a series of openings at the top and bottom (the left figure in Figure 2.1(a)), and a ray-tracing method was applied to the constructed keyhole to calculate the absorption, which served as the ground truth for the deep-learning model. To validate the accuracy of the ray-tracing method, an electrodynamic simulation was performed on several keyholes using the FDTD method. In the input image (the left figure in Figure 2.1(b)), the red and green ellipses represent the upper and lower openings of the keyhole, respectively.

In this study, the divergence of the incident laser beam was taken into account using the method used in [5]. To validate the ray tracing method used in this study, FDTD simulation was performed for the 15 test locks shown in Figure 2.3. Figure 2.4 shows the FDTD and ray tracing simulation results (left: FDTD; right: ray tracing) with their calculated absorption values.

In each figure, the FDTD results are shown on the left and the ray tracing result is shown on the right (red: incident laser beam; yellow: back reflected beam). Note that the domain size was 1.01 mm × 1.12 mm and the laser beam was located at the center of the domain for all cases.

Figure 2.2 shows the schematic drawing of computational domain and keyhole. In computational domain, the thickness of specimen was set to 2 mm and 0.5 mm vacuum space was established at the top and bottom side for laser beam propagation

Deep Residual Model

The model was trained from scratch, and in Figure 2.7 the loss curves for the training dataset (black) and the validation dataset (blue) according to the epochs are presented. Only the losses for ResNet-34 are presented, which was the optimal depth for our dataset.

Figure 2.7. Loss profile according to the epochs (ResNet-34).

Results and Discussion

Deep-Learning-Based Real-Time Monitoring of Full-Penetration Laser Keyhole

Data Preparation for Deep Learning Model

The object detection deep learning model (YOLOv4) was trained based on the measurement of the keyhole opening (assuming it is elliptical), and this model automatically extracted the location and size of the keyhole openings. Consequently, only the keyhole openings were observed and the melt pool was ignored in the recording. The data set for the deep learning model for object detection was prepared based on the keyhole aperture measurements obtained from observations of the welding process.

Examples of the top and bottom openings of the keyhole, marked as black ellipses, are shown in figure 3.4. The top opening of the keyhole was measured as an ellipse intersecting the bright white part, as shown in Figure 3.4 (a). In the bottom opening of the keyhole, a blue or green elliptical shape was observed located inside a bright white spot and a black spot, as shown in Figure 3.4 (b), (c) and (d) respectively.

The keyhole opening detection models were trained separately for each dataset because the shapes of the upper and lower keyhole openings were different, as shown in Figure 3.4. In the dataset for the deep learning model, the input image resolution was set to 192 x 256 pixels, with only the keyhole openings captured using the high-speed cameras.

Figure 3.1. Outline of the proposed AI real-time monitoring system for laser keyhole welding

Deep Learning Model

As shown in Figure 3.5, only half of the input cards were involved in the operation, and these cards were combined with the other half after the operation. The number of parameters in the calculation was reduced, which led to an improvement in the calculation speed. To increase the amount of information about the location of the object, PAN, composed of a top-down and bottom-up path, was applied in the neck part, as shown in Figure 3.6.

In the header, as shown in figure 3.6, they present maps of the three sizes passed through the detection filter. In model training using dual TITAN RTX GPUs, the warm step was set as 1000 epochs with a single GPU, and the initial learning rate was divided by the number of GPUs. The input image (number of image channels and resolutions) was a 2-D image of the combined top and bottom keyhole apertures, as shown in the dotted box at the bottom right of Figure 3.1.

In the figure, the red and yellow ellipses represent the upper and lower openings, respectively. The ground-truth laser absorbance was determined by performing a ray-tracing simulation on the keyhole, which was constructed in the shape of a cone by connecting the top and bottom openings in a linear fashion.

Figure 3.6. Architecture of the object detection model (YOLOv4).

Deep-Learning-Based Synchronized Coaxial Monitoring of Full-Penetration Laser

Experiment and Simulation for Laser Keyhole Welding of Al Alloy

The laser beam was collimated through a 160 mm focal lens delivered by a 100 μm fiber cable and had a Gaussian intensity profile with a focused beam diameter of 100 μm at the sample surface. Al alloys have a high reflectivity for the 1070 nm laser beam, and the vertical irradiation of the laser beam may damage the laser head due to the back-reflected light. For this reason, the laser head was tilted by 10° to avoid damage to the optical devices inside the laser head, as shown in Figure 4.1.

However, in this study, coaxial observation was not possible in the same method as Chapter 3 because the laser head was tilted to prevent damage to optical devices. It was the same angle as the coaxial observation performed without tilting the laser head, but it led to a slight distortion of the x-axis. In this study, the laser beam absorption inside the keyhole was calculated using the ray-tracing method, finding out the multiple reflection patterns inside the keyhole.

For the energy source in the ray-tracing simulation, the laser beam had a circularly polarized continuous wave, a Gaussian intensity profile, a wavelength of 1070 nm, and the focused beam diameter of 100 μm. In this study, the laser beam was tilted 10° and the focal position was on the top surface of the sample.

Figure 4.1. Schematic diagram of synchronized high-speed coaxial observation method in Al alloy welding

Data Preparation and Deep Learning Model

In this study, the model that automatically detects keyhole openings is developed using the YOLOv4 [63] model, an object detection deep learning framework used in Chapter 3. Due to the characteristics of data-driven deep learning models, the performance and purpose of the model is determined by the dataset used in deep learning framework. Therefore, to develop a keyhole detection model in Al alloy welding, the training was carried out using the dataset obtained from the observation of Al alloy welding, and the conditions for training were almost similar to those used in Chapter 3.

As the input image resolution increases, the minibatch size was 10 due to GPU memory limitation.

Deep-Learning-Based Defects Detection Model in Keyhole-mode Laser Welding of

Experiment of Thin Foil Welding

The beam was delivered through a fiber cable, collimated with a 160 mm focal lens, and the focal position was on the top surface of stainless steel foil. The schematic diagrams of vacuum assisted jig are shown in Figure 5.1 (a) and (b), and the jigs were connected to the motorized linear stage when the laser head was fixed at the starting point of beam path. To make the specimens for tensile test, welding was performed on the jig shown in Figure 5.1 (b) and a laser cutting machine (K2CM S1 Wide by K2 Laser System Inc.) was used for cutting ASTM-E8 tensile test specimen shown in Figure 5.1 (c) (dotted red line).

Schematic diagrams of (a) experimental setup for cross-sectional measurement and (b) tensile strength test. c) Specimen for tensile strength test (ASTM-E8). Depending on the cutting directions, bead cross sections were measured perpendicular to the bead shown in Figure 5.2 (a) and along the bead shown in Figure 5.2 (b). The samples were cut at 10 mm intervals from the starting point of the bead in order to obtain a total of nine pieces.

In high-speed camera observation, the top surface of the welding process was recorded at a frame rate of 5000 frames per second. To evaluate the degree of oxidation in the welded beads, EDS analysis was performed to analyze the chemical compositions in the beads.

Figure 5.1. Schematic diagrams of (a) experimental set up for measuring cross-section and (b) tensile strength test

Data Preparation and Deep Learning Model

Results and Discussion

Under the above conditions, the coupling was successful regardless of the shielding gas flow. This means that the welding process became more unstable as the shielding gas flow increased. As the shielding gas flow rate increased, these patterns occurred more frequently, resulting in an increase in penetration depth variations during the welding process.

Moreover, the standard deviation of penetration was dramatically increased after shielding gas flow rate of 5/min. As the flow rate of shielding gas was increased, the average penetration depth tended to decrease. Note that the EDS analysis was only performed at the screen gas flow rate of 5 l/min in two experimental conditions (laser power W and scanning speed: 13, 9 m/min, respectively).

However, the welding processes were already unstable at the shielding gas flow rate of 5 l/min, as shown in Figure 5.9. Through the EDS analysis results, the oxidation in the bead was completely prevented after the shielding gas flow of 5 l/min.

Figure 5.5. Cross-sectional weld images at the bead positions of 10 mm intervals under laser power of 450 W, scanning speed of 9 m/min, shielding gas flow rate of 10 l/min

Conclusion and Future Works

Conclusion

Future Works

Deng, "Observation of Spatter Formation Mechanisms in High Power Laser Welding of Thick Plate", vol. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, p. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, p.

Sun, "Squeeze-and-excitation networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. Transfer, “Numerical Study of Keyhole Dynamics and Keyhole Induced Porosity Formation in Remote Laser Welding of Al Alloys,” vol. Lei, “Effect of plate thickness on weld pool dynamics and keyhole-induced porosity formation in laser welding of Al alloy,” vol.

Huang et al., “Effects of Mg Content on Keyhole Behavior During Deep Penetration Laser Welding of Al-Mg Alloys,” vol. Farhadi, “YOLO9000: better, faster, stronger,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, p.