© J. Moske et al., Published by EDP Sciences 2025

1 Introduction

The production of complex components using stamping, deep-drawing, and bending operations represents central manufacturing methods within the sheet metal industry [1,2]. These components are used in many different sectors, including mechanical and automotive engineering, medical technology, electrical engineering, and consumer goods, and are used in applications ranging from tiny plug contacts to large car body panels [3]. Owing to high stroke rates and low process costs in serial production, unit costs remain comparatively low [4]. In the face of international competition, companies must optimize their processes to maximize productivity and reduce costs. Even minor downtimes or deviations in surface quality can lead to substantial negative consequences. Therefore, a primary goal is to reduce both planned and unplanned downtime to an absolute minimum [5].

Established monitoring strategies for multi-stage forming processes typically rely on time-series force signals combined with anomaly detection methods, primarily aimed at preventing overloads in machines and tools [6, 7]. However, numerous methodologies have already been successfully developed and implemented for monitoring wear in blanking, including force-based process monitoring and optical post-blanking measurements in combination with machine learning and deep learning approaches [8]. To improve wear detection in high-speed forming operations, this study introduces an optical sensor specifically developed for inline wear analysis on a stroke-by-stroke basis. Image data, especially when fused with time-series signals, offers promising potential for identifying defects, their causes, and their location in complex processes. A key limitation, however, remains the feasibility of real-time processing, particularly in multi-stage processes operating at rates above 1,000 strokes per minute [9,10]. Conventional image-based systems often fail to meet the real-time data processing demands of such high-speed environments [11]. The soft sensor that is presented here uses photometric stereo. This system is capable of real-time data processing. It does not require a machine learning approach [12,13]. Its algorithm enables observation of a part formed through several forming operations. Typical optical observation systems (industrial cameras) can only monitor a single work tool [14]. However, this system can differentiate between different production steps by leveraging domain-specific knowledge. Domain-specific knowledge includes, in particular, knowledge of critical component zones in which wear is caused either by wear mechanisms during deep drawing or ironing. This process knowledge is crucial for correctly interpreting the resulting wear patterns and differentiating between the various causes of wear.

Most photometric stereo methods usually require numerous image inputs and are only applicable when non-Lambertian (specular) observations occupy only a small part of the entire image [15]. The challenges arise from the fundamental assumption that the method only works with matte surfaces and high-reflectance scattering. In this respect, this publication is the first to investigate these specular metallic surfaces. These challenges are compounded by the increasing prevalence of coated and reflective materials in industrial production, such as tinned white sheet metal, which introduce optical complexities. Recent research highlights the critical role of surface characteristics in the design of automated visual inspection systems, particularly for metallic components. Saiz et al. demonstrate the effectiveness of photometric stereo techniques in detecting small surface defects on specular, nickel-plated components, an area in which traditional imaging techniques often fail due to strong reflections, specular highlights and shadows [16,17]. The aforementioned effects distort intensity measurements and significantly degrade the accuracy of normal estimation and defect detection. Recent advancements in deep learning have shown that neural networks can effectively address these challenges [17]. By learning complex reflection patterns directly from data, neural network-based photometric stereo methods have demonstrated robustness to non-Lambertian effects. This approach enables accurate surface normal estimation and defect localization even on highly reflective surfaces. Current research indicates that this data-driven strategy offers a promising solution for practical applications where classical models fail [14]. These findings underscore the necessity of adapting the illumination setup to the specific optical behavior of the surface under inspection. Rather than relying on uniform lighting conditions, the authors propose a compact representation combining topographic (e.g., curvature, gradient magnitude) and spectral (e.g., texture) information into an RGB image format. Consequently, the presence of mirror-like surfaces can invalidate the assumptions of numerous photometric methodologies, which rely on diffuse reflection. This undermines the efficacy of detection processes. In light of the wide range of materials used in production today, robust detection accuracy across varying surface properties is of critical importance.

To tackle this, the presented optical sensor is integrated as a scanning module within the production tool. Its initial implementation involves placing it in a sub-tool of a modular progressive die. Through the use of photometric stereo technology, the system compares target and actual reflective properties of the component surface, enabling a stroke-by-stroke evaluation of surface changes. These deviations provide valuable information about the wear condition of both punch and die, as well as changes to the surface roughness of the workpiece. This facilitates a more accurate wear assessment and supports effective monitoring and maintenance strategies. Alongside the conceptual approach, the system’s function is experimentally validated under real operating conditions within a progressive die.

This publication extends the system introduced by Moske et al. [12], enabling a differentiated analysis of process-specific defects arising during deep drawing and ironing operations. The investigation covers two distinct types of material surface: one with a matte finish and one with a specular finish.

2 Material and methods

This chapter provides a comprehensive overview of the experimental setup and methodology applied in this study. In the first section, the technical configuration and the execution of the forming experiments are described in detail. This includes the specification of the tool systems, materials, and test procedures used to generate reproducible wear conditions across defined tool regions. The second section focuses on the optical acquisition and processing pipeline based on the Photometric Stereo technique. It outlines the procedure for generating surface normal maps from multi-illumination images and describes how these normal maps are further processed to derive localized wear masks. Particular attention is given to the challenges associated with inspecting reflective and specular surfaces, which are common in metallic forming tools. The image processing workflow, which comprises calibration, normalization, artefact suppression and segmentation, is discussed in detail as it forms the basis for reliable and robust wear detection on complex geometries.

2.1 Experimental setup

To evaluate the compatibility and performance of a photometric stereo-based scanning system in an industrial forming environment, the sensor module was systematically integrated into a modular progressive die [12]. To investigate the robustness of the system across different surface conditions, trials were conducted with both CR DC04 steel and ETSR TS245 material. For all tests, the application of an additional lubricating film was deliberately omitted in order to rule out potential interference and to be able to specifically investigate the influence of different surface roughnesses. This enabled the assessment of matte and specular surface finishes. For the CR DC04 Ra = 1.13 μm and Rz = 6.42 μm. For the ETSR TS245 Ra = 0.143 μm and Rz = 1.03 μm. The measurement of the tool surface was performed along the drawing radius in the radial drawing direction. Also, Figure 1 shows that the surface topographies of the sheet metal prior to forming were captured using a μ Surf system. This is a white-light confocal optical 3D surface measurement device that enables non-contact characterization of surface topography with high spatial resolution. These measurements serve to illustrate the distinction between matte and specular surfaces. In Figure 1a, the surface of CR DC04 is shown, whereas Figure 1b depicts the ETSR TS245 material. The surface of ETSR TS245 appears significantly smoother and therefore exhibits a specular, mirror-like appearance to the observer.

The experimental setup was implemented on a Bruderer BSTA 410-110 high-speed stamping press, operating at a stroke rate of 200 strokes per minute. Each test series comprised 200 strokes and was performed using ∼ 0.5 mm thick and 50 mm wide sheet metal. The die comprises sequential stages for cutting, deep drawing, ironing, and final separation (see Fig. 2). The formed component is a cup with an inner diameter of 14 mm and a depth of 13 mm with a wall thickness of 0.4 mm.

Inline surface inspection is achieved through a high-speed camera system equipped with directional LED lighting, enabling real-time photometric reconstruction of surface normal maps between ironing and cut out stage (see Fig. 2). These are compared to reference geometries to detect deviations and generate pixel-wise wear masks. The system is controlled via a Raspberry Pi 5, with local data acquisition and external post-processing. Embedded piezoelectric force sensors (Kistler 9051C and 9240A) monitor process forces for correlation with surface anomalies [12]. The experimental data serve as a foundation for developing predictive wear models and refining tool maintenance strategies under realistic production conditions. Two types of dies were utilized in the wear tests for each material: unworn and synthetically worn dies. The unworn dies represent a new condition, while the synthetically worn dies were modified by artificial roughening so that they correspond to a worn condition. Although the condition observed does not correspond to actual wear patterns, the tests deliberately avoided generating actual wear over several hundred thousand strokes, as this would not have been conducive to investigating the method. Instead, specific surface modifications were introduced to demonstrate that even the smallest changes can be reliably detected and distinguished from one another by the optical sensor. All tests were carried out with a stroke rate of 200 strokes per minute. Table 1 provides a structured overview of four wear test series conducted with two different materials: CR DC04 (a cold-rolled steel with low surface reflectivity) and ETSR TS245 (a coated, highly reflective material). Each test series combines different wear conditions of the deep drawing die and ironing die, either unworn or synthetically worn, to investigate their influence on surface quality and sensor detection capabilities. For each material, the roughness parameters Ra (arithmetical mean roughness) and Rz (average maximum height of the profile) are listed for both forming stages.

Fig. 1 Topography difference between the a) CR DC04 and b) ETSR TS245.

Fig. 2 Sequence of processes with integrated scanning module.

2.2 Building the wear mask

The experiments are carried out as outlined in Figure 2, following the deep drawing and ironing operations. The scanning process begins with a trigger signal from an inductive sensor at the bottom dead center, ensuring the component is positioned within the camera’s focal area. The first LED of the ring light is then activated to reveal the component’s reflective properties, which are captured by the camera. This procedure is repeated for all n(n=18) LEDs, resulting in a dataset of n images per component at the conclusion of the scan (see Fig. 3). Additional information on the scanning process can be found in [12]. A complete dataset of a single component (raw data, normal map, and wear mask) amounts to just under 20 MB on average.

Since the position of the LED responsible for the reflection and shading characteristics is known for each captured image, the normal map can be efficiently calculated by a simple matrix multiplication, as described by the equation in Figure 3. Specifically, N denotes the normal vector, L the light direction, I the image intensity, and ρ the albedo at each pixel position (x, y). Inline optical wear detection is performed by comparing the normal map of the currently produced part (ACTUAL) with that of a flawless (TARGET) component. If the normal vector in the actual measurement deviates from that in the target measurement, tool wear at that location can be assumed, as illustrated in Figure 4. The tolerance ranges is an 8-bit value from 0 to 255 for each color channel and is equal to the intensity I. The tolerances have been defined for the purpose of this study. Black: 0–9; Blue: 10–14, green: 15–24, and red: 25–255. The color mapping must be empirically evaluated for each component. The deviations are represented using a histogram, and the thresholds are selected dynamically. For a metric quantification of the deviations, the system would additionally require geometric calibration, which was not performed in this case.

To ensure that comparisons are made at identical surface points, it is essential to align the target and actual measurements so that the same normal vector at the exact object location can be evaluated. In a production environment, however, individual measurements may be displaced by a few millimeters caused by vibrations in the metal strip during production. While such displacements do not disrupt the production line, they pose a challenge by preventing accurate verification of the same surface point using the calculated normal map. To achieve precise, point-by-point surface inspection, the normal maps must therefore be adjusted. This involves translating and, if necessary, scaling the actual measurement – a process known as homography. To compute the homography between the reference and actual measurements, feature points are detected and extracted from the camera images using the SIFT detector [18].

Feature points represent distinctive points within an image, such as corners or edges, depicted by the colored dots in the first row of images in Figure 5. Once feature points have been detected in both the reference and actual measurements, correlations between these points are identified, shown on the bottom image of Figure 5. These correlations make it possible to determine which surface points in target and actual measurements correspond to each other. If at least four different surface points are successfully correlated, a homography can be computed and applied to one of the measurements. In this case, the homography was applied to the actual measurement, transforming it in such a way that identical surface points are compared at the same pixel positions in both the target and actual measurements, which is critical for calculating the wear mask. After successfully applying the homography, the deviation between the reference and actual normal vectors can be calculated, as illustrated in Figure 6. Here, the distance between the normal vectors at the same surface points is evaluated. If this distance exceeds a defined tolerance range, a surface change due to tool wear is assumed, as visualized in Figure 4. Conversely, if the distance lies within the tolerance range, it is attributed to normal production variations.

Since homography now allows the actual measurement to be compared with the target measurement at identical pixel positions, a simple subtraction between the two measurements can be performed to determine the distances between the normal vectors. Additionally, several tolerance ranges were defined to account for different levels of deviation between the reference and actual measurements, which are represented by color coding within the wear mask. No deviation is indicated in black, minor deviations in blue, significant deviations in green, and extreme deviations in red, as shown in Figure 6. It should be noted that only surface modifications, e.g., resulting from wear, are highlighted at this stage. The system is not capable of differentiating between wear categories; the color mapping is solely based on the optical evaluation of the deviation of the surface normal vector from the empirically defined tolerance. In this way, an inline-capable method is established that enables stroke to stroke quality assurance at high forming speeds, with data being processed within a single stroke.

The calculation of photometric stereo assumes a non-Lambertian surface. However, this condition is not always met when processing sheet metal components. In the case of CR DC04, the surface exhibited diffuse reflection, allowing normal maps to be calculated without further intervention, as demonstrated in Section 3.1.

However, if the surface is highly reflective, problems can arise in calculating the normal maps. Highly reflective areas in the input images may lead to overexposure, resulting in a loss of information in these regions, as the reflected light completely masks the relevant surface structures. Consequently, these overexposed regions lack the necessary data and cannot be used for normal calculation. One approach to address this issue is to reduce the camera’s exposure time, thereby limiting the amount of light captured by the image sensor. In our setup, the camera was already set to its minimum exposure time. Another option is to dim the light source to reduce reflection from highly reflective surfaces. However, in the current setup, the LEDs cannot be dimmed.

To overcome this limitation, overexposed regions in the input images were masked out. Specifically, grayscale values in the range of 250−254 were detected and excluded from normal calculation, as these areas contain no relevant information, as shown in Figure 7. As a result, areas that are masked out due to overexposure must not be used for normal calculation within the masked region, as this would otherwise lead to distorted normal vectors. It is important to ensure that no region is masked out in every image, as otherwise no normal vector can be determined for that area, which means that a sufficient number of input images and light sources must be used so that each region remains unmasked in at least three images. A prediction becomes infeasible if a surface point is not captured in at least three images. As long as this condition is met, reliable prediction remains possible. However, if a point is consistently overexposed across nearly all images, prediction cannot be performed. Such a case also raises the question of whether the illumination and/or camera settings are correctly adjusted, since this situation should not ordinarily occur. While the use of polarization filters presents a promising approach, it was not feasible to implement them within the hardware setup on short notice, as they require careful calibration. Consequently, a software-based solution was introduced, which could be further refined and validated in subsequent work. With the presented setup, this requirement is fulfilled, as shown by the results in Section 3.2.

Another challenge with highly reflective surfaces is the appearance of reflections from the camera, LEDs, or the surrounding environment on the component’s surface. These reflections can be misinterpreted as actual geometry in the normal map, distorting the calculated normals in those areas. While this can be addressed by more complex image acquisition and processing methods, such approaches significantly increase the processing time per component, making them unsuitable for progressive dies [19]. In our case, this issue was mitigated by keeping the enclosure used for photometric reconstruction uniformly black, ensuring that reflections from the surroundings appeared uniformly dark. Additionally, to avoid reflections of the camera and LEDs on the component surface, the scanner’s optical unit was swiveled as needed, as visualized in Figure 8. The camera was positioned in such a way that neither the camera nor the LEDs were visible in the surface reflections. The camera angle was empirically adjusted through preliminary trials until the desired image quality was achieved.

Following the implementation of the aforementioned adjustments, the ETSR TS245 sheet metal was successfully digitized and analyzed, notwithstanding its highly reflective nature.

Fig. 3 Calculation of the normal map based on multiple images of the same object illuminated from different light directions.

Fig. 4 The deviation of the normal vectors at a given object point is calculated. If this deviation exceeds a predefined tolerance, the corresponding object point at that pixel position in the normal map is classified as not acceptable (red dashed line). If the deviation is within the tolerance, the pixel position is considered acceptable (green dashed line).

Fig. 5 Calculation of the homography for sub-pixel accurate alignment of the captured images within a scan of a component.

Fig. 6 Calculation of the wear mask after transforming the actual normal map to ensure reliable verification of the normal vectors at identical object points.

Fig. 7 Detecting and removing highlights for the calculation of normal maps. The marked region on the right side is excluded from normal vector calculation within this area, but normal vectors are still computed for the remainder of the image.

Fig. 8 Swiveling optical module for viewing objects at angles between 0° and 50°.

3 Results

During the operational life of a progressive die, both adhesive and abrasive wear mechanisms affect the active tool components, particularly the forming dies and drawing punches, as a result of the prevailing mechanical and thermal loads. It is evident that, within this group, the dies are subjected to the most severe frictional and pressure stresses, rendering them particularly vulnerable to wear-induced degradation. Consequently, the occurrence of wear phenomena at the die surface has the potential to initiate defects in the semi-finished product. The impact of tool wear is particularly evident when manufacturing components using dies that have been worn down synthetically, as this results in measurable quality deviations. The progressive die examined in this study consists of two primary forming stages, deep drawing and ironing, used to manufacture a cup-shaped component. Each of these forming stages can introduce distinct types of production-related defects, which may arise from inherent process characteristics and progressive tool wear.

3.1 Evaluation of the results for CR DC04

The deep drawing stage is predominantly associated with defects such as scoring and localized material accumulations, whereas the ironing stage primarily influences the component surface by introducing waviness. These typical defect manifestations are exemplified in Figure 9.

The conducted test series focused on defect types that are particularly relevant for evaluation by the scanning module, as well as for feeding defect information back into the production process. The experiments were designed around synthetically induced wear patterns applied to the forming dies, as elaborated in detail in Section 2.1. The figure provides a visual representation of how different wear states in the deep-drawing and ironing dies affect the surface quality of the produced cups. Figure 9a shows an overview of the cup, highlighting the analyzed region with a red rectangle. This marked area serves as a point of reference for the subsequent detailed investigations of the surface topography depicted in subfigures b) to f), which were obtained using a confocal microscope. In Figure 9b, a three-dimensional surface image is shown for a cup produced using both an unworn deepdrawing and an unworn ironing die. This reference configuration yields a defect-free component characterized by a homogeneous and uniform surface texture. In contrast, Figure 9c depicts the effect of synthetic wear applied solely to the ironing die while maintaining an unworn deep-drawing die. This configuration leads to pronounced increases in surface roughness and distinct irregularities. Figure 9d presents the corresponding normal map generated by the integrated scanning module, representing the outcome of the surface reconstruction process used to derive the wear mask. Figure 9e illustrates the surface structure resulting from a synthetically worn deep-drawing die combined with an unworn ironing die. The image reveals distinct linear features and surface elevations, indicating that wear on the deep-drawing die significantly impacts the surface morphology. Finally, Figure 9f demonstrates the cumulative effect of simultaneous synthetic wear on both forming dies. This state results in severely developed grooves, increased surface roughness, and an overall highly irregular texture. The 3D surface images use a color scale to visualize topographical height differences: red denotes the highest elevations, approximately 15 μm above the base level, while blue indicates the lowest areas corresponding to the zero plane.

In addition, Figure 10 provides several normal maps obtained during this study. The color mapping (equal in Fig. 12) was empirically assigned based on the deviation of the actual normal vector in the pixel position from the defined tolerance, with blue indicating minor deviations, green representing moderate deviations, and red corresponding to large deviations. Subfigure a) presents a normal map of a defect-free reference cup, while subfigure d) shows a normal map from test series 4, where both the deep-drawing and ironing dies were synthetically worn. Despite the known differences in die conditions, a visual comparison of these two maps reveals minimal distinguishable variations.

Subfigure b) displays the direct comparison between fault-free cup images, where the resulting mask appears nearly black. This indicates the use of identical data sets and demonstrates the method’s limitation in detecting variations under these conditions. Figure 10c presents the scenario in which the deep-drawing die is synthetically worn while the ironing die remains unworn. In contrast, Figure 10e illustrates the inverse configuration, i.e., an unworn deep-drawing die combined with a worn ironing die. Finally, Figure 10f depicts the case where the deep-drawing and ironing dies exhibit synthetic wear simultaneously. From these images, it becomes evident that specific regions within the normal maps can be unambiguously attributed to distinct stages of the forming process. The visualizations clearly delineate (highlighted by red boxes) those areas in which surface defects are associated with particular process steps. A direct comparison of the marked regions in Figures 10b, c, e, and f with the corresponding surface structures shown in Figure 9 confirms that the wear masks consistently highlight the same zones. These zones can be systematically assigned to either the deep drawing or ironing process. Consequently, the soft sensor facilitates a robust and unambiguous correlation between observed surface defects and their originating process steps. For instance, defects such as scoring and scratches, which appear in the upper part of the image, can be attributed to the deep-drawing stage. Waviness in the lower part of the cup, on the other hand, is indicative of wear originating from the ironing process. This substantiates a reliable allocation of causal relationships. The quality of the surface finish constitutes a critical parameter within this technological context, with the condition of the forming dies playing a dominant role in influencing this characteristic. Due to the wide variability of production parameters, surface quality cannot be defined through a universal tolerance specification. Instead, it requires a process- and product-specific empirical definition that accounts for the inherent attributes of the forming operation and the geometry of the manufactured component. These internal characteristics are central to defining acceptable deviations and formulating specific quality criteria. To enable consistent and dependable system performance, a foundational database is established using a limited volume of data. With the acquisition of approximately 200 semi-finished parts, a reliable data basis is established. This teaching data enables qualitative assessments of surface condition changes and supports the efficient adaptation and integration of the system into specific application environments. This is evidenced by the self-comparison (see Figs. 10b and 12b), where the wear mask remains predominantly black, indicating the absence of significant deviations.

Fig. 9 Images of the CR DC04 cup surface: a) Photo of the cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the surface within the red rectangle, recorded with a confocal white light microscope; d) normal map from the scanning module.

Fig. 10 Comparison normal map from the scanning module and wear mask of the four wear test series CR DC04: a) normal map of a fault-free cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the corresponding wear mask; d) normal map of a bad cup.

Fig. 11 Images of the ETSR TS245 cup surface: a) Photo of the cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the surface within the red rectangle, recorded with a confocal white light microscope; d) normal map from the scanning module.

Fig. 12 Comparison normal map from the scanning module and wear mask of the four wear test series ETSR TS245: a) normal map of a fault-free cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the corresponding wear mask; d) normal map of a bad cup.

3.2 Evaluation of the results for ETSR TS245

Figure 11 follows the same structure as Figure 9, illustrating the corresponding test series for the alternative material ETSR TS245. Figure 11b presents a three-dimensional surface reconstruction of a cup manufactured using both an unworn deep-drawing die and an unworn ironing die. This reference configuration results in a component with a comparatively smooth surface, exhibiting a homogeneous and defect-free topography. In Figure 11c, the influence of synthetic wear applied solely to the ironing die is shown, while the deep-drawing die remains unworn. The resulting surface displays pronounced waviness and irregular structures, which can be attributed to intense stick-slip phenomena. These are caused by elevated friction and intermittent material adhesion-typical effects of a worn ironing die. Figure 11e depicts the surface structure obtained using a synthetically worn deep-drawing die in combination with an unworn ironing die. The surface is characterized by distinct linear grooves and elevations, indicating that die wear in the deep-drawing stage induces directional scratching and significantly increases surface roughness and anisotropy. Finally, Figure 11f illustrates the condition in which the deep-drawing and ironing dies are synthetically worn. Notably, the resulting surface degradation is not additive, as might be expected based on the individual effects observed in Figures 11c and 11e. In contrast to the behavior observed with CR DC04, the overall surface texture under combined wear conditions appears less severely affected. This suggests material-specific interactions or potential compensatory mechanisms during the forming process. This observation underlines the necessity of a differentiated evaluation of wear effects depending on material properties and die combinations and sets the stage for the subsequent analysis.

Figure 12 follows the same pattern as Figure 10, presenting the results in an analogue layout. However, this time the material is ETSR TS245. These areas are highlighted again in the partial images by red rectangles. As already mentioned, the scanning module in Figure 12f shows the area previously examined under the microscope based on the test series with both worn dies. It should be noted at this point that the coloring is less pronounced due to the algorithm. This is due to the opposing effects of the processes cancelling out the wear marks that occur. Furthermore, the limitations caused by the reflective material are most noticeable in this context. This limitation is due to the lower information density caused by reflections and shadowing effects on the highly reflective surface of the ETSR TS245 material. This optical interference impairs the real-time detection of subtle deviations and surface features. In this context, particular reference should be made to the surface features shown in Figures 11 and 12e,f, as these configurations are subject to closer examination in order to analyze the phenomena observed in Figures 11f and 13f.

Figure 13 presents microscopic surface images of the cup mantle generated in different tool wear configurations. In Figure 13a (microscope refence Fig. 11c) the deep-drawing die remains unworn, while the ironing die exhibits significant wear. This condition results in a surface characterized by pronounced, uninterrupted grooves along the drawing direction, indicative of severe tribological interaction during the ironing stage. The surface appears highly textured with dense and sharp striations, reflecting direct contact and insufficient material smoothing due to tool wear. In contrast, Figure 13b (microscope refence Fig. 11f) shows the surface resulting from a worn tool configuration, where both the deep-drawing and ironing dies exhibit advanced wear. Interestingly, the surface appears comparatively smoother and less sharply striated. It can thus be hypothesized that, due to the altered condition of both stages, wear particles adhere to the cup surface, resulting in a thin abrasive layer on the cup when it is subsequently pressed into the ironing stage. This unexpected improvement in visual surface quality may be explained by tribological mechanisms that are hypothesized to occur during the initial deep-drawing phase. Indications suggest that wear debris, oxide particles, or residual contaminants originating from the worn deep-drawing die could adhere to the cup surface. In comparison, Figure 11c,f show the corresponding microscopic images. It can be observed that in Figure 11c, more pronounced peaks and valleys (ranging from yellow to blue) are present, as indicated by the vertically oriented grooves. In contrast, the surface in Figure 11f appears generally rougher; however, the height difference between peaks and valleys is less pronounced (ranging from green to yellow). During the subsequent ironing step, it is conceivable that these particles become embedded in the surface topography (see red ellipses in Figure 13b), particularly within the existing grooves, where they might fill micro-valleys. The bright, droplet-like and patchy white structures that appear to disrupt the linear groove pattern in Figure 13b could indicate particle entrapment and redistribution during ironing, which may, in turn, contribute to a partial sealing or masking of surface roughness. While the embedded particles visually reduce apparent surface texture, their presence indicates a complex interplay of tool wear, surface adhesion, and particle transfer phenomena that merit further investigation regarding their impact on functional surface properties and product quality. For a conclusive assessment, this phenomenon must be subjected to further investigation.

Fig. 13 Images of the ETSR TS245 cup surface: a) Microscope image of the cup with an unworn deep drawing die and a worn ironing die; b) Microscope image of the cup with a worn deep drawing die and a worn ironing die.

3.3 Evaluation force signals

All subsequently presented force signals were acquired with a sampling frequency of 25 kHz. Figure 14 shows the direct force signals recorded during the deep-drawing process for the two different sheet materials: CR DC04 a) and ETS R TS245 b). In both cases, the process force is plotted over time and reflects the forming behavior under identical geometric and kinematic conditions, with variations introduced only by the die wear state and the material properties. Each diagram compares four tool configurations, distinguished by combinations of worn and unworn deep-drawing and ironing dies.

The blue and orange curves in a) and b) correspond to test series with an unworn deep-drawing die (Du_Ix), while the green and red curves represent series using a synthetically worn deep-drawing die (Dw_Ix). The force profile exhibits the characteristic shape of a deep-drawing process: an initial increase in force as deformation begins, followed by a relatively stable plateau, and a subsequent drop as the forming process concludes. Notably, in the series with worn deep-drawing dies (green and red), distinct wave-like structures or double peaks appear within the plateau region. These fluctuations are likely to be indicative of stick-slip effects or non-uniform material flow. Both of these are typical signs of increased friction and altered tribological conditions, which are caused by tool wear. In contrast, the curves associated with the unworn dies exhibit a more consistent and smoother plateau, suggesting more stable forming conditions.

The force curve of Figure 14b again follows the typical deep-drawing pattern, characterized by an initial rise, a force plateau during steady-state forming, and a final drop as the punch completes its stroke. Compared to the previous configuration, higher overall force levels are observed, with peak values exceeding 10 kN, indicating a more demanding process. This may be attributed to the higher material strength and the associated higher plastic resistance. Distinct differences emerge within the plateau region: while the unworn-die configurations (blue, orange) show relatively smooth force profiles, the worn-die setups (green, red) again exhibit slight undulations or localized peaks, particularly in the red curve (Dw_Iw), which shows the highest overall forces. These fluctuations are again indicative of increased friction and interrupted material flow, likely due to tool wear-induced changes in contact conditions and surface roughness. The differences in maximum force and plateau behavior across the configurations underline the sensitivity of the process to tool wear, especially when both deep-drawing and ironing dies are affected.

Figure 15 shows the evolution of the indirectly measured process force during the ironing stage for four different test series and for each material in the same pattern. The blue and green curves represent configurations with an unworn ironing die (Dx_Iu), while the orange and red curves correspond to a synthetically worn die (Dx_Iw). Unlike the direct deep-drawing force signals shown previously, the indirect signals are more abstract and based on compression and strain. The interpretation of indirectly measured strain signals poses a significantly higher cognitive challenge for human operators compared to directly measured force signals. These effects are especially visible in the material ETSR TS245, where distinct force spikes and signal irregularities appear.

For the CR DC04 curves exhibit the characteristic force evolution typical of ironing processes: an initial increase in force due to progressive material deformation, peaking around 750 milliseconds, followed by a symmetric decrease towards the end of the stroke. Notably, configurations with a worn ironing die (Dx_Iw) exhibit a slightly elevated maximum force compared to their unworn counterparts-by approximately 5% of the peak value. This can be attributed to increased friction and resistance caused by surface roughness or micro-adhesions associated with the worn die. Furthermore, subtle differences in the steepness and local oscillations of the force curve can be observed. These oscillations are particularly pronounced in the configurations involving worn tools and may be indicative of localized stick-slip effects, where alternating static and kinetic friction leads to dynamic force fluctuations. Such behavior can negatively influence surface integrity and process stability and becomes more evident with increasing tool wear.

However, it can be seen that, unlike with CR DC04, there is no bell curve when using ETSR TS245. The lower force level observed during the ironing process can be explained by multiple factors. Firstly, the formation of a solid lubricant layer due to the tin coating significantly reduces friction at the tool-workpiece interface, which is also evident in Figure 12b. Secondly, the material used, ETSR TS245, has a relatively low initial sheet thickness of only 0.499 mm, which inherently leads to lower ironing forces due to the reduced cross-sectional area to be plastically deformed. Despite these effects, the experimental results remain comparable, as the primary process conditions and material properties, apart from thickness and coating-induced lubrication, are consistent across the investigated cases. The most significant discrepancies manifest themselves at the end of the signal. Across all test conditions for ETSR TS245, several distinct force peaks are observed between approximately 1200 and 1500 milliseconds, indicative of localized resistance changes during the upstroke of the ironing process. These pronounced peaks are mainly caused by a reduction in punch velocity, which promotes stick-slip effects due to adhesive friction phenomena. Compared to CR DC04, the force progression differs significantly. A comparative analysis shows that the configuration with a worn ironing die (Dx_Iw) consistently exhibits the highest force levels, particularly around 1250 milliseconds, where a pronounced peak is evident. Notably, the force signals of the worn tool conditions reveal higher characteristic sawtooth-like waveforms, which are typical of stick-slip phenomena. These patterns arise from alternating phases of adhesion and sliding at the tool-material interface, where static friction is temporarily overcome by dynamic friction. This behavior manifests as abrupt force increases followed by sudden drops, indicating intermittent local adhesion followed by rapid slip events. Such oscillations are known to impair surface quality and dimensional accuracy and are often exacerbated by tool wear or inhomogeneous material flow. In contrast, the Du_Iu curve displays a smoother and more stable force progression, reflecting a uniform and well-controlled ironing process with minimal friction-induced disturbances.

Fig. 14 Force curves of the direct process force during deep drawing for the time of one stroke. a) CR DC04, b) ETSR TS245 (D= deep drawing die, I= ironing die, u= unworn, w= worn).

Fig. 15 Force curves of the indirect process force during ironing for the time of one stroke. a) CR DC04, b) ETSR TS245 (D=deep drawing die, I=ironing die, u=unworn, w=worn).

4 Discussion

This study systematically evaluated the applicability of photometric stereo as an inline surface inspection tool for progressive forming processes under real industrial conditions. The discussion section highlights the strengths and limitations of this approach, as well as its implications, based on the collected data and observations.

While process force analysis is widely used to monitor forming operations, the present results clearly show its limited sensitivity to early-stage wear effects and surface-related anomalies. As demonstrated in Figures 14 and Figure 15, deviations between different die wear conditions manifest only subtly in the force curves. This limitation is particularly problematic in high-speed environments, where small deviations may accumulate into substantial defects if not detected promptly. In contrast, the photometric stereo-based scanning module enables detailed spatial and causal analysis of tool wear, offering significantly more granular information about the forming process. Especially the wear masks derived from normal map deviations provide localized and process-step-specific insights, enabling the attribution of defects to individual forming stages such as deep drawing or ironing.

The capability of the optical system to discriminate between defect types and their origins was validated using both matte (CR DC04) and specular (ETSR TS245) materials. For CR DC04, the scanning module successfully resolved wear-induced surface features such as scoring and waviness, as shown in Figures 9 and Figure 10. Moreover, the system could consistently assign wear signatures to the corresponding process stages, verifying the system’s diagnostic accuracy. With ETSR TS245, the application of photometric stereo was challenged by the material’s high reflectivity. However, by implementing targeted pre- and postprocessing steps such as highlight masking, optical shielding, and angular adjustment of the scanning unit, valid surface reconstructions could be achieved. This confirms that the system is adaptable to non-Lambertian surface behavior without compromising inline capability.

Interestingly, the combined wear state in ETSR TS245 yielded smoother surface structures than expected, potentially due to particle embedding and redistribution effects, as indicated in Figure 11 through Figure 13. These observations suggest material-dependent interactions between tool wear, surface topography, and tribological behavior, which cannot be fully captured by force signals or conventional inspection techniques. Therefore, a deeper understanding of such interactions may further enhance the diagnostic value of image-based inspection.

In terms of system performance, the scanning module demonstrated real-time capability at 200 strokes per minute without the need for machine learning-based inference. This represents a significant step towards the practical deployment in production environments. Furthermore, the ability to process both matte and glossy materials broadens the application range of the system and addresses a key challenge in optical quality control.

Although the results are encouraging, there are still several limitations. Overexposed areas due to specular reflections can lead to data loss, requiring careful configuration of lighting and exposure parameters. The application of an HDR filter is not feasible in this context, as the image acquisition is performed using monochrome sensors. High Dynamic Range (HDR) techniques typically rely on combining multiple exposures to reconstruct a broader range of luminance and color information. Due to the absence of color channels in monochrome imaging, the potential for dynamic range enhancement is inherently limited, rendering HDR-based reflection suppression ineffective. Additionally, although image-based methods offer high-resolution insights, they currently require post-processing steps to ensure sub-pixel alignment and accurate comparison with target data. These steps introduce latency and may limit applicability in ultra-high-speed setups; however, the real-time capability of the system is not compromised, as it merely requires slightly more computation time compared to configurations without this additional step.

Looking ahead, the integration of convolutional neural networks (CNNs) may offer a solution by enabling real-time segmentation and classification of wear patterns, potentially automating calibration and compensating for optical distortions dynamically. Such advancements would transform the system into an adaptive quality monitoring tool, capable of continuous learning and optimization during operation.

Overall, this study confirms that, when appropriately adapted, photometric stereo is a robust, modular and real-time capable method for monitoring tool wear and surface quality in progressive forming processes. It outperforms conventional force-based approaches in terms of spatial resolution, defect attribution and material adaptability, paving the way for the next generation of in-line inspection systems in smart manufacturing.

Acknowledgments

The presented results are part of the research project “Real-time capable wear models” of the European Research Association for Sheet Metal Working (EFB). Furthermore, the authors would like to thank Bruderer AG for providing the high-speed press BSTA 410-110 on which the tests were carried out.

Funding

The authors would like to thank the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt, DLR) and the German Federal Ministry for Economic Affairs and Climate Action (Bundesministerium für Wirtschaft und Klimaschutz, BMWK) for funding within the framework of project no. IGF 22250 N.

Data availability statement

The data is proprietary to the Institute for Production Engineering and Forming Machines – PtU Research and cannot be made publicly available. Nonetheless, the authors are willing to share selected datasets with qualified researchers upon reasonable request and subject to institutional approval.

Author contribution statement

Conceptualization: Jonas Moske, Hasan Kutlu; Methodology: Jonas Moske, Hasan Kutlu; Software: Hasan Kutlu; Validation: Jonas Moske, Hasan Kutlu; Formal Analysis: Jonas Moske, Hasan Kutlu; Investigation: Jonas Moske, Hasan Kutlu; Resources: Jonas Moske, Hasan Kutlu, Phil Groenewald; Data Curation: Jonas Moske, Hasan Kutlu; Writing – Original Draft Preparation: Jonas Moske; Writing – Review & Editing: Jonas Moske, Hasan Kutlu, Pedro Santos, Peter Groche; Visualization: Jonas Moske, Hasan Kutlu; Supervision: Pedro Santos, Arjan Kuijper, Andreas Weinmann, Peter Groche; Project Administration: Jonas Moske; Funding Acquisition: Peter Groche.

References

All Tables

All Figures

	Fig. 1 Topography difference between the a) CR DC04 and b) ETSR TS245.
In the text

	Fig. 2 Sequence of processes with integrated scanning module.
In the text

	Fig. 3 Calculation of the normal map based on multiple images of the same object illuminated from different light directions.
In the text

	Fig. 4 The deviation of the normal vectors at a given object point is calculated. If this deviation exceeds a predefined tolerance, the corresponding object point at that pixel position in the normal map is classified as not acceptable (red dashed line). If the deviation is within the tolerance, the pixel position is considered acceptable (green dashed line).
In the text

	Fig. 5 Calculation of the homography for sub-pixel accurate alignment of the captured images within a scan of a component.
In the text

	Fig. 6 Calculation of the wear mask after transforming the actual normal map to ensure reliable verification of the normal vectors at identical object points.
In the text

	Fig. 7 Detecting and removing highlights for the calculation of normal maps. The marked region on the right side is excluded from normal vector calculation within this area, but normal vectors are still computed for the remainder of the image.
In the text

	Fig. 8 Swiveling optical module for viewing objects at angles between 0° and 50°.
In the text

	Fig. 9 Images of the CR DC04 cup surface: a) Photo of the cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the surface within the red rectangle, recorded with a confocal white light microscope; d) normal map from the scanning module.
In the text

	Fig. 10 Comparison normal map from the scanning module and wear mask of the four wear test series CR DC04: a) normal map of a fault-free cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the corresponding wear mask; d) normal map of a bad cup.
In the text

	Fig. 11 Images of the ETSR TS245 cup surface: a) Photo of the cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the surface within the red rectangle, recorded with a confocal white light microscope; d) normal map from the scanning module.
In the text

	Fig. 12 Comparison normal map from the scanning module and wear mask of the four wear test series ETSR TS245: a) normal map of a fault-free cup; b) test series 1, c) test series 2, e) test series 3, and f) test series 4, each showing the corresponding wear mask; d) normal map of a bad cup.
In the text

	Fig. 13 Images of the ETSR TS245 cup surface: a) Microscope image of the cup with an unworn deep drawing die and a worn ironing die; b) Microscope image of the cup with a worn deep drawing die and a worn ironing die.
In the text

	Fig. 14 Force curves of the direct process force during deep drawing for the time of one stroke. a) CR DC04, b) ETSR TS245 (D= deep drawing die, I= ironing die, u= unworn, w= worn).
In the text

	Fig. 15 Force curves of the indirect process force during ironing for the time of one stroke. a) CR DC04, b) ETSR TS245 (D=deep drawing die, I=ironing die, u=unworn, w=worn).
In the text