Quality Metrics: The KPIs That Actually Matter

What are quality metrics?

Quality metrics are the measurable indicators a manufacturing organisation uses to judge whether its products and processes are doing what they are supposed to. The classical set covers three layers: product-level metrics like first pass yield and defect rate, process-level metrics like Cp/Cpk and process stability, and customer-level metrics like complaint rate and on-time delivery. Every serious quality management system runs on some combination of these.

Having spent three years as a Six Sigma Black Belt on the DMAIC side of the Johnson Controls headliner lines, and a decade rolling out MES and traceability across 900+ machines on four continents, I have seen every version of the quality scorecard imaginable. The number of metrics is rarely the problem. The honesty of the underlying measurement almost always is. This article is less about listing KPIs, more about separating the ones that drive real improvement from the ones that just make the monthly report look calm.

The three layers of quality metrics

Textbooks list dozens of quality metrics. In practice they collapse into three layers, and each layer exists for a different purpose. Mixing them up is where most quality scorecards become noise instead of signal.

A functioning quality scorecard uses all three. Product metrics tell you what came out today. Process metrics tell you whether what comes out tomorrow is predictable. Customer metrics tell you whether any of it matters to the people paying for it. Skip a layer and the picture lies.

The metrics that actually matter

Out of the full catalogue, five carry most of the weight in real improvement work. Everything else is either derivative or cosmetic.

First Pass Yield (FPY). The share of units that pass every inspection step the first time, with no rework and no second chance. It is the most honest product metric there is because it cannot be inflated by rework loops. A plant reporting 98 % scrap rate but 72 % FPY is a plant with a rework problem, and you would never see that from the scrap number alone.

Rolled Throughput Yield (RTY). FPY multiplied across all process steps. A ten-step line with 98 % FPY per step has an RTY of 82 %. RTY exposes compounding quality loss that single-step metrics hide. Most plants that track only final yield are shocked the first time they see their RTY.

Cpk. Process capability relative to specification limits, considering both variation and centring. A Cpk above 1.33 is the automotive baseline, above 1.67 for safety-critical characteristics. Cpk is the single metric that best predicts whether tomorrow will look like today. A high FPY with a low Cpk is a warning sign: you are currently making good parts, but not because the process knows how to.

PPM defects (parts per million). The industry standard for defect reporting in automotive and electronics supply chains. Used for both internal (production PPM) and external (customer PPM, 0-km PPM, field PPM). Useful precisely because it is blunt. 500 PPM is 500 PPM regardless of batch size, plant or product mix.

Complaint rate / customer PPM. The only metric on this list that the customer sets. Every other metric can be optimised internally. This one tells you whether the internal optimisation actually translated into something the customer experienced as quality.

If your scorecard contains these five, honestly measured, you have the instrumentation for real improvement. If it contains twenty others but not these, you have a reporting ritual.

Why most quality metrics lie

This is the part of the topic no textbook writes about and the part my book OEE: One Number, Many Lies is built around. The numbers are not wrong because people are dishonest. They are wrong because the measurement system makes honest measurement harder than slightly flattering measurement. Five patterns cover the majority of what I see.

1. FPY reported after rework. A part fails, gets reworked, passes on the second try, and gets counted as good. Technically not false, but it is no longer FPY. It is "final pass yield", and the distinction matters. Plants that conflate the two lose the ability to diagnose their actual first-time quality.

2. Cpk calculated on the wrong sample. Cpk on a single capability study done during a good week, from a hand-picked batch, under ideal conditions, is a number. Cpk tracked continuously from production data across shifts, operators and tool wear cycles is a different number, usually 20 to 40 percent lower. The second one is reality.

3. Defects that never get written down. If the operator fixes a small defect at the station without logging it, it never becomes a PPM number. This is not malice, it is friction. Paper defect forms, clunky terminals, and "don't stop the line" pressure all conspire to under-report. I have seen production PPM jump by a factor of three within a month of deploying an automatic defect capture system. The defects existed before. The measurement did not.

4. Complaint rate filtered through account management. A customer complains informally to their buyer, who talks to your key account manager, who agrees to credit the batch, and the event never enters the quality system. The real customer PPM is higher than the one in your dashboard.

5. The aggregation hides the story. A monthly quality report shows 450 PPM average. The detail shows 1,800 PPM on the night shift and 120 PPM on the day shift. The average is "fine". The night shift is not. Averages are where quality signals go to die.

Practical rule: the first sign that a quality measurement programme is improving is not better numbers. It is worse numbers. When under-reporting stops and corner-cutting becomes visible, the honest baseline appears, and it is almost always 20 to 50 percent worse than the previously reported baseline. That is the moment real improvement becomes possible.

What changes when quality metrics are captured automatically

The shift from manual, interval-based quality reporting to automatic, event-based capture is the single biggest upgrade a quality organisation can make. It touches every metric in the catalogue.

1. Defects captured where they happen. Rejection counters at the press, the welder, the end-of-line tester, tied to the running production order. No paper form, no shift-end tally, no lost events.
2. Cpk continuous and current. SPC calculated from live measurement data, not from a capability study done last quarter. Capability drift is visible in hours, not in a retrospective audit.
3. FPY, RTY and rework separated. When every pass through a station is logged individually, rework is visible as rework instead of disappearing into the yield number. The three metrics can finally be reported honestly.
4. Correlation with process data. A defect at the end of the line cross-referenced automatically with the temperature, torque and pressure at the upstream station at that exact cycle. Root-cause analysis in hours instead of the three-week investigation it used to be.
5. Closed loop into production. When the quality metric crosses a threshold, the MES triggers a sampling rule, an alert, or a process stop, before the next 500 parts are out. The metric stops being a report and becomes a control signal.

That last point is where quality metrics earn their name. A number that sits in a PowerPoint is telemetry. A number wired into a decision is a metric.

A real case: Neoperl

Neoperl is an international manufacturer of precision water-flow components, headquartered in Müllheim with plants in Bulgaria, the UK and Italy. The starting point was the familiar one: fully automated assembly lines producing at high cycle rates, PLC-level alarm data that nobody was using systematically, and a quality organisation running on periodic sampling and post-hoc analysis of customer returns.

The SYMESTIC deployment started with a four-week proof of concept on a single line. The scope was narrow by design: capture PLC alarms automatically, categorise them, and correlate them with downstream quality defects. After the PoC, three lines went live, then more, continuously. The system was framed from day one as a continuous-improvement tool, not as a monitoring dashboard.

The measured results after the first year across the connected lines:

• 15 % less scrap, driven almost entirely by correlation of PLC alarms with quality defects and the targeted actions that followed
• 10 % fewer stops, from automatic capture and classification of short-stop causes that operators had previously not logged
• 8 % higher availability, through structured root-cause work on the top alarm categories
• 15 % productivity gain, as the combined effect of the three above

None of those numbers came from new equipment or new inspection steps. They came from quality metrics that were finally honest, tied to the process events that caused them, and visible in time to act on.

FAQ

What is the difference between FPY and RTY?
FPY is the first-pass yield of a single process step. RTY is FPY multiplied across every step of the line. A ten-step process with 98 % FPY per step has an RTY of 82 %. RTY exposes compounding quality loss that single-step metrics hide, which is why it is the more honest whole-line measure.

What is a good Cpk value?
1.33 is the common automotive baseline for standard characteristics. 1.67 is the baseline for safety-critical or heavily-regulated characteristics. Below 1.0, the process is not capable of consistently meeting specification and needs work. Above 2.0, you are either genuinely excellent or your specification limits are too wide, and both deserve a second look.

Do we need Six Sigma to run quality metrics properly?
No. Six Sigma provides a rigorous framework (DMAIC) and a statistical toolkit, and it is genuinely useful for variation-reduction projects. But the core quality metrics (FPY, RTY, Cpk, PPM, complaint rate) exist independently and can be run well in plants that never adopt a formal Six Sigma programme, provided the measurement is honest and the data is captured automatically.

How often should quality metrics be reviewed?
The metric and the review cadence must match. Station-level FPY and defect counts: continuously, visible on the shopfloor. Cpk and process stability: daily for active processes, weekly minimum. Complaint rate and customer PPM: weekly. Board-level quality review: monthly. Quarterly quality reports are a reporting ritual, not a control mechanism.

What is the connection between quality metrics and OEE?
Quality is the Q in OEE. The Quality Rate component measures the share of produced parts that are good first pass, which makes it the OEE-equivalent of FPY. Plants that capture quality metrics automatically typically see their reported OEE drop by 3 to 7 percentage points when under-reported rejects and rework finally appear in the number. That drop is the honest baseline.

How does SYMESTIC support quality metrics?
Automatic capture of defects, scrap and rework at station level, tied to the running production order. Continuous Cpk and SPC on live measurement data. Correlation between quality events and upstream process parameters. Bidirectional integration with CASQ-it, Böhme & Weihs, and other quality systems, so the MES can trigger sampling rules and quality actions automatically. See SYMESTIC Production Metrics.

Related: OEE · Production Quality · Process Improvement · Production Data · MES · SYMESTIC Production Metrics

About the author

Christian Fieg

Head of Sales at SYMESTIC. 25+ years in manufacturing including Johnson Controls, Visteon, iTAC and Dürr. Six Sigma Black Belt with three years on the DMAIC side of the Johnson Controls headliner lines. Led global MES and traceability rollouts across 900+ machines in China, Mexico, USA, France, Tunisia and Russia. Author of OEE: One Number, Many Lies (2025). · LinkedIn