The Test Said Pass. Was It the Right Test?
A pass result is meaningful only when the product revision, test programme, limits, equipment and execution evidence are all valid for that unit.
The short answer
An electronics test result is valid only when it connects the tested unit to the correct product revision, authorised programme and limits, suitable equipment, complete execution and immutable outcome history. “Pass” without that context is a colour, not evidence.
Test programme version control should prevent the wrong test from running and preserve exactly what happened when the result is later investigated.
The unit passed in twelve seconds
The station scanned the serial, executed a programme and returned pass. The operator moved the unit forward. Days later, engineering discovers that the programme was released for the previous product revision.
The measured values may look normal. The system did perform a test. Yet the factory cannot claim that the intended requirements were verified.
This is why test control begins before execution and extends beyond the final result.
Treat the programme as controlled product data
A test programme has identity, revision, owner, approval, effectivity and change history. Limits, sequence, scripts, fixture configuration and firmware dependencies may all change its meaning.
Store the released version in controlled access. Development copies should not appear as ordinary production choices. Superseded versions remain available for history but blocked from unauthorised use.
Change reason matters: product revision, defect correction, false-failure reduction, customer requirement or equipment update. Each can affect retrospective interpretation.
Resolve product to programme automatically
At the station, the system should use scanned serial or work-order identity to determine the valid product revision and authorised programme.
Manual selection increases risk, particularly where model names are similar. If selection is unavoidable, present only eligible options and require confirmation for exceptional use.
The programme should verify its own compatibility where possible. A mismatch must stop or create an authorised deviation, not merely display a warning operators learn to ignore.
Control test limits separately from display
Limits may be embedded in code, stored in configuration or supplied by product data. Wherever they live, the executed revision must be identifiable.
Changing a dashboard threshold does not necessarily change the production test. Conversely, changing a programme can alter acceptance without updating a report definition.
Create one controlled relationship between requirement, limit and execution. Preserve the actual measured value where risk and volume justify it.
Equipment and fixture are part of context
Programme validity does not guarantee measurement validity. Record the station, equipment and fixture used, with calibration or verification status where applicable.
A worn connector, damaged probe or unsuitable range can create false pass or false fail. Fixture maintenance and golden-unit checks should be controlled according to risk.
The Gauge Was Calibrated. Was It Valid for This Run? explores equipment suitability and out-of-tolerance impact.
Preserve every attempt
Do not overwrite a failure when the unit later passes. The sequence of attempts reveals intermittent conditions, repair effectiveness, programme instability and hidden capacity use.
Each result should include time, programme revision, limits, equipment, measured values or failure codes, and relationship to repair or retest.
The latest accepted result determines disposition. The complete history supports quality learning.
Manage aborted and incomplete tests
A test that stops halfway is not pass or fail. It is incomplete and may require controlled restart.
Record why execution aborted: equipment communication, power loss, operator action, fixture issue or product response. Define whether completed steps can be reused or the full sequence must repeat.
Prevent partial data from being interpreted as a valid result simply because no failing limit was recorded.
Separate symptom from cause
The test identifies observed behaviour. Diagnosis determines why it occurred. Keep failure code, diagnostic conclusion and repair action distinct.
This lets the same symptom map to several causes and supports corrective action based on confirmed evidence. A generic “test fail” code is insufficient for repeated improvement.
Use structured codes for analysis and concise notes for unusual context.
Validate programme changes before release
Test the changed programme against known good, known failure and boundary conditions. Confirm that required functions execute and data is recorded correctly.
Use representative product revisions and fixtures. Compare false-failure and escape risk. Independent approval may be appropriate for high-risk products.
Release should identify effective work orders, serial ranges or dates. Open production requires an exposure decision when the change corrects a significant gap.
Assess retrospective exposure
When a wrong programme, limit or invalid station is discovered, identify every unit tested under the affected combination and time window.
Then decide whether evidence can support release, whether retest is required, and which shipments or customers are exposed. The scope should come from connected execution history rather than broad assumption.
This relationship to containment is examined in One Failed Unit. How Far Should Containment Go?.
Protect test data integrity
Control who can release programmes, change limits, rerun tests and alter dispositions. Corrections should preserve original evidence and reason.
Synchronise station time and ensure offline operation has a defined reconciliation method. Duplicate or missing serial events can create false history.
Retain data according to customer, regulatory, reliability and commercial needs. The retention rule should apply to raw detail and operational summary appropriately.
Measure the test process
Track first-pass rate, repeat attempts, aborted tests, no-fault-found repairs, programme mismatches, false failures, station downtime and result-upload delays.
Analyse by product revision, programme, fixture and station. A yield shift after a programme change may reflect measurement change rather than production change.
Review test capacity consumed by repeats, not only nominal cycle time.
Connect test to the quality thread
The pillar The Product Passed Every Gate. Why Did the Customer Still Find the Failure? places test evidence within product, material, process and field history.
A test result should support shipment release today and investigation years later. That requires stable identity and context, not merely storage of a final flag.
Field failures should be compared with original measured values and programme revision to improve limits and detection strategy.
Run a test-evidence audit
Select one recently shipped serial. From the result alone, identify product revision, programme and limit revision, station, fixture, time, all attempts and final authorisation.
Then select one programme change and find every unit tested before and after effectivity. Confirm that the boundary is exact.
Any answer requiring filenames, technician memory or station logs outside the controlled record identifies a gap.
Control station deployment
Releasing a programme centrally does not prove every station received it correctly. Track deployment by station, version, time and acknowledgement. Remove or block superseded packages from production selection.
Where networks are unreliable, use signed or checksum-verified packages and a controlled offline update. Reconcile results when connectivity returns.
Govern manual overrides
Some products need engineering mode, skipped steps or temporary limits during investigation. Restrict these functions by role and record serial, reason, authoriser and result.
An override should never create an ordinary production pass without visible exception status. Review override frequency; repeated use often indicates programme or process weakness.
Use field feedback to challenge coverage
When a customer failure was not detected internally, compare the field mode with test sequence, stress, limits and measured values. Ask whether the test could detect the condition under realistic variation.
Update coverage through validated change rather than simply tightening every limit. Excessive sensitivity can create false failures and hide the real relationship between test and reliability.
Frequently asked questions
Is storing pass or fail enough?
Usually not. At minimum, preserve product, serial or batch, programme revision, equipment, time and result. Higher-risk products may require measured values and full sequence.
Can operators choose a test programme?
Prefer automatic resolution from product identity. If manual choice remains, restrict eligible programmes and control exceptions.
Should retest replace the first failure?
No. Preserve every attempt. Final disposition can be pass while history still shows failure, repair and retest.
What happens when limits change?
Apply controlled effectivity and assess whether previously tested units are exposed. Do not overwrite the old limit history.
Is machine integration sufficient?
It improves capture, but integration must still connect the correct unit, programme, limits, equipment and operational disposition.
Make pass mean the right test passed
The factory should be able to defend more than the colour shown at the station. It should prove that the correct requirement was tested through an authorised method on valid equipment for this exact product.
That context turns a result into quality evidence.
It also makes future improvement safer: engineers can change coverage, limits and sequence while preserving exactly which units followed every previous rule.