Acceptance Testing: Why PASS Does Not Automatically Mean Technical Acceptance

Why a PASS result is not enough for technical acceptance: criteria, traceability, native files, sampling, retesting, independent verification and commissioning.

Check it out!

A PASS result in an acceptance test means only that the tested item met the criterion recorded in that procedure, under that condition and in that execution. By itself, it does not prove that the entire contracted scope is compliant, that the sample represents the population, that the method was correctly applied, that the instrument was fit for use, that the result file is traceable, that outstanding items were closed, or that the documentation required for acceptance is complete.

Technical acceptance therefore requires a chain of evidence that goes beyond the word “PASS” printed in a report. It is necessary to relate requirement, tested object, procedure, acceptance criteria, test conditions, instrument, result, responsible person, witnessing, source files, deviations, retests, final documentation and formal decision. If any link in this chain is weak, the result may be technically true and still insufficient to support acceptance.

This distinction is especially important in engineering contracts, critical systems and technology supplies. Equipment may pass an isolated test and still fail when integrated; a link may show an approved result but be associated with the wrong physical identifier; a system may operate during a demonstration and still lack backup, final configuration, As-Built documentation, licenses, manuals or performance evidence; a sample may pass without demonstrating the condition of the overall population.

The correct question, therefore, is not simply “did the test pass?”. The engineering question is: was the contractual requirement demonstrated by valid, traceable, sufficient and representative evidence to support the acceptance decision?

What a PASS result actually proves

A test has a limited scope. It verifies a condition defined by a method and a criterion. If the test was properly planned and executed, a PASS result proves that that specific verification met the applicable limit or condition.

The strength of the evidence depends on four inseparable elements:

• the requirement that must be demonstrated;
• the method used to verify it;
• the criterion that distinguishes conformity from nonconformity;
• the record that allows the execution to be reconstructed.

IEC 62381:2024 structures FAT, FIT, SAT and SIT precisely as planned activities intended to demonstrate compliance with applicable specifications, with previously established scope, responsibilities, procedures and checklists. The logic matters: the test demonstrates a requirement; it does not replace the specification that originated the requirement or the owner’s acceptance decision.

Layer	Question that must be answered
Requirement	What should the object do or comply with?
Criterion	What condition characterizes conformity?
Method	How will that condition be verified?
Execution	Was the test performed under the specified conditions?
Result	What value, state or response was observed?
Evidence	Can the way the result was obtained be demonstrated?
Coverage	Does the test represent the item, system or population intended for acceptance?
Decision	Does the evidence set support release, conditional acceptance, retesting or rejection?

The word PASS appears in only one of these layers. Technical acceptance depends on the whole set.

Test result, conformity and acceptance are different decisions

It is useful to separate three concepts that are often confused.

Test result is the output of a verification. It may be a measured value, a functional response, an observed sequence or a PASS/FAIL classification.

Conformity is the conclusion that the result meets the applicable requirement or criterion. To declare conformity, it is necessary to know the reference against which the result was compared.

Technical acceptance is a decision by the contracting party or the authority defined by governance, made after reviewing the set of requirements, evidence, outstanding items and deliverables applicable to the contractual milestone.

A system may have hundreds of approved tests and still not be ready for acceptance because documents, integrations, retests, corrections, training, As-Built documentation, Data Book, licenses, backups or closure of nonconformities are still missing. The reverse is also relevant: an isolated failure does not necessarily invalidate the entire system, provided that its criticality, extent and treatment are technically assessed and that the contract permits the corresponding treatment.

The delivery chain already established in A3A content separates installation, operation, testing, technical delivery, acceptance and handover. This separation prevents a partial event from being treated as evidence of overall completion.

The identifier of the tested object is part of the result

A report without unequivocal identification of the object may be technically useless for acceptance. Knowing that “a point,” “a cable,” “a device” or “a function” passed is not enough if the record cannot be related to the physical or logical element actually installed.

Minimum traceability may include, depending on the type of system:

• equipment TAG;
• serial number;
• circuit code;
• panel and position identification;
• link origin and destination;
• logical or physical port;
• IP address or asset identifier;
• physical location;
• revision of the applicable drawing or list;
• work package or system to which the item belongs.

In structured networks, for example, the association among outlet identifier, patch panel, port, outlet and certification file is as important as the measured value. In electrical systems, the test must be associated with the correct circuit or equipment. In automation, functional evidence must make clear which logic, version, I/O, loop or scenario was verified.

If identification is shifted, duplicated or based only on the position of a row in a spreadsheet, the owner may have many records and very little real traceability.

Acceptance criteria must exist before the result

A criterion created after the test risks adapting the rule to the observed result. Acceptance criteria should therefore be defined in the specification, design, procedure, PIT/ITP, commissioning plan or another applicable contractual document before execution.

A robust criterion should state, where relevant:

• parameter to be verified;
• unit;
• minimum or maximum limit;
• tolerance;
• operating condition;
• test duration;
• number of cycles;
• expected response;
• blocking condition;
• retest rule;
• rule for acceptance with outstanding items, when permitted;
• evidence that must be preserved.

Expressions such as “normal operation,” “satisfactory test,” “no anomalies” or “approved result” are weak when there is no reference that allows the decision to be repeated.

Management of acceptance criteria begins when requirements are defined and continues through commissioning. This prevents construction, inspection and the supplier from using different references to assess the same delivery.

The procedure must also be valid

Two tests with the same name can produce evidence of very different quality. The procedure must specify the execution method with a level of detail consistent with the risk and technology.

Before considering the result, it is necessary to verify:

• procedure revision used;
• test scope;
• prerequisites;
• initial system configuration;
• instruments and tools;
• sequence of steps;
• measurement points;
• loads or stimuli applied;
• observed parameters;
• tolerances;
• interruption criteria;
• failure handling;
• recording method;
• condition for repetition.

A result obtained under an outdated, incomplete or unapproved procedure may not demonstrate what the contract requires, even when it is shown as PASS.

Inadequate preconditions can produce a misleading PASS

A test is representative only when the execution conditions match the scenario intended for validation. A simplified demonstration can conceal problems that arise under load, integration, redundancy, contingency or actual operation.

Some relevant preconditions are:

• system at the correct revision/configuration;
• installation completed for the tested scope;
• permanent power supply or an equivalent controlled condition;
• required interfaces available;
• correct network environment;
• instruments stabilized;
• associated sensors and actuators;
• databases and parameters loaded;
• active permissions and licenses;
• environmental conditions recorded when they affect the result;
• previous outstanding items classified.

Equipment that works on a test bench does not necessarily demonstrate behavior in the final environment. This is precisely why FAT and SAT have different roles, and why integrated tests may be required even after satisfactory individual results.

Instrument, calibration and metrological traceability

When the conclusion depends on measurement, the reliability of the evidence depends on the resource used. The instrument must be suitable for the quantity, range and accuracy required by the test. When metrological traceability is a requirement or is needed to provide confidence in the result, evidence of calibration or verification compatible with the intended use must exist.

Attaching any certificate is not enough. It is necessary to verify whether:

• the instrument identified in the report is the same one used in the test;
• the certificate corresponds to the correct serial number;
• the validity was current on the test date, when applicable to the adopted management system;
• the measurement range is suitable;
• uncertainty and resolution do not make the comparison irrelevant;
• there are no use restrictions that compromise the result;
• the configuration and accessories used are compatible with the method.

The ISO 9001 Auditing Practices Group guidance on measurement traceability reinforces the need for objective evidence when traceability is necessary to trust the validity of results.

The record must preserve more than the word PASS

A report containing only item, date and PASS makes auditing, retesting and later investigation difficult. The evidence must allow the test to be reconstructed without relying on the memory of the people involved.

Depending on criticality, the record may include:

• object identification;
• procedure and revision;
• requirement or criterion;
• raw measured values;
• applicable limits;
• date and time;
• executor;
• witnesses;
• instrument;
• system condition;
• screenshots;
• logs;
• contextualized photographs;
• files exported by the equipment;
• observed deviations;
• field notes;
• signature or digital approval;
• reference to RNC/NCR or punch item when applicable.

The calculated result is useful, but the data that originated it are what allow its integrity to be verified.

A native file can be the difference between evidence and a simple printout

When test equipment or software produces a native file, preserving that file can significantly increase auditability. PDF and spreadsheet formats are excellent for reading and consolidation, but they may not retain all metadata, curves, parameters, logs and source information available in the instrument file.

For this reason, in higher-criticality systems, the test plan or document matrix may simultaneously require:

• readable PDF report;
• native instrument or software file;
• structured export, when available;
• checksum or integrity control when justified by risk;
• identification of the tested equipment;
• link to the document repository;
• version control.

The requirement must be proportional and predefined. Not every test generates a native file and not every contract needs a checksum. The principle is to preserve sufficient evidence so the owner can audit the result without depending exclusively on a printout produced by the executor.

Sampling: a PASS may represent only a small portion

One of the most common mistakes is extrapolating a sample result to the entire population without a statistical, technical or contractual criterion. Test extent must be defined according to risk, applicable standard, system type, process repeatability and consequence of failure.

Three strategies are common:

Strategy	Typical application	Limitation
100% of items	critical requirements, individual certifications, mandatory functions	greater effort and time
Defined sampling	repetitive items with a controlled process	requires a selection criterion and expansion rule
Scenario-based testing	integrated systems and complex functions	coverage depends on scenario quality

The sample must be identifiable. In a set with hundreds of items, selecting only those with easier access or those previously prepared reduces the strength of the evidence. Independent, random, stratified or risk-directed sampling can provide a more reliable view, depending on the case.

If an independent verification finds a failure in an item previously approved, the response should not be merely to correct that item. It is necessary to assess whether the finding indicates an isolated problem or a systemic failure and, when justified, expand the sample or repeat the test campaign.

Retesting must close the cause, not merely produce a new PASS

When an item fails, the correct sequence is not simply to run the test again until PASS appears. Retesting must come after analysis of the deviation.

The minimum chain is:

1. record the failure;
2. preserve the original evidence;
3. identify the probable or confirmed cause;
4. perform the authorized correction;
5. assess impact on similar items or interfaces;
6. define the extent of retesting;
7. execute again under a controlled procedure;
8. record the result and link it to the original deviation;
9. verify closure of the nonconformity.

If a configuration change solved the problem, that change must be reflected in the baseline, As-Built, backup and applicable records. Otherwise, the test passes but the delivered documentation continues to describe a system different from the one actually corrected.

Witness test, Hold Point and independent verification

Evidence quality increases when the owner does not rely exclusively on the self-declaration of the party that performed the work. This does not mean fully repeating every test, but rather structuring supervision mechanisms proportional to risk.

A Witness Point allows the contracting party, inspection team or Owner’s Engineering to observe a given test. A Hold Point prevents progress until the specified condition is released. Independent verification may repeat part of the tests or perform selected counterchecks.

The choice depends on criticality, repeatability and failure cost. Independent verification is especially useful when:

• the result supports a significant payment;
• the activity will be concealed or become inaccessible;
• the supplier alone controls evidence generation and interpretation;
• there is a history of inconsistency;
• the system is critical;
• a later failure would have high operational impact;
• acceptance closes an important portion of contractual responsibility.

Independence does not eliminate the contractor’s responsibility for quality. It increases the owner’s confidence in the decision.

Individual PASS does not replace integrated testing

Complex systems may operate perfectly in isolation and fail at interfaces. Therefore, unit tests, FAT, SAT and component verifications do not necessarily replace integration tests and operating scenarios.

Examples of failures that may escape individual tests:

• loss of communication between systems;
• incorrect command sequence;
• timeout or latency;
• inadequate alarm priority;
• time inconsistency;
• loss of redundancy;
• unexpected behavior during power loss;
• incorrect recovery after restoration;
• incompatible access permissions;
• failover failure;
• data not propagated between platforms;
• inadequate response in an emergency scenario.

IEC 62381:2024 includes FIT and SIT precisely to address integration at the factory and site. The principle is general: conformity of parts does not automatically prove performance of the system as a whole.

The test must be reconciled with final documentation

Technical closeout cannot keep tests in one silo and As-Built documentation in another. The acceptance record must be reconciled with the final configuration delivered.

This includes, depending on the scope:

• revised drawings;
• diagrams;
• point lists;
• asset inventory;
• firmware and software;
• parameters;
• addressing;
• licenses;
• backups;
• cable lists;
• tags;
• certificates;
• inspection reports;
• RNCs/NCRs;
• punch list;
• procedures;
• manuals;
• training;
• Data Book.

If the test was performed on a temporary configuration and the final documentation describes another one, the PASS has lost part of its ability to support delivery. Traceability must connect what was tested to what remained installed.

Data Book should not be a repository of results without context

File quantity is not equivalent to document quality. A Data Book can contain hundreds of reports and still fail to answer which requirements were tested, which items failed, which were retested and which final version should be considered.

A mature acceptance structure should allow logical navigation:

requirement → item → procedure → execution → result → deviation → correction → retest → final evidence → acceptance.

When this chain is maintained in a GED/EDMS or another controlled source, the operations team receives much more useful information than a collection of disconnected PDFs.

Contractual acceptance is broader than test approval

For contracts governed by Brazilian Law No. 14,133/2021, Article 140 distinguishes provisional and final acceptance and establishes that works and services are accepted upon verification of compliance with technical and contractual requirements. The same provision states that tests, examinations and other proofs required by official technical standards, unless otherwise provided, are borne by the contractor.

This reinforces an important distinction: the test is evidence for acceptance; it is not acceptance itself. The decision must consider the contract as a whole.

Technical acceptance may also include, in addition to tests:

• document deliverables;
• warranties;
• correction of outstanding items;
• training;
• spare parts;
• legal documentation;
• ART/RRT when applicable;
• As-Built;
• Data Book;
• manuals;
• licenses;
• operation and maintenance criteria;
• remaining obligations.

In private contracts or contracts subject to their own regulations, the logic remains contractual: the applicable instrument must be the reference, without automatically applying rules from Brazilian Law No. 14,133 to regimes not governed by it.

How to turn test results into an acceptance decision

A technically defensible decision can be structured through gates. The responsible person does not need to review every file in the same way; they need to verify that the necessary evidence was produced, that exceptions are identified and that blocking items were resolved.

Situation	Recommended treatment
Valid, traceable PASS with no associated outstanding item	eligible to support acceptance
PASS with incomplete documentation	keep document outstanding item open
PASS without unequivocal object identification	do not use as conclusive evidence until reconciled
PASS without a previously defined criterion	submit for technical analysis; do not presume acceptance
PASS from an insufficient sample	expand coverage according to risk/criterion
PASS after correction without linkage to the original failure	complete retest traceability
Individual PASS while integration remains unverified	keep the integration/commissioning gate open
FAIL or critical deviation	block progress as defined by the test plan
Non-blocking outstanding item	treat according to classification and contractual rules

The goal is not to create signature bureaucracy over every result. It is to prevent partial data from being transformed, under schedule pressure, into a conclusion broader than what it technically demonstrates.

The criticality of outstanding items must be defined

Not every outstanding item has the same effect on acceptance. Classification should consider safety, functionality, performance, operational risk, maintenance, documentation and contractual impact.

A simple taxonomy may separate:

• blocking: prevents operation, safety, essential performance or contractual compliance;
• major: requires correction and validation before final acceptance, although it may not prevent subsequent tests;
• minor: does not compromise an essential function and may receive controlled treatment under the contract;
• documentary: missing evidence or record that must be completed;
• informational: observation with no corrective action required.

This classification must be associated with a responsible party, deadline, closure evidence and authority for closure.

Independent testing does not mean adversarial testing

Independent verification should be understood as a confidence mechanism, not an attempt to find faults at any cost. The goal is to reduce conflicts of interest and increase evidence quality.

A well-designed independent campaign may:

• select samples without influence from the executor;
• repeat critical measurements;
• verify identifier consistency;
• compare native files with exported reports;
• review instrument parameters;
• observe procedure execution;
• check consistency between testing and As-Built;
• validate closure of nonconformities;
• recommend sample expansion when inconsistencies arise.

The result may fully confirm the contractor’s work. That is also a valuable outcome: it gives the owner additional evidence that the original campaign is reliable.

The role of commissioning in this chain

Commissioning organizes progressive verification of delivery from requirements and planning through testing, integration, documentation, readiness and handover. It prevents all validation from being concentrated at the end.

IEC 62337:2012 establishes phases and milestones between completion of erection and plant acceptance by the owner in the context of electrical, instrumentation and control systems in the process industry. The specific application must be adapted to the type of project, but the principle is relevant to other systems: intermediate states of completion and readiness must be demonstrated before acceptance.

A PASS in SAT may therefore release the next stage and still not mean final acceptance. An approved integrated test may demonstrate functional readiness and still depend on final documentation. A system with a controlled punch list may move into assisted operation when the plan permits, even while some contractual obligations remain open.

The value of commissioning lies precisely in turning these milestones into explicit decisions.

Experience: when evidence must be auditable by third parties

The warning sign appears when there is a large volume of approved results, but the owner’s team cannot reliably reconstruct which item was tested, by which method, under what condition, with which source file and against which criterion.

The risk is accepting a conclusion statistically or documentarily broader than the available evidence. This may happen because of incorrect identification, report exports without native files, retests without traceability, inadequate sampling or a simple disconnect between field execution and documentation.

The control barrier is to separate production of evidence from validation of evidence. The contractor remains responsible for performing and documenting its tests; the owner, inspection team or Owner’s Engineering assesses whether the set is sufficient for the decision. When justified by risk, independent counterchecks increase confidence.

The lesson that should return to the next contract is objective: criteria, file formats, test extent, sampling, witness points, retest rules, identification, documentation and acceptance conditions must be defined before execution.

Final considerations

PASS is a result; acceptance is a decision. Between the two there is an engineering evidence chain that must remain intact.

The owner must be able to answer what was tested, against which requirement, by which method, under which conditions, with which instrument, by whom, with which record, with what coverage, which deviations existed and which final version of the system was actually delivered. When those answers are available and reconciled, the test result gains strength as evidence.

When they are not, the number of approved reports may create only an appearance of control.

Best practice is to design acceptance from the beginning: verifiable requirements, predefined criteria, PIT/ITP, procedures, source files, traceability, witness testing, nonconformity management, retesting, commissioning, As-Built and final documentation working as a single evidence chain.

Technical references

[1] INTERNATIONAL ELECTROTECHNICAL COMMISSION. IEC 62381:2024 — Automation systems in the process industry — Factory acceptance test (FAT), site acceptance test (SAT), and site integration test (SIT). Geneva: IEC, 2024. Available at: https://webstore.iec.ch/en/publication/67572

[2] INTERNATIONAL ELECTROTECHNICAL COMMISSION. IEC 62337:2012 — Commissioning of electrical, instrumentation and control systems in the process industry — Specific phases and milestones. Geneva: IEC, 2012. Available at: https://webstore.iec.ch/en/publication/6871

[3] BRAZIL. Law No. 14,133, of April 1, 2021. Public Procurement and Administrative Contracts Law, especially Article 140. Available at: https://www.planalto.gov.br/ccivil_03/_ato2019-2022/2021/lei/l14133.htm

[4] ISO 9001 AUDITING PRACTICES GROUP. Guidance on Measurement Traceability. Geneva: ISO/IAF. Available at: https://www.iso.org/files/live/sites/tc176sc2/files/documents/ISO%209001%20Auditing%20Practices%20Group%20docs/Auditing%20to%20ISO%209001%202015/APG-MeasurementTraceability2015.pdf

Frequently asked questions

Does a PASS result mean the equipment or system is accepted?

No. PASS demonstrates that a specific verification met the applied criterion. Technical acceptance also considers coverage, traceability, documentation, outstanding items, integrations and other contractual requirements.

What is the difference between acceptance testing and technical acceptance?

Testing produces evidence about specific requirements. Technical acceptance is the formal decision made from the full set of evidence, deliverables, outstanding items and criteria applicable to the contractual milestone.

Is the test PDF file sufficient?

It may be sufficient in some cases, but when the instrument or software generates a native file and criticality requires auditability, the source file, metadata and link to the tested item should also be preserved.

When should independent verification be performed?

When risk, criticality, financial impact, difficulty of later access, a history of inconsistencies or reliance on the executor’s self-declaration justify an additional verification layer.

Does an approved SAT close commissioning?

Not necessarily. SAT verifies the system in the installation environment, but the plan may require integrated tests, closure of outstanding items, documentation, As-Built, training, handover and other gates before final acceptance.

How should an item that failed and later passed retesting be handled?

The original failure must remain recorded, with cause, correction, impact on similar items, retest extent and linkage to the new result. The new PASS must not erase evidence of the previous nonconformity.

Complementary technical materials

Related solutions

• Requirements, Evidence and Acceptance Criteria Management

Related services

• Technical Testing: verification, performance, conformity and acceptance
• Engineering Commissioning: planning, testing, readiness and handover
• Technical Audit of Data Book and Final Engineering Documentation

Core content on the topic

• FAT and SAT: definitions, differences, integrated testing and acceptance criteria
• Engineering Acceptance Criteria: requirements, evidence and technical validation
• An installed system is not a delivered system: physical completion, technical delivery and acceptance

Related technical content

• Inspection and Test Plan (PIT/ITP): control points, criteria and records
• Independent QA/QC in Engineering: when the Owner needs a third party
• Engineering Documentation as a Condition for Measurement and Technical Acceptance
• Commissioning: Complete Guide to Planning, Testing, Acceptance and Handover