At every stage of quality measurement there is a human decision: which category does this finding fall into, what severity does it carry, is the text fit for its purpose? The number makes these decisions visible and consistent, but it does not remove them. What makes measurement reliable is not hiding the decisions but giving reasons for them.
The decision is unavoidable
There is a human decision in every measurement step. The number does not form by itself. The fact is accepted.
Category decision
Which type a finding falls into is an expert decision. The definitions give guidance. The decision still belongs to a person.
Severity decision
The consequence the finding will produce is evaluated. This evaluation takes experience. It cannot be automated.
The decision to count a finding
Whether a difference is a finding or a preference is also a human decision. The line is fine. Discipline is needed.
The role of the number
The number supports the decision and makes it comparable. It does not replace the decision. Its role is auxiliary.
Writing down the reasoning
The reason for the evaluation decision is recorded. A decision without reasons cannot be audited. A record is kept.
Auditability
Written reasoning allows the decision to be examined later. The discussion proceeds objectively. Transparency increases.
The need for subject expertise
The person evaluating must know the subject. Someone who does not may count what is correct as an error. The condition is firm.
The language competence condition
The evaluator must be strong in the target language. Insufficient competence produces false findings. The criterion is applied.
The share of experience
An experienced evaluator handles borderline cases more consistently. The difference can be measured. The investment is worthwhile.
Managing bias
Knowing who translated the text may affect the decision. Blind evaluation reduces this risk. The method is preferred.
The effect of first impression
Flaws on the first page of the text may make the rest of the evaluation harsher. The effect is known. Care is taken.
The effect of fatigue
Attention falls in long evaluations. Breaks are taken. The result is protected.
Second opinion
In debatable jobs a second evaluator is brought in. The decision is strengthened. The procedure is written down.
Settlement procedure
If two evaluators cannot agree, a third person is consulted. The process is defined. The decision is concluded.
Relationship with client evaluation
Findings from the client also pass through the expert filter. Not every request is a finding. The distinction is preserved.
Strengthening through training
Evaluators are trained with shared examples. Consistency increases. The investment is made.
Putting the decision on record
The decisions made are added to the example bank. The same case is not debated again. Knowledge accumulates.
Evaluation environment
The examination is carried out in an uninterrupted environment. Divided attention misses findings. The condition is provided.
Evaluation time
Realistic time is set aside for the examination. A squeezed evaluation stays superficial. The time is written into the plan.
Separating the number from the decision
The value of a measurement system lies not in removing the human decision but in making that decision repeatable and reasoned; an expert decision that is hidden is arbitrary, while one that is written down is auditable. That is why the reasoning is always recorded. Transparency is the basis.
Limits of the approach
No arrangement can make an expert decision entirely objective. The aim is to keep the difference manageable. The limit is accepted.