Back to the Archive

LibraryFoundational AI: Do's and Don'ts3 min read

Use AI to check the cutoff, not just the easy cases

When a work rule changes at a number or deadline, ask AI to draft examples just below, at, and just above it. Check each result against the rule you own.

An office worker checks AI expense cases of $99, $100, and $101 against a policy requiring approval for $100 or more.

A rule that says “after 48 hours, route the request to a manager” has an awkward question built into it: what happens at exactly 48 hours? A few ordinary examples might never reveal that the rule is being applied inconsistently.

Try boundary-value testing. In software testing, it means checking values at a rule’s edge and just on either side. The ISTQB syllabus describes three-value boundary testing as checking the value before, on, and over a boundary, with more thorough checks appropriate when the risk is higher. Its testing syllabus explains the method and its limits. Using AI to draft these examples is a practical adaptation; it does not mean the model verifies your process. NIST’s voluntary AI Risk Management Framework also calls for testing AI systems before deployment and regularly during operation. Read the NIST measurement guidance.

Try it on one cutoff

Choose a low-risk rule with a clear numeric or time threshold. First, copy the authoritative wording and decide what the correct result should be. Don’t ask AI to decide what the policy ought to say.

Illustrative example: Suppose an internal rule says a routine support request goes to a manager when it has been open for 48 hours or more. A person who owns that rule can set the expected results:

| Request age | Expected routing | | ----------- | ------------------------- | | 47 hours | Keep in the routine queue | | 48 hours | Route to a manager | | 49 hours | Route to a manager |

Ask AI to draft cases around the boundary, not to decide how the boundary works:

The rule is: route a request to a manager when it has been open for 48 hours or more. Draft three test cases at 47, 48, and 49 hours. For each, show the input and the routing result implied by the rule. Do not invent exceptions or change the rule. Mark anything ambiguous. Do not update or route a real request.

This is an illustrative prompt, not a tested recipe. Check that AI used the exact cutoff, units, and inclusive or exclusive wording. Compare each result with the rule owner’s expected result. If the policy also depends on business hours, request type, or holidays, list those conditions separately and decide whether they change the cases.

Know what the check tells you

If an answer is wrong at 48 hours, you have found a discrepancy worth investigating. It could be a prompt issue, an unclear policy, or a system behavior that needs a real test. The AI’s result alone does not identify the cause.

Boundary checks are useful only when the rule has an ordered cutoff and you know the smallest meaningful unit. They won’t uncover every exception, test every combination of conditions, or show that a workflow behaves the same in production. For a consequential routing rule, verify the behavior in an approved test environment and have the process owner review the result before relying on it.

Pick one cutoff your team applies repeatedly. Write down what should happen just before it, at it, and just after it. Then use AI to draft those cases and check them against the rule you actually own.

Sources: ISTQB boundary value analysis guidance and NIST AI RMF measurement guidance.

Sources

Every claim above traces back to one of these. Go read them yourself.

  1. 01
  2. 02
    AI RMF Core

    National Institute of Standards and Technology / airc.nist.gov