Automated Query Lifecycle Management: Raise, Route, Resolve, Close

A clinical query is not an event; it is a long-lived, regulated electronic record that moves through a defined lifecycle — raised against a discrepancy, routed to an owner, answered by a site, reviewed by a CRA, and finally closed or reissued — with every transition attributable, contemporaneous, and permanent. Managing that lifecycle by hand across thousands of subjects produces the failure that keeps clinical data managers awake before a database lock: duplicate queries for the same discrepancy, aging clocks nobody watches, and a closure with no evidence the underlying data was ever fixed. This guide sits inside the broader Clinical Query Generation & Discrepancy Management discipline and is the operational backbone that connects Automated Clinical Query Generation to resolution: generation produces discrepancies, and this layer turns each one into a single owned, time-boxed, auditable work item pushed into the EDC through a validated interface. Because the pipeline is a read-only consumer of source clinical data, queries are never written by direct database mutation — they are raised, updated, and closed exclusively through the official, role-controlled EDC API, under 21 CFR Part 11 and ALCOA+.

The Query State Machine #

The lifecycle is a finite state machine with a small, closed set of legal transitions. Queries advance from open to closed along a main path, reissue back to a site when a review finds the response insufficient, and reopen from closed only when new corrected data arrives. Every edge is an appended, attributable event — never an in-place edit of a status column.

The states form a closed vocabulary: OPEN (raised, not yet assigned), ROUTED (owned, with an SLA clock running), ANSWERED (a site response is attached), UNDER_REVIEW (a CRA is verifying the correction), and CLOSED (terminal). Two feedback transitions keep the model honest. Reissue (UNDER_REVIEW → ROUTED) fires when a review finds the site response does not resolve the discrepancy; reopen (CLOSED → OPEN) fires only when genuinely new corrected data lands against a rule that a closed query already covered. Encoding these as data — not as scattered if branches — is what makes the lifecycle testable, reproducible, and inspectable years later.

Concepts and Prerequisites #

Lifecycle management is downstream of validation and generation and upstream of nothing — it is the terminal owner of a query’s regulated history. It consumes an immutable discrepancy already bound to a rule_id and rule_version, and it writes only to its own append-only event store plus the EDC’s official query interface. It never edits source clinical data. Before implementing the patterns below, the following are mandatory.

• Regulatory baseline. A working command of 21 CFR Part 11 electronic-record requirements and ALCOA+. Every raise, route, answer, review, close, reissue, and reopen is a regulated record and must be Attributable, Contemporaneous, Original, and Enduring. The precise audit fields a read-only consumer may record are scoped by Audit Trail Boundaries in EDC Systems.
• Data standards. Queries reference CDISC ODM FormOID and ItemOID paths for the exact CRF location under query, aligned to the annotated forms described in CDISC ODM vs CDASH Schema Mapping.
• Upstream contracts. Severity arrives on the discrepancy and is calibrated by Discrepancy Threshold Tuning; routing to an owner is governed by Deterministic Query Routing Workflows for CRAs. This layer treats severity and owner as inputs, not values it recomputes.

Pin every dependency so a lifecycle decision made today replays byte-identically during an inspection two years from now:

# 21 CFR Part 11 relevance: version pinning is a reproducibility control. Replaying
# the archived event log MUST reproduce the same state, due date, and audit chain.
pydantic==2.7.1
python-dateutil==2.9.0.post0
PyYAML==6.0.1
httpx==0.27.0

Environment assumptions: the lifecycle service is stateless and runs against an append-only event store. It holds no mutable per-query status in memory — current state is always derived by folding the event log, which lets the service scale horizontally and replay history without corrupting the ledger.

The Query Object and Deduplicated Raise #

A query record carries a stable identity, the CRF coordinates it acts on, its lifecycle state, and its aging clock. The schema is a frozen model so malformed inputs fail at the boundary rather than producing a non-reproducible record:

# ALCOA+ relevance: an immutable query object enforces Accurate + Original — the
# coordinates and rule binding that justify the query cannot be mutated after raise.
from datetime import datetime
from enum import Enum
from pydantic import BaseModel, ConfigDict


class State(str, Enum):
    OPEN = "OPEN"
    ROUTED = "ROUTED"
    ANSWERED = "ANSWERED"
    UNDER_REVIEW = "UNDER_REVIEW"
    CLOSED = "CLOSED"


class Query(BaseModel):
    model_config = ConfigDict(frozen=True)

    query_id: str          # deterministic surrogate, derived below
    subject: str           # SubjectKey
    form: str              # CDISC ODM FormOID
    item: str              # CDISC ODM ItemOID under query
    rule_id: str           # the edit-check rule that raised it
    state: State
    severity: str          # from generation; never recomputed here
    opened_ts: datetime    # UTC instant of first raise
    due_ts: datetime       # SLA aging deadline
    owner: str             # role or named principal from routing

The single most common lifecycle defect is duplicate queries: a nightly re-run raises a second query for a discrepancy that is already open. The structural fix is a deterministic deduplication key derived from the discrepancy’s natural coordinates, so an identical discrepancy always maps to the same query_id. Raising is then idempotent — it checks for an existing non-closed query on that key before creating anything:

# 21 CFR Part 11 relevance: idempotent raise prevents duplicate regulated records for
# one discrepancy. The dedup key is the natural identity of the underlying finding.
import hashlib


def dedup_key(subject: str, form: str, item: str, rule_id: str) -> str:
    raw = "|".join((subject, form, item, rule_id))
    return "Q-" + hashlib.sha256(raw.encode()).hexdigest()[:16]


def raise_query(disc: dict, open_index: dict, now, due_ts, owner) -> Query | None:
    """Return a new Query, or None if an open query already covers this discrepancy."""
    qid = dedup_key(disc["subject"], disc["form"], disc["item"], disc["rule_id"])
    existing = open_index.get(qid)
    if existing and existing["state"] != State.CLOSED.value:
        return None                      # already live -> do not duplicate
    return Query(
        query_id=qid, subject=disc["subject"], form=disc["form"],
        item=disc["item"], rule_id=disc["rule_id"], state=State.OPEN,
        severity=disc["severity"], opened_ts=now, due_ts=due_ts, owner=owner,
    )

Deduplication keys on the discrepancy identity, not the query text, so a reworded edit-check message never fragments one finding into two queries. A closed query on the same key does not block a new raise — that is the reopen path, handled as an explicit transition rather than a silent second record.

The Lifecycle State Machine and Immutable Audit Ledger #

State is never stored as a mutable column. It is derived by folding the append-only ledger, and a transition is accepted only when it is legal for the current state. This is what structurally prevents illegal jumps such as CLOSED → ANSWERED, and it is the mechanism that preserves audit continuity across a reopen:

# 21 CFR Part 11 relevance: an append-only ledger with validated transitions IS the
# audit trail. State is reconstructed by replay, so history can never be rewritten
# (ALCOA+ Enduring, Consistent, Attributable).
import json
from datetime import timezone

LEGAL_TRANSITIONS = {
    State.OPEN: {State.ROUTED},
    State.ROUTED: {State.ANSWERED, State.CLOSED},      # close = cancelled/duplicate
    State.ANSWERED: {State.UNDER_REVIEW},
    State.UNDER_REVIEW: {State.CLOSED, State.ROUTED},  # reissue on insufficient answer
    State.CLOSED: {State.OPEN},                         # reopen on corrected data only
}


def append_transition(query_id: str, prev: State, nxt: State,
                      actor: str, prev_hash: str, reason: str, store) -> dict:
    if nxt not in LEGAL_TRANSITIONS[prev]:
        raise ValueError(f"Illegal transition {prev} -> {nxt} for {query_id}")
    event = {
        "query_id": query_id,
        "from_state": prev.value,
        "to_state": nxt.value,
        "actor": actor,                                 # system or named principal
        "reason": reason,                               # why this edge fired
        "occurred_at": datetime.now(timezone.utc).isoformat(),
        "prev_hash": prev_hash,                         # hash chain over the ledger
    }
    event["event_hash"] = hashlib.sha256(
        (prev_hash + json.dumps(event, sort_keys=True)).encode()
    ).hexdigest()
    store.append(event)                                 # never updated in place
    return event

A reopen appends an OPEN event whose prev_hash points at the closing event, so the reopened query shares one continuous, tamper-evident history with its earlier life rather than starting a disconnected new chain — the property that lets a reviewer reconstruct why a query was closed in March and reopened in May. Current state is a fold over that chain:

# ALCOA+ relevance: deriving state by replay (Consistent) means two services never
# disagree about a query's status, and a corrupted "current" value cannot hide.
def current_state(events: list[dict]) -> State:
    return State(events[-1]["to_state"]) if events else State.OPEN

SLA and Aging Clocks #

Each routed query carries a due_ts computed from its severity’s SLA window. Aging is a side effect of the wall clock, not of any inbound event, so it must be evaluated by a scheduled sweeper rather than waited on. The sweeper is idempotent — it never double-counts a query already past its deadline — and it emits aging metadata that downstream escalation consumes:

# 21 CFR Part 11 relevance: aging surfaces queries before they leave the monitoring
# window (Contemporaneous). The sweep is idempotent so re-runs create no duplicates.
from datetime import timedelta

SLA_HOURS = {"critical": 24, "borderline": 72, "low": 168}


def compute_due(opened_ts: datetime, severity: str) -> datetime:
    return opened_ts + timedelta(hours=SLA_HOURS.get(severity, 72))


def age_open_queries(open_queries: list[dict], now: datetime) -> list[dict]:
    aging = []
    for q in open_queries:
        if q["state"] in (State.CLOSED.value,):
            continue
        overdue = now > datetime.fromisoformat(q["due_ts"])
        aging.append({
            "query_id": q["query_id"], "overdue": overdue,
            "age_hours": (now - datetime.fromisoformat(q["opened_ts"])).total_seconds() / 3600,
        })
    return aging

This aging signal is the input to tiered notification and escalation; the business-day arithmetic, holiday calendars, and multi-tier escalation ladder are covered in depth in Escalating Aging Queries with Python SLA Timers.

Writing Back Through a Validated EDC API #

Because the pipeline is a read-only consumer of source data, it never writes to the EDC database. Every raise, reissue, and close is pushed through the vendor’s official, validated query API, which enforces its own role-based access, applies the sponsor’s electronic-signature manifest, and returns the authoritative edc_query_id. The client is idempotent on the query_id so a retried request never creates a second EDC query:

# 21 CFR Part 11 relevance: writes go through the validated EDC interface only, never
# direct DB mutation. The interface applies attributable, role-controlled signatures.
import httpx


def push_to_edc(query: "Query", op: str, client: httpx.Client, principal: str) -> dict:
    """op in {'raise','reissue','close'}. Idempotency-Key dedupes on the EDC side."""
    resp = client.post(
        f"/rws/queries/{op}",                        # official RWS-style query endpoint
        headers={
            "Idempotency-Key": query.query_id,        # server-side dedupe
            "X-Principal": principal,                 # attributable actor
        },
        json={
            "StudyOID": query.subject.split("-")[0],
            "SubjectKey": query.subject,
            "FormOID": query.form,
            "ItemOID": query.item,
            "QueryText": f"Discrepancy per rule {query.rule_id}",
            "MarkingGroup": query.severity,
        },
        timeout=30.0,
    )
    resp.raise_for_status()
    return resp.json()                                # includes authoritative edc_query_id

The operational store records the returned edc_query_id alongside the local query_id, giving a two-way reconciliation key. If the EDC rejects a write (a deleted form, a locked casebook), the failure is dead-lettered for human handling rather than retried blindly — a rejected write must never be swallowed, because a query that a data manager believes was raised but that never reached the EDC is an integrity gap. Routing the write through the vendor interface also inherits the platform’s own electronic-signature and role checks for free, so the pipeline never has to reimplement Part 11 signature semantics; the access model behind those role checks is described in Role-Based Access Control for Clinical Data. Every response, including the authoritative edc_query_id and the applied marking group, is logged so a reviewer can trace a local transition to the exact EDC record it produced.

Configuration and Parameterization #

The state machine, SLA windows, and reissue policy live in version-controlled YAML, tagged on every database-lock milestone. Changing an SLA or an allowed transition is then a reviewable change-management event, not a code deploy:

# 21 CFR Part 11 relevance: config-as-code. Every SLA or transition change is a tracked
# Git commit subject to impact assessment and sign-off before promotion.
lifecycle_version: "2026.07.0"       # tag aligned to the database-lock milestone
sla_hours:
  critical: 24
  borderline: 72
  low: 168
transitions:                          # the legal edges enforced at runtime
  OPEN:          [ROUTED]
  ROUTED:        [ANSWERED, CLOSED]
  ANSWERED:      [UNDER_REVIEW]
  UNDER_REVIEW:  [CLOSED, ROUTED]     # ROUTED = reissue
  CLOSED:        [OPEN]               # OPEN = reopen on corrected data
reissue:
  max_reissues: 3                     # after N, force human medical-monitor review
  require_reason: true

Configuration binds to a deployment through environment variables that keep development, validation, and production strictly segregated:

CI rejects any lifecycle config change that lacks a lifecycle_version bump, so a silent edit to an SLA window or a transition rule can never reach production unnoticed.

Testing and Validation #

Each transition rule is a regulated artifact, so the lifecycle demands the same test rigor as an edit check. Tests assert idempotent raise, legal-transition enforcement, audit-chain continuity across a reopen, and idempotent aging. Mock discrepancy fixtures let tests run hermetically in CI:

# 21 CFR Part 11 relevance: automated regression tests are the OQ evidence that lifecycle
# behavior is unchanged across releases, retained with the IQ/OQ/PQ package.
import pytest


def test_raise_is_idempotent(disc, open_index, now):
    q1 = raise_query(disc, open_index, now, compute_due(now, "critical"), "lead_cra")
    open_index[q1.query_id] = {"state": q1.state.value}
    q2 = raise_query(disc, open_index, now, compute_due(now, "critical"), "lead_cra")
    assert q2 is None                                    # no duplicate for one discrepancy


def test_illegal_transition_rejected(store):
    with pytest.raises(ValueError):
        append_transition("Q-abc", State.CLOSED, State.ANSWERED,
                          actor="test", prev_hash="0" * 64, reason="x", store=store)


def test_reopen_preserves_hash_chain(store):
    close_evt = append_transition("Q-abc", State.UNDER_REVIEW, State.CLOSED,
                                 "cra01", "0" * 64, "verified", store)
    reopen_evt = append_transition("Q-abc", State.CLOSED, State.OPEN,
                                  "system", close_evt["event_hash"], "new data", store)
    assert reopen_evt["prev_hash"] == close_evt["event_hash"]   # continuous history

Run a new or modified lifecycle config in shadow mode against a historical event log first, confirming the derived states and due dates match the validated baseline before it governs live queries. Retain the test plan, executed results, and a traceability matrix linking each transition rule to its monitoring-plan requirement with the IQ/OQ/PQ package described in Validation and CSV Frameworks.

Production Gotchas and Failure Modes #

The race between an auto-close and a new site response is the most insidious: if the pipeline closes a query in the same window the site submits a corrected value, a naive last-writer-wins update can mark a still-unresolved discrepancy closed. Serializing transitions per query and gating the close on the ledger head is the structural fix — the close simply loses the race and the query returns to review. The response-parsing logic that decides whether a close is even warranted is detailed in Automating Query Closure with Response Parsing.

Compliance Checklist #

Use this as the change-management gate before promoting a lifecycle configuration to production. Each item maps to an ALCOA+ or 21 CFR Part 11 control.

• Every transition event carries query_id, from_state, to_state, actor identity, a reason, and a UTC timestamp (Attributable, Contemporaneous).
• State is derived by folding an append-only ledger; status is never an in-place update (Enduring, Consistent).
• The ledger is hash-chained (prev_hash → event_hash) so historical edits are tamper-evident (Original).
• Raise is idempotent on a deterministic dedup_key; one discrepancy never yields two open queries.
• A reopen appends to the existing chain via the closing event’s hash, preserving continuous history.
• All EDC writes go through the validated official API with an idempotency key and an attributable principal — never direct DB mutation.
• EDC write rejections are dead-lettered and reconciled, never silently retried or swallowed.
• SLA aging is evaluated by an idempotent sweeper against the wall clock; re-runs create zero duplicate aging records.
• The lifecycle config is version-controlled and tagged to the database-lock milestone; CI fails on untagged changes.
• Development, validation, and production run against segregated EDC tenants via the QLC_ENV guard.
• IQ/OQ/PQ artifacts and a transition-rule-to-protocol traceability matrix are retained for inspection.

Related #

• Clinical Query Generation & Discrepancy Management — the parent section this discipline belongs to.
• Automated Clinical Query Generation — produces the discrepancies this lifecycle raises, deduplicates, and closes.
• Deterministic Query Routing Workflows for CRAs — resolves the owner and SLA that the routed state consumes.
• Discrepancy Threshold Tuning — calibrates the severity that drives SLA windows and aging.
• Automating Query Closure with Response Parsing — decides when the UNDER_REVIEW → CLOSED transition is warranted.
• Escalating Aging Queries with Python SLA Timers — the business-day aging and tiered escalation built on these clocks.