Skip to content

API Reference

preceptron.score

Score a clinical response using an LLM judge.

from preceptron import score

result = score(
    task="cpc_bond",
    response="...",
    final_diagnosis="...",
    model="gpt-4o",
    client=client,
)

Parameters

Parameter Type Required Description
task str No One of: management_reasoning, diagnostic_reasoning, r_idea, cpc_bond, cpc_management. If omitted, a router LLM picks the best-matching rubric(s) and scores the response against each.
response str Yes The clinical response to score
client OpenAI or Anthropic Yes An initialized API client
model str Yes Model identifier (e.g. "gpt-4o", "claude-sonnet-4-20250514")
rubric dict, list, or str Varies Required for management_reasoning. Optional override for other tasks
case_vignette str Varies Required for management_reasoning and diagnostic_reasoning
question_text str Varies Required for management_reasoning, diagnostic_reasoning, and r_idea
final_diagnosis str Varies Required for diagnostic_reasoning and cpc_bond
test_plan str Varies Required for cpc_management

Returns

When task is specified (or the router picks exactly one task):

{
    "score": int | None,        # numeric score
    "justification": str | None, # LLM's explanation
    "raw": str,                  # full LLM response
}

When task is omitted and the router picks multiple tasks:

{
    "router": {"tasks": ["cpc_bond", "diagnostic_reasoning"]},
    "results": {
        "cpc_bond": {"score": ..., "justification": ..., "raw": ...},
        "diagnostic_reasoning": {"score": ..., "justification": ..., "raw": ...},
    },
}

Router

If you call score() without task=, a router LLM inspects the response and any context you supply (case vignette, question, final diagnosis, reference test plan) and picks one or more task rubrics to apply. Tasks whose required context is missing are skipped automatically — so a bare differential list with a known final_diagnosis routes to cpc_bond, while the same list with a case vignette also routes to diagnostic_reasoning.

Override the router model with router_model=; by default it reuses model.

Preset Rubrics

from preceptron import BOND_SCORE, TESTING_PLAN, R_IDEA

Tasks with a preset rubric (cpc_bond, cpc_management, r_idea) use it automatically. Pass rubric= to override.

Client Compatibility

Any OpenAI-compatible client works:

from openai import OpenAI
client = OpenAI()

from anthropic import Anthropic
client = Anthropic()

from openai import OpenAI
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-..."
)

from openai import AzureOpenAI
client = AzureOpenAI(
    api_key="...",
    azure_endpoint="https://your-endpoint.openai.azure.com",
    api_version="2024-12-01-preview",
)