Source snapshot bcc79fcc · Declared package version 2.2.4. This page combines the checked-in package guide and manifest; commands and runtime behavior were not executed in this review.
Deterministic agent evaluation runners, model adapters, graders, and recommendation evidence for ContractSpec.
What it provides
AgentEvaluationRunner for candidate-vs-baseline trial aggregation.
AgentEvalsFeature package metadata for docs and discovery.
Scripted and live model runner adapters.
Exact-match, schema, policy, artifact, and environment-outcome graders.
An LLM-judge adapter that turns an injected judge client into a grader.
Replay bundle refs, redaction leak detection, and cost/latency summaries.
Fail-closed output, blocking-grader, deterministic-safety, and trial-integrity gates for production-shaped candidate comparisons.
Route- and release-bound simulated qualification decisions with mandatory baseline evidence for model upgrades.
A checkpointed rollout state machine covering offline, shadow, internal, 1%, 10%, 50%, production, and evidence-backed rollback stages.
V2 immutable dataset and exact evaluation-subject bindings covering route, model profile, invocation settings, application purpose, evaluator set, environment, and release versions.
Reproducible verified-success, cost-per-verified-success, p50/p95 latency, structured-output, tool, fallback, provider, and human-review metrics by dataset segment.
Dataset digests include an application-owned confidence-interval or power analysis method, minimum sample count, and rationale reference. Missing samples, invalid segment weights, uncovered segments, or rebound cases fail before recommendation.
V2 comparison artifacts project trials to sanitized case/segment refs, numeric metrics, grader verdicts, replay digests, and boolean failure flags; prompts, responses, artifact bodies, metadata payloads, and replay URIs are never included.
CAS-ready qualification promotion/revocation and verified canary histories; expired, revoked, rebound, discontinuous, or re-digested rollout evidence fails closed.
Model evaluation never grants provider access or effect authority. A candidate becomes live_verified only after every rollout checkpoint has complete health evidence; a newer model is not promoted merely because it exists or scores well on average.
Local commands
bun run build
bun run typecheck
bun run test
bun run lint:check
Declared capabilities.
Lib Agent Evals
Agent evaluation runners, adapters, graders, and recommendation evidence.
Use it to
Adopt agent evaluation runners, adapters, graders, and recommendation evidence
Requires
TypeScript
Choose another boundary when
The @lssm-tech/lib.agent-evals package constraints do not match the target runtime or platform.
These are package-manifest entry points. Named symbols listed above come from the capability catalogue; this is not an exhaustive TypeScript symbol reference.
