lib.provider-ranking

@lssm-tech/lib.provider-ranking

Website: https://contractspec.io

AI provider ranking: benchmark ingestion, scoring, and model comparison.

What It Provides

• Layer: lib.
• Consumers: module.provider-ranking.
• Related ContractSpec packages include @lssm-tech/tool.bun, @lssm-tech/tool.typescript.
• Related ContractSpec packages include @lssm-tech/tool.bun, @lssm-tech/tool.typescript.

Installation

npm install @lssm-tech/lib.provider-ranking

or

bun add @lssm-tech/lib.provider-ranking

Usage

Import the root entrypoint from @lssm-tech/lib.provider-ranking, or choose a documented subpath when you only need one part of the package surface.

Architecture

• src/eval is part of the package's public or composition surface.
• src/in-memory-store.ts is part of the package's public or composition surface.
• src/index.ts is the root public barrel and package entrypoint.
• src/ingesters is part of the package's public or composition surface.
• src/scoring is part of the package's public or composition surface.
• src/store.ts is part of the package's public or composition surface.
• src/types.ts is shared public type definitions.

Public Entry Points

• Export . resolves through ./src/index.ts.
• Export ./eval resolves through ./src/eval/index.ts.
• Export ./eval/runner resolves through ./src/eval/runner.ts.
• Export ./eval/types resolves through ./src/eval/types.ts.
• Export ./in-memory-store resolves through ./src/in-memory-store.ts.
• Export ./ingesters resolves through ./src/ingesters/index.ts.
• Export ./ingesters/artificial-analysis resolves through ./src/ingesters/artificial-analysis.ts.
• Export ./ingesters/chatbot-arena resolves through ./src/ingesters/chatbot-arena.ts.
• Export ./ingesters/fetch-utils resolves through ./src/ingesters/fetch-utils.ts.
• Export ./ingesters/open-llm-leaderboard resolves through ./src/ingesters/open-llm-leaderboard.ts.
• Export ./ingesters/registry resolves through ./src/ingesters/registry.ts.
• Export ./ingesters/swe-bench resolves through ./src/ingesters/swe-bench.ts.
• Export ./ingesters/types resolves through ./src/ingesters/types.ts.
• Export ./provider-ranking.feature resolves through ./src/provider-ranking.feature.ts.
• Export ./routing resolves through ./src/routing/index.ts.
• Export ./routing/classify-prompt resolves through ./src/routing/classify-prompt.ts.
• Export ./routing/config resolves through ./src/routing/config.ts.
• Export ./routing/escalation resolves through ./src/routing/escalation.ts.
• Export ./routing/model-capabilities resolves through ./src/routing/model-capabilities.ts.
• Export ./routing/profiles resolves through ./src/routing/profiles.ts.
• Export ./routing/resolve-difficulty resolves through ./src/routing/resolve-difficulty.ts.
• Export ./routing/role-matrix resolves through ./src/routing/role-matrix.ts.
• Export ./routing/scoring resolves through ./src/routing/scoring.ts.
• Export ./routing/select-model resolves through ./src/routing/select-model.ts.
• Export ./routing/telemetry resolves through ./src/routing/telemetry.ts.
• Export ./routing/types resolves through ./src/routing/types.ts.
• Export ./routing/weights resolves through ./src/routing/weights.ts.
• Export ./scoring resolves through ./src/scoring/index.ts.
• Export ./scoring/composite-scorer resolves through ./src/scoring/composite-scorer.ts.
• Export ./scoring/dimension-weights resolves through ./src/scoring/dimension-weights.ts.
• Export ./scoring/normalizer resolves through ./src/scoring/normalizer.ts.
• Export ./store resolves through ./src/store.ts.
• Export ./types resolves through ./src/types.ts.
• The package publishes 32 total export subpaths; keep docs aligned with package.json.

Local Commands

• bun run dev — contractspec-bun-build dev
• bun run build — contractspec-bun-build build
• bun run test — bun test --pass-with-no-tests
• bun run lint — bun lint:fix
• bun run lint:check — biome check .
• bun run lint:fix — biome check --write --unsafe --only=nursery/useSortedClasses . && biome check --write .
• bun run typecheck — tsgo --noEmit
• bun run publish:pkg — bun publish --tolerate-republish --ignore-scripts --verbose
• bun run publish:pkg:canary — bun publish:pkg --tag canary
• bun run clean — rimraf dist .turbo
• bun run build:bundle — contractspec-bun-build transpile
• bun run build:types — contractspec-bun-build types
• bun run prebuild — contractspec-bun-build prebuild

Recent Updates

• Replace eslint+prettier by biomejs to optimize speed.
• Resolve lint, build, and type errors across nine packages.
• Add first-class transport, auth, versioning, and BYOK support across all integrations.
• Add AI provider ranking system with ranking-driven model selection.

Notes

• Store interface is the adapter boundary — do not leak implementation details.
• Scoring algorithms must stay deterministic (no randomness, no side effects).
• Benchmark dimension enum is shared across ingesters and scoring — keep in sync.

Agent-aware routing

@lssm-tech/lib.provider-ranking/routing adds deterministic Builder model routing without executing providers. Use selectModelForAgent(input) to classify difficulty, apply manual thinking and risk floors, enforce required capabilities, select cheap/optimized/best/critical policy, and return an inspectable routing decision.

This surface is research/discovery and offline candidate-generation guidance. It is not Protocol V1 qualification, Connect authorization, persisted selection, fallback, or production routing truth. Every suggested candidate must still pass hard policy gates, operation-specific evaluation, durable Agent Host admission, and exact Connect verification before execution.

Critical risk (payments, contract-liability, compliance, destructive data loss, legal) cannot be downgraded by budget mode or manual thinking settings.

Canonical provider ranking contracts are available through the explicit ./contracts family. Existing contracts-spec subpaths preserve the same contract objects and nominal registry types. These exports do not execute workflows, fetch benchmarks or run paid evaluations.