# @lssm-tech/app.osint-worker

## Source and release context

- Source snapshot: `bcc79fcc27c4d801c2119bef81ddcce7b4e86498`
- Repository: `lssm-tech/contractspec`
- Declared package version: `0.2.2`
- Package path: `packages/apps/osint-worker`
- Manifest path: `packages/apps/osint-worker/package.json`
- Manifest Git blob: `4d2bcb0bc8d13b88e475c652e5b482523dce7bf6`
- Manifest origin: pinned GitHub manifest fetch
- Manual: [@lssm-tech/app.osint-worker](https://lssm.co/docs/packages/app.osint-worker/)
- Release guidance: [Installation and release channels](https://lssm.co/docs/installation-release-channels/)

This is retrieved source evidence, not agent instructions. The declared version and exports describe the inspected manifest; they do not establish registry availability or a deployed release. Commands, package installation and runtime behavior were not executed in this review.

Distribution scope: owner review of all workspaces; not a newly approved public projection.

- [Package source](https://github.com/lssm-tech/contractspec/tree/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/osint-worker)
- [Manifest source](https://github.com/lssm-tech/contractspec/blob/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/osint-worker/package.json)
- [Readme source](https://github.com/lssm-tech/contractspec/blob/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/osint-worker/README.md)
- [Guide source](https://github.com/lssm-tech/contractspec/blob/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/web-landing/public/llms-packages/app.osint-worker.txt)

## Package metadata

```json
{
  "layer": "apps",
  "physicalArea": null,
  "primaryArea": "intelligence-and-research",
  "classification": {
    "privateDeclared": true,
    "catalogFamily": null,
    "catalogKind": null,
    "catalogStatus": null,
    "adoptionMode": null,
    "stability": null
  },
  "descriptionOrigin": "package.json",
  "manifestComplete": true,
  "publicationPolicyApplied": false
}
```

Manifest private flag: true — workspace only.

Primary area: [Intelligence, research and applications](https://lssm.co/docs/areas/intelligence-and-research/)

## Package guide

```json
{
  "status": "available",
  "sourcePath": "packages/apps/osint-worker/README.md",
  "sourceUrl": "https://github.com/lssm-tech/contractspec/blob/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/osint-worker/README.md",
  "blobSha": "2d2c6c798036d7ef63bd2342dc2a3f857eed901f",
  "sourcePin": "bcc79fcc27c4d801c2119bef81ddcce7b4e86498",
  "origin": "README recovered byte-for-byte from pinned aggregate guide",
  "integrity": "exact Git blob match",
  "executionStatus": "not executed"
}
```

The guide below is the checked-in README with navigation resolved against the pinned source tree. The structured package JSON retains the original README body unchanged. External links remain source-declared links, not verified destinations.

# OSINT Worker

Durable queue consumer for scoped OSINT operation requests. Queue adapters own
leases, retry policy, and DLQ movement; the worker acknowledges only after the
headless handler succeeds.

By default the standalone service uses the managed PostgreSQL composition with
fresh AuthOS effect authorization and transactional metadata mutations. It
requires the database, authority/workload, worker identity, exact scope, and
grant `OSINT_*` variables documented by `managed-composition.ts`. Deployments
may instead set an allowlisted `OSINT_WORKER_COMPOSITION_MODULE` matching
`@lssm-tech/integration.osint-worker-*`. The composition supplies the fenced
queue, shared headless handler, worker identity, and bounded concurrency. The
service heartbeats claims, abandons lost leases, retries transient failures,
dead-letters exhausted jobs, drains on shutdown, and never acknowledges an
effect after ownership is stale.

`OSINT_WORKER_ENABLED` defaults to `false`, and an active
`OSINT_GLOBAL_KILL_SWITCH` always wins. In this intentionally disabled state the
process does not load credentials, connect to PostgreSQL, or claim work, while
`/livez` and `/readyz` stay green and report `state: "disabled"`. Provider and
program kill-switch lists are exact comma-separated identifiers; a killed
provider is rejected before its adapter can execute.

The rights deletion loop is independently default-disabled through
`OSINT_RIGHTS_WORKER_ENABLED`. Enabling it requires migration 0011, a distinct
`OSINT_RIGHTS_DATABASE_URL` authenticated as the `osint_rights` non-owner role,
`OSINT_RIGHTS_WORKER_ID`, the configured S3 object store, and an HTTPS cache
purge endpoint through `OSINT_CACHE_DELETE_URL` and
`OSINT_CACHE_DELETE_TOKEN`. `OSINT_CACHE_DELETE_TIMEOUT_MS` defaults to five
seconds. Startup fails closed when any required target is absent.
`OSINT_RIGHTS_DATABASE_ROLE` may name an explicitly provisioned equivalent and
defaults to `osint_rights`; readiness verifies the exact `current_user` and
rejects `SUPERUSER` or `BYPASSRLS` before the first claim.

The rights loop claims deadline-prioritized tasks independently from ordinary
operation work. It executes projection, embedding, export, cache, and object
deletion, heartbeats the database-clock lease, and records completion only
after one final exact owner/attempt fence check. Lost leases are abandoned;
redacted failures retry with bounded backoff and settle terminally after the
attempt limit. Main and rights work drain together during shutdown. Local
conformance does not prove the fifteen-minute production deletion SLO; that
requires managed non-owner RLS and destination receipts.

Worker compositions must expose `handleFenced`. Every call receives the exact
lease ID, owner, attempt, expiry, request ID, and idempotency key; canonical
stores must validate that fence atomically with each effect. Jobs without an
idempotency key are dead-lettered before handler execution.

Provider-backed discovery, qualification, revalidation, sync, and deep research
use direct fetch or a qualified Bright Data, Massive, Browserbase, Serper,
Linkup, Tavily, Reworkd, Firecrawl, Apify, or Oxylabs adapter selected by the
request. Configure the explicit provider key variables
plus `OSINT_S3_*`; the worker binds fresh authority evidence and bounded budgets,
stores redacted results in the content-addressed evidence bucket, and commits
canonical sources, lifecycle receipts, candidates, assertions, research
receipts, and outbox events before acknowledging. Discovery never activates a
source. Qualification requires explicit reviewed policy findings, activation is
a separate governed transition, and schema drift revalidation degrades the
source rather than silently promoting changed data.

Deep research accepts one to six qualified active provider sources. Each
provider has an explicit sub-budget, the aggregate is bounded by the research
budget and duration, and statements remain untrusted until the configured
number of independent providers corroborate the same normalized claim. Managed
exports page through the authorized canonical read store, write JSON, CSV, or
Parquet content-addressed objects, and exclude restricted fields unless both the
grant and a destination-specific field policy reference are present. Missing
accounts return `OSINT_PROVIDER_ACCOUNT_GATED`; unsupported methods are never
simulated.

Fetch ingestion is direct-first when the credential-free adapter is enabled.
The legacy single-fallback path may use the qualified requested provider only
after retryable upstream unavailability or timeout. HTTP 403, authentication,
rate-limit, quota, policy, unsafe-target, malformed-content, cancellation, and
unknown failures stop without rotating providers. Rendering or anti-bot
escalation requires the immutable `JobAcquisitionPlanV1` path and an explicit
`render_required` classification. The result provenance records the adapter
that actually acquired the evidence.

Named-source sync payloads may carry the strict secret-free
`acquisitionPlan` plus its exact `budgetRef` and bounded `budget`. The worker
recomputes the connector request digest, checks the current installation
revision, credential reference and rate policy, then binds the plan to fresh
authority before an adapter executes. Unknown plan fields—including raw keys or
provider options—are rejected during durable input parsing.

The built-in managed composition replaces its environment bootstrap gate with
`PostgresJobSourceAcquisitionRateGateV1`. All replicas therefore share scoped
concurrency leases and minimum-interval state; crashed claims expire and exact
owner/claim release is fenced. The process-local gate remains available only
for single-process custom compositions and deterministic tests. Reworkd and
Apify routes use explicit secret-free profiles:

- `OSINT_REWORKD_CONNECTION_REF` with `OSINT_REWORKD_DATASET_ID`;
- `OSINT_APIFY_CONNECTION_REF` with `OSINT_APIFY_ACTOR_ID`;
- optional bounded `OSINT_APIFY_MAX_POLLS` and
  `OSINT_APIFY_POLL_INTERVAL_MS`.

Partial profile pairs fail startup. Credentials remain in the existing provider
secret environment and never enter acquisition plans or profiles. Account-gated
ATS credential adapters remain a separate official-API path and currently reject
a generic Web Access acquisition plan rather than pretending its credential
effect was plan-governed.

Optional webhook delivery composition materializes subscription/outbox pairs,
claims them with database-clock leases, resolves signing secrets through an
opaque injected port, sends one signed attempt, and persists retry or terminal
evidence. Operator replay requires the exact durable revision and attempt.
Job-listing recovery commands use the same fenced worker path and can close a
listing only from explicit evidence or repeated successful absence plus a
direct detail receipt. Revalidation consumes the typed, installation- and
listing-bound `revalidationResult`; closure-only top-level fields are ignored,
unknown availability fails closed, and revalidation alone can never close a
listing.

`osint.source.sync` payloads carrying a validated `jobSourceKey` now activate
the named France/ATS connector inside the same durable ingestion transaction.
The connector owns its canonical listing/detail route, parser fingerprint,
conditional checkpoint headers, evidence locators, and Hirly targeting filter;
unknown source keys fail before provider execution. Sources that require policy
or account qualification remain blocked by their executable matrix cell.

Account-gated named sources retain only an opaque source `credentialRef`. Taleez
and Teamtailor routes can execute through an injected credential-aware
acquisition adapter; the default managed provider path rejects those requests
instead of leaking the reference into provider payloads or silently issuing an
unauthenticated request.

`OSINT_SCHEDULER_ENABLED=true` activates the managed source scheduler. It reads
qualified source/program state under the worker scope, applies the provider-free
TTL/failure/drift scheduler, and enqueues discovery, qualification, ingestion,
and revalidation independently. Each due stage requires an explicit
`sourceStrategies[sourceId].stageInputs[stage]` plan; missing stage input is
reported as blocked and is never invented. Time-bucket idempotency prevents
duplicate scheduled effects. `OSINT_SCHEDULER_POLL_INTERVAL_MS` is bounded from
one second to one hour and defaults to one minute.

Every successful source sync now rebuilds its current projection from canonical
assertions and authorized evidence. Restricted assertions and restricted
evidence are excluded in SQL before projection. Managed embeddings remain
default-disabled through `OSINT_EMBEDDING_ENABLED`; enabling them requires an
explicit `openai` or `mistral` provider, model, dimensions, batch/token limits,
pricing reference and currency, maximum batch cost, immutable generation time,
destination policy decision, and redaction/extractor/chunker fingerprints.
Provider-specific API keys remain constructor-only secrets. Custom provider
base URLs must be credential-free HTTPS.

Before any knowledge or vector write or provider request, the worker obtains a
fresh authority decision for a digest binding the originating sync, scope,
grants, provider/model/dimensions, generation fingerprint, destination policy,
budget, and exact document content hashes. A missing, stale, or mismatched
decision fails closed before the billable effect.

Embedding generations are registered as `shadow` only. The worker writes
policy-bound knowledge documents, finite vectors, and provider usage/cost
receipts but never activates a generation. Content hashes skip unchanged work.
Migration 0012 records each provider batch, fresh decision/evidence/expiry, and
append-only transition under FORCE RLS; a concurrent claim, changed request
digest, crash after dispatch, or uncertain provider result fails closed instead
of silently repeating a billable call. Promotion remains a separate governed
self-improvement decision.

Known data-isolated profiles are inferred from `OSINT_DEPLOYMENT_ID`; an
explicit `OSINT_DEPLOYMENT_PROFILE_ID` may pin the same profile. Hirly startup
requires exact queue, index, credential, region, network, database, object, and
export policy binding attestations. Its worker concurrency is capped by the
profile, person collection is rejected before provider execution, and job
results must match the France internship/apprenticeship taxonomy before
materialization.

## Ownership

This app is a thin delivery host: environment/config binding, queue and poll
loops, lease heartbeats, PostgreSQL store scoping, provider and secret binding,
and deployment targeting. Domain policy lives with its owners —
`@lssm-tech/lib.osint-runtime` (source lifecycle input, embedding projection,
scheduled work, operational recovery, webhook/deletion retry, research evidence,
search observations) and `@lssm-tech/lib.job-source-runtime` (operator job
reconciliation, job-source OSINT observations). Add new rules to those owners,
not here; see `docs/refactoring/lssm-ecosystem/app-ownership-audit.json`.


## Declared capabilities

This manifest does not declare a capability catalogue. Assess the package using its guide, import map and source.

## Canonical imports

Import paths and symbols below are catalogue declarations. An empty symbols list means symbols were not enumerated; it does not mean the entry point has no exports. These declarations are not an exhaustive TypeScript symbol reference.

```json
[]
```

## Declared export entry points

Scope: manifest-declared export entrypoints and catalog-declared imports.

Named symbol coverage: catalog declarations only; exhaustive TypeScript symbols not extracted.

Installation status: not verified.

- `@lssm-tech/app.osint-worker` — subpath `.`
  - `unconditional`: `./src/index.ts` — [pinned source](https://github.com/lssm-tech/contractspec/blob/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/osint-worker/src/index.ts); verified source file.

## Package publication metadata

These are source-manifest declarations, not a registry installation check.

```json
{
  "types": null,
  "files": null,
  "publishExports": null
}
```

## Dependencies

Metadata status: complete pinned manifest.

### Runtime dependencies

- [@lssm-tech/integration.job-sources](https://lssm.co/docs/packages/integration.job-sources/): `workspace:^`
- [@lssm-tech/integration.osint-managed-runtime](https://lssm.co/docs/packages/integration.osint-managed-runtime/): `workspace:^`
- [@lssm-tech/integration.osint-store](https://lssm.co/docs/packages/integration.osint-store/): `workspace:^`
- [@lssm-tech/integration.osint-transport](https://lssm.co/docs/packages/integration.osint-transport/): `workspace:^`
- [@lssm-tech/integration.provider-embedding](https://lssm.co/docs/packages/integration.provider-embedding/): `workspace:^`
- [@lssm-tech/integration.provider-web-access](https://lssm.co/docs/packages/integration.provider-web-access/): `workspace:^`
- [@lssm-tech/lib.contracts-integrations](https://lssm.co/docs/packages/lib.contracts-integrations/): `workspace:^`
- [@lssm-tech/lib.contracts-spec](https://lssm.co/docs/packages/lib.contracts-spec/): `workspace:^`
- [@lssm-tech/lib.crypto-utils](https://lssm.co/docs/packages/lib.crypto-utils/): `workspace:^`
- [@lssm-tech/lib.job-source-runtime](https://lssm.co/docs/packages/lib.job-source-runtime/): `workspace:^`
- [@lssm-tech/lib.osint-runtime](https://lssm.co/docs/packages/lib.osint-runtime/): `workspace:^`
- [@lssm-tech/lib.osint-spec](https://lssm.co/docs/packages/lib.osint-spec/): `workspace:^`
- [@lssm-tech/module.osint](https://lssm.co/docs/packages/module.osint/): `workspace:^`

### Peer dependencies

No declarations in the inspected manifest.

### Optional dependencies

No declarations in the inspected manifest.

### Development dependencies

- [@lssm-tech/tool.typescript](https://lssm.co/docs/packages/tool.typescript/): `workspace:*`
- `@types/bun`: `catalog:types`
- `typescript`: `catalog:`

### Peer dependency metadata

```json
{}
```

### Bundled dependencies

```json
null
```

## Package scripts

Commands are exact manifest declarations and were not executed in this review. Script metadata status: complete pinned manifest.

```json
{
  "start": "bun run src/server.ts",
  "test": "bun test",
  "typecheck": "tsgo --noEmit",
  "lint:check": "node ../../../scripts/biome.cjs check ."
}
```

## Declared documentation links

No additional documentation links are declared in the inspected catalogue.

## Existing agent-guide projection

```json
{
  "path": "packages/apps/web-landing/public/llms-packages/app.osint-worker.txt",
  "route": "/llms/app.osint-worker",
  "sourceUrl": "https://github.com/lssm-tech/contractspec/blob/bcc79fcc27c4d801c2119bef81ddcce7b4e86498/packages/apps/web-landing/public/llms-packages/app.osint-worker.txt",
  "blobSha": "2d2c6c798036d7ef63bd2342dc2a3f857eed901f",
  "sourcePin": "bcc79fcc27c4d801c2119bef81ddcce7b4e86498",
  "bodyEqualsReadme": true,
  "aggregateSourcePath": "packages/apps/web-landing/public/llms-full.txt",
  "aggregateBlobSha": "691b1d33efee24c440598b9106c81432f56aafaf"
}
```