Skip to content

TypeScript SDK design

laya-client is a dependency-free HTTP client for a self-hosted laya-serve server. It uses the existing prediction and routing endpoints and adds no Python production code or server dependencies. The npm package starts at version 0.1.0, independently of Python releases.

Boundaries

Component Responsibility
sdk/typescript Question/answer types, presets, validation, native fetch, errors and cancellation
laya/serve.py Existing HTTP endpoint, Bearer authentication, health and request limits
laya/router.py Checkpoint selection, loading and inference routing
laya/agent.py Tokenization, PyTorch inference and calibrated answer formatting
laya/presets.py Source for the five generated TypeScript question presets
flowchart LR
    A[JavaScript or TypeScript application] --> B[laya-client]
    B -->|POST /v1/systemone, /v1/systemone/batch or /v1/route| C[Existing Laya server]
    C --> E[Router and local checkpoint]

The SDK exports predict, predictBatch, route and a Laya-only health probe. It ships ESM, CommonJS and declarations, retaining inferred question IDs and choice labels. Use laya-client when a JavaScript or TypeScript application talks over HTTP to self-hosted Python laya-serve. Use laya-ts when inference must run directly inside JavaScript through its local ONNX runtime, without a Python server.

Shared contract

Requests contain state and questions. Unless configured or supplied for a prediction, laya-client omits model, letting laya-serve select a local checkpoint automatically. A client-wide or per-call model can select a local checkpoint, as can the other per-request controls in the table below. Choice label arrays are normalized to maps with null descriptions before transport.

Responses preserve model, answers, and token usage. Laya's routing and answer action fields are optional extensions; Noul confidence is optional too. Choice and Score confidence, distributions, and Score legends remain required. Every answer carries answer_confidence, the max(p) mass on the reported answer, which is the same quantity on all three question types. A call that passed min_confidence reports abstention and abstention_threshold on each of its answers and low_confidence: true on the ones below the threshold; with no threshold set, none of those three keys are sent, and that absence is the report. Optional extensions are validated when present.

predict posts /v1/systemone, predictBatch posts /v1/systemone/batch, route posts /v1/route, and health gets /health. Each call makes one request. route(state, questions = {}, options = {}) reads the deployed Router's current selection without requesting inference. Its RouteSnapshot contains model, repo and reason, with diagnostic when explain: true. Registered names are valid and repo can be null for a source-less attached model. This is separate from prediction's richer RouteDecision; the snapshot does not include detection or workflow.

Routing accepts model, task, lang, langGuess, boolean explain, signal and timeoutMs. Unsupported option keys are refused locally. Questions may be empty; only their JSON/object shape and choice-array shorthand are checked locally, so a route result does not validate a later prediction. The server still enforces its request limits. An explanation is the final post-hook JSON value, including null, rather than a closed set of detector reasons. Missing requested diagnostics are refused as an invalid response. Diagnostics can carry caller-chosen field names, and route reasons can reflect control values; operator hooks can add their own JSON. This is a snapshot, not a promise that a later prediction will choose the same checkpoint. Older servers without the endpoint preserve their 404/405 errors; no call falls back to prediction or local detection. Abort and timeout stop waiting and do not cancel server-side routing or hooks.

The controls shared by both prediction endpoints are per-request, and each is sent only when the caller supplied the option -- an absent option leaves the deployment's own Router(...) settings in charge instead of overriding them with a client-side default:

option request field
model model
task task
lang lang
langGuess lang_guess
maxLen max_len
headMaxLen head_max_len
minConfidence min_confidence

An option that cannot mean anything is refused locally, before the request goes out: a blank task, a budget that is not a positive integer, a threshold outside [0, 1], or a threshold map that is empty or holds a value outside [0, 1]. Nothing is silently ignored. Laya's public /health returns status, loaded, revisions, device, device_is_preference, checkpoint_devices, and cpu_fallbacks to a caller it can authenticate, plus idle_unload_seconds and idle_seconds on a deployment that sets LAYA_IDLE_UNLOAD_SECONDS -- seven keys, or nine there, exactly as HTTP API documents them. An anonymous probe on a deployment with LAYA_API_KEY set gets {"status": "ok"} alone, because the withheld fields name resident checkpoints, their revisions and the host's device state; that body is a healthy response, and health() returns it rather than raising. Prediction never probes health first.

FastAPI detail strings and validation arrays are preserved as LayaAPIError messages/details. Structured error envelopes from compatible backends are also accepted. Requests have configurable deadlines and caller cancellation, and are never retried automatically.

Batch predictions

predictBatch(states, questions, options) sends a nonempty array of independent states and one shared question set. The return type is BatchPrediction<Q>: results[i] belongs to states[i] and retains the same question IDs and choice labels as predict. total_usage contains only aggregate input_tokens and output_tokens; truncation metadata belongs to each result's usage. Counts and every prediction are validated before returning anything. A bad item reports its results[i] index in LayaResponseError. Unknown response extensions are preserved.

All prediction controls above apply to the whole batch. batchSize maps to batch_size, and sortByLength to sort_by_length: these control server-side inference batching, never HTTP splitting, and length sorting still returns input order. Both are omitted unless supplied. Changing the inference batch shape can change floating-point results.

The client never splits, retries, or falls back to single predictions, including on timeout or an unsupported endpoint's 404/405. A batch fails as a whole. Server limits remain authoritative; the current server's 64-state default is not a client-side cap. Cancellation stops waiting, not work already running on the server. See the SDK guide for an example.

Verification and release

Unit tests cover request construction, all answer shapes, Laya extensions, FastAPI errors, JSON validation, deadlines and cancellation, and hold this page's control table to the fields the client actually puts on the wire. Type checks cover optional metadata, inferred answer types, and ESM/CommonJS consumers. The live integration test starts the unchanged laya.serve application with a tiny offline checkpoint, compares SDK predictions against direct Python inference, and exercises routing, presets, authentication and request limits. Batch parity uses the same inference batch shape, in extended and strict modes, and includes tournament results. CI also checks a pinned v0.4.7 server and installs the npm tarball into an isolated ESM/CommonJS and TypeScript consumer. CI runs the SDK checks on Node.js 22 and 24.

Tiny random weights verify transport and numerical parity, not pretrained quality or performance.

See the SDK guide for setup, examples, and npm publishing. The package will be published under the laya-client name. Python release workflows are unchanged.