TypeScript SDK design¶
laya-client is a dependency-free HTTP client for a self-hosted laya-serve
server. It uses the existing prediction and routing endpoints and adds no Python production
code or server dependencies.
The npm package starts at version 0.1.0, independently of Python releases.
Boundaries¶
| Component | Responsibility |
|---|---|
sdk/typescript |
Question/answer types, presets, validation, native fetch, errors and cancellation |
laya/serve.py |
Existing HTTP endpoint, Bearer authentication, health and request limits |
laya/router.py |
Checkpoint selection, loading and inference routing |
laya/agent.py |
Tokenization, PyTorch inference and calibrated answer formatting |
laya/presets.py |
Source for the five generated TypeScript question presets |
flowchart LR
A[JavaScript or TypeScript application] --> B[laya-client]
B -->|POST /v1/systemone, /v1/systemone/batch or /v1/route| C[Existing Laya server]
C --> E[Router and local checkpoint]
The SDK exports predict, predictBatch, route and a Laya-only health probe. It ships ESM,
CommonJS and declarations, retaining inferred question IDs and choice labels.
Use laya-client when a JavaScript or TypeScript application talks over HTTP to
self-hosted Python laya-serve. Use laya-ts when inference must run directly
inside JavaScript through its local ONNX runtime, without a Python server.
Shared contract¶
Requests contain state and questions. Unless configured or supplied for a
prediction, laya-client omits model, letting laya-serve select a local
checkpoint automatically. A client-wide or per-call model can select a local
checkpoint, as can the other per-request controls in the table below. Choice label
arrays are normalized to maps with null descriptions before transport.
Responses preserve model, answers, and token usage. Laya's routing and
answer action fields are optional extensions; Noul confidence is optional too.
Choice and Score confidence, distributions, and Score legends remain required.
Every answer carries answer_confidence, the max(p) mass on the reported answer,
which is the same quantity on all three question types. A call that passed
min_confidence reports abstention and abstention_threshold on each of its
answers and low_confidence: true on the ones below the threshold; with no
threshold set, none of those three keys are sent, and that absence is the report.
Optional extensions are validated when present.
predict posts /v1/systemone, predictBatch posts /v1/systemone/batch, route posts /v1/route, and health gets /health.
Each call makes one request. route(state, questions = {}, options = {}) reads the deployed
Router's current selection without requesting inference. Its RouteSnapshot contains model,
repo and reason, with diagnostic when explain: true. Registered names are valid and
repo can be null for a source-less attached model. This is separate from prediction's richer
RouteDecision; the snapshot does not include detection or workflow.
Routing accepts model, task, lang, langGuess, boolean explain, signal and timeoutMs.
Unsupported option keys are refused locally. Questions may be empty; only their JSON/object
shape and choice-array shorthand are checked locally, so a route result does not validate a
later prediction. The server still enforces its request limits. An explanation is the final
post-hook JSON value, including null, rather than a closed set of detector reasons. Missing
requested diagnostics are refused as an invalid response. Diagnostics can carry caller-chosen
field names, and route reasons can reflect control values; operator hooks can add their own JSON.
This is a snapshot, not a promise that a later prediction will choose the same checkpoint.
Older servers without the endpoint preserve their 404/405 errors; no call falls back to prediction
or local detection. Abort and timeout stop waiting and do not cancel server-side routing or hooks.
The controls shared by both prediction endpoints are
per-request, and each is sent only when the caller supplied the option -- an absent option leaves the
deployment's own Router(...) settings in charge instead of overriding them with a client-side default:
| option | request field |
|---|---|
model |
model |
task |
task |
lang |
lang |
langGuess |
lang_guess |
maxLen |
max_len |
headMaxLen |
head_max_len |
minConfidence |
min_confidence |
An option that cannot mean anything is refused locally, before the request goes out: a blank task, a
budget that is not a positive integer, a threshold outside [0, 1], or a threshold map that is empty
or holds a value outside [0, 1]. Nothing is silently ignored. Laya's
public /health returns status, loaded, revisions, device, device_is_preference,
checkpoint_devices, and cpu_fallbacks to a caller it can authenticate, plus
idle_unload_seconds and idle_seconds on a deployment that sets LAYA_IDLE_UNLOAD_SECONDS --
seven keys, or nine there, exactly as HTTP API documents them. An anonymous probe
on a deployment with LAYA_API_KEY set gets {"status": "ok"} alone, because the withheld fields
name resident checkpoints, their revisions and the host's device state; that body is a healthy
response, and health() returns it rather than raising. Prediction never probes health first.
FastAPI detail strings and validation arrays are preserved as LayaAPIError
messages/details. Structured error envelopes from compatible backends are also
accepted. Requests have configurable deadlines and caller cancellation, and
are never retried automatically.
Batch predictions¶
predictBatch(states, questions, options) sends a nonempty array of independent states and one
shared question set. The return type is BatchPrediction<Q>: results[i] belongs to states[i]
and retains the same question IDs and choice labels as predict. total_usage contains only
aggregate input_tokens and output_tokens; truncation metadata belongs to each result's usage.
Counts and every prediction are validated before returning anything. A bad item reports its
results[i] index in LayaResponseError. Unknown response extensions are preserved.
All prediction controls above apply to the whole batch. batchSize maps to batch_size, and
sortByLength to sort_by_length: these control server-side inference batching, never HTTP
splitting, and length sorting still returns input order. Both are omitted unless supplied.
Changing the inference batch shape can change floating-point results.
The client never splits, retries, or falls back to single predictions, including on timeout or an unsupported endpoint's 404/405. A batch fails as a whole. Server limits remain authoritative; the current server's 64-state default is not a client-side cap. Cancellation stops waiting, not work already running on the server. See the SDK guide for an example.
Verification and release¶
Unit tests cover request construction, all answer shapes, Laya extensions,
FastAPI errors, JSON validation, deadlines and cancellation, and hold this
page's control table to the fields the client actually puts on the wire.
Type checks cover optional metadata, inferred
answer types, and ESM/CommonJS consumers. The live integration test starts the
unchanged laya.serve application with a tiny offline checkpoint, compares SDK
predictions against direct Python inference, and exercises routing, presets,
authentication and request limits. Batch parity uses the same inference batch shape, in
extended and strict modes, and includes tournament results. CI also checks a pinned v0.4.7
server and installs the npm tarball into an isolated ESM/CommonJS and TypeScript consumer.
CI runs the SDK checks on Node.js 22 and 24.
Tiny random weights verify transport and numerical parity, not pretrained quality or performance.
See the SDK guide for setup, examples, and npm
publishing. The package will be published under the laya-client name. Python release workflows
are unchanged.