Helpers¶
Language detection¶
laya.detect_language is laya.lang.analyse.
analyse ¶
Full detection result for a state.
Returns script, script_profile, language (best effort, may be None),
is_english and non_latin_fraction.
detect_script ¶
Dominant script of text: 'latin', 'han', 'devanagari', ... or 'unknown' if there are no letters.
is_english ¶
True when the English checkpoint can be expected to read this state.
Email¶
clean_email_body ¶
Remove quoted email history, signatures and disclaimers to keep input focused.
email_state ¶
email_state(
subject: str,
body: str,
sender: Optional[str] = None,
clean: bool = True,
**extra,
) -> Dict
Construct a clean state dictionary for email classification.
Question presets¶
email_questions ¶
Preset questions for inbound email triage and threat filtering.
moderation_questions ¶
Preset questions for content safety and moderation.
Shortlisting¶
shortlist_choice ¶
shortlist_choice(
state: Any,
criteria: Any,
embed_fn: Callable[[Sequence[str]], Any],
k: int = DEFAULT_SHORTLIST_K,
*,
instructions: Optional[str] = None,
) -> List[Any]
Return the top-k choice labels for state.
embed_fn maps a list of strings to an array of shape (len(texts), dim).
It is called once, with the query text first and then one string per option in
criteria order. Option strings match render_options for a choice question.
When k is at least the number of labels, every label is returned in its
original order and embed_fn is not called.
Ties keep the earlier label. A zero vector scores 0 and does not outrank a label that came before it.
predict_shortlist ¶
predict_shortlist(
agent: Any,
state: Any,
questions: Dict[str, Dict[str, Any]],
embed_fn: Callable[[Sequence[str]], Any],
k: int = DEFAULT_SHORTLIST_K,
**predict_kwargs: Any,
) -> Dict[str, Any]
Shortlist each choice question, then call predict or system_one once.
Non-choice questions are forwarded unchanged. A choice whose label count is
<= k is forwarded unchanged and does not call embed_fn. The caller's
questions dict is not mutated.
The returned dict is the model result plus a shortlist entry. Probabilities
on a shortlisted choice are over the kept labels only. shortlist[qid] holds
labels (rank order), scores (cosine, or None when nothing was
dropped), k, n, and passthrough.
Extra keyword arguments are forwarded to predict / system_one (for
example model= on a Router).
embed_fn_from_agent ¶
embed_fn_from_agent(
agent: Any, max_length: int = 512, batch_size: int = 32
) -> Callable[[Sequence[str]], np.ndarray]
Mean-pool the checkpoint encoder already loaded on agent.
The callable embeds a list of strings with agent.tok and agent.model.encoder.
It does not run the decision head and does not download weights. A dedicated
bi-encoder passed as embed_fn will usually shortlist better; this helper is
for callers who only have the Laya checkpoint in memory.
Padding positions are excluded from the mean. The encoder's train/eval flag is
left as the caller set it (a loaded Agent is already in eval).
Each call uses the current agent.device, including after CPU fallback.
Calibration and training¶
confidence_from_probs ¶
Normalized Shannon entropy confidence: 1 - H(p) / log(k).
How concentrated the whole distribution is. Useful, but not calibrated: it is not what
temperature scaling fits and not what the reported ECE measures. See answer_confidence.
ece_score ¶
Expected Calibration Error across confidence bins.
render_options ¶
Render option texts in label-index order. Noul semantic order is always [false, true].
proper_reward ¶
proper_reward(
q: Tensor,
target: Tensor,
qtype: Tensor,
mask: Tensor,
w_sph: float = 0.5,
w_rps: float = 1.0,
log_floor: float = -9.21,
) -> torch.Tensor
Strictly proper scoring rule reward: log score + spherical score + ranked probability score.
q: [..., N, K] reported distributions target: [N, K] (one-hot or soft target distributions)
td_lambda_targets ¶
TD(lambda) targets for multi-turn conversation trajectories.