Building a Chatbot Message Router with TypeSafe's Jev API: A Field Report
Over the past day I built a small but non-trivial prototype: a support-chat router that
classifies incoming customer messages along four independent dimensions and routes
them to different handling tiers - blocked (adversarial), declined-by-simple-model (offtopic),
answered-by-simple-model (chit-chat, no context), or escalated to the main model. The classification engine is TypeSafe's Jev "System One" model, accessed through its
HTTP API and the Noul primitive (a yes/no judgment that returns a probability
rather than a label). The main goal here is to reduce model costs and add a guardrail
layer. Something I was on the lookout for some time, until a saw the Jev demos.
What the API actually gives you
The mental model is Jev doesn't generate text and it doesn't explain itself - you send a `state` (in my case, a two-turn conversation snippet) plus a map of named questions, and you get back one typed answer per question, each with an actual probability. A Noul near 0.5 doesn't mean "medium severity," it means the model is genuinely unsure whether the answer is yes or no. That distinction matters a lot once you start setting thresholds for a live decision. The four Nouls I ended up with were is_chitchat, is_negative_feedback, is_offtopic, and is_malicious - each answered independently, in parallel, on the same conversation state, in a single request. That "ask everything at once" pattern was one of the best parts of the API: no back-and-forth, no per-question round trips, just one POST with a `questions` map and one JSON response with a matching `answers` map.
Wording is the real lever
The most important lesson: the criteria text is where the actual behavior lives, not the code. Early on I tried to fix classification errors by tuning thresholds. That was the wrong move almost every time. A Noul needing an unusually low threshold (e.g. 0.25 instead of 0.5) to catch an obvious case is a signal that the instructions/criteria wording is miscalibrated or the ground-truth label is wrong - not that the threshold needs adjusting. Once I accepted that and iterated on wording instead (with a genuinely useful trick: running two GitHub Copilot CLI model instances against each other, one writing criteria and one red-teaming it with edge cases), accuracy moved from the 60s% into the 90s% range across all four dimensions, without ever touching a threshold. That also required quite a lot of manual guidance.
Overlap is fine if your code owns the policy
A design question that felt tricky at first: is_offtopic and is_malicious can both fire on the same message (e.g. a prompt-injection payload is arguably both "off-topic" and "malicious"). I initially treated this as a wording bug. It wasn't. The actual fix was architectural: give the routing code an explicit precedence order (malicious → blocked, then off-topic → declined, then chitchat → simple-reply, else → main model) and let the Nouls be honestly independent judgments. TypeSafe's own guidance - keep policy in code, keep judgments reusable and orthogonal - turned out to be exactly right here. Once routing precedence lived in code instead of being smuggled into prompt wording, the false "overlap" complaints disappeared.
Practical friction points
A few things worth flagging for anyone doing this on Windows/PowerShell:- Timing: measuring latency via `Invoke-RestMethod` includes .NET client/runtime overhead. Asking `curl.exe` directly with `-w "%{time_total}"` gives a cleaner number - pure network + server time, which is what you actually want when deciding whether to parallelize requests.
- Token usage: the response includes a `usage.input_tokens` / `usage.output_tokens` block. Useful for understanding request shape and the expected billing (output tokens are free, there is no token caching). Overall the pricing looks reasonable. I sent 2500 requests (since my harness is 200 questions that is not a lot) they produced 2.8M tokens for an API cost of $0.11. CSV-driven test harnesses work well: a 200-row conversation-pair corpus (100 English, 100 German), driven through one script, with expected-route ground truth baked into the CSV, made it trivial to rerun the same regression test after every wording change and get an immediate accuracy delta. The harness was AI generated by a generative model.
- The measured 700ms latency is good for a cross-atlantic LLM, but its still a quite steep hit. Probably this would be less of an issue with a european service, or even better a local model. For customer chats and also things like PII classification the model is genuine a good idea, but I would require a local model or an European contract for that.
- I have a 90% success ratio and no risky false detections. This makes the message router probably useable in real world scenarios - but keep in mind I only tested with synthetic chats. But positive to note: the german test cases had the same success ratio than english.
- Having Copilot CLI run powershell scripts and read and delete files is a pain on our managed systems. I frequently had to execute the scripts myself since they seem to have been delayed or blocked (without feedback) by EDR or other scanners.
- The Copilot Harness seem to redact `Authorisation:` headers with **** even if the value is a benin $env:APIKEY variable. Copilot noticed that itself and worked around it, but had quite a few extra steps due to this.