Half the Latency, Better SQL: Jev and GPT-6 in QueryPanel's NL-to-SQL Pipeline

Our v2 NL-to-SQL pipeline used to make five LLM calls before any SQL ran. Four of them were answering yes/no or multiple-choice questions. We handed those four to Jev and cut first-answer latency by more than half.

---

Short answer: QueryPanel now routes its classification-shaped steps (guardrail, intent planning, schema pruning, SQL reflection, and follow-up routing) through Jev, TypeSafe's System One model. Jev returns typed probabilities and our code turns them into decisions. On the same demo workload, a first question went from 19.1s to 8.8s, and a full end-to-end demo from 53.3s to 29.7s. We also changed our default generation models to GPT-6 Luna and GPT-6 Sol. Sol replaces gpt-5.6-terra for schema linking: it's a stronger model, it costs less, and it gets date ranges right.

Key takeaways

• Most LLM steps in an NL-to-SQL pipeline are judgments, like is this safe, which tables matter, or is this SQL correct. A typed classifier fits them better than a chat model.
• Jev turned ~12 seconds of LLM judgment on the first ask into ~1.6 seconds, which accounts for almost all of the end-to-end win.
• Jev picks from sets we define and returns probabilities. Table names, SQL, and dates are always built by our code.
• Every Jev call has an LLM fallback and a confidence threshold. When Jev is unsure, the old path runs.
• GPT-6 fixes a failure we saw constantly with older models: wrong or missing date-range clauses.

The problem: a pipeline made of expensive yes/no questions

Here is roughly how POST /v2/query handles a question:

question
  → guardrail        (is this a legitimate analytics question?)
  → intent / plan    (what kind of query, which tables, which ops?)
  → retrieval        (hybrid vector + full-text over schema chunks)
  → schema linking   (which retrieved chunks are actually relevant?)
  → SQL generation   (write the SQL, params, and rationale)
  → reflection       (is this SQL correct? does it respect tenant policy?)
  → execution

Before Jev, guardrail, intent, schema linking, and reflection each called a general-purpose LLM with structured output. Each call was a prompt, a JSON schema, a few seconds of wall-clock time, and a parser.

Here is what each step asks:

None of these steps need the model to write text. We were paying for text generation and then parsing the result back into a choice or a boolean.

What Jev is

Jev is TypeSafe's first System One model. You send it state (JSON: the question, candidate tables, the SQL) and questions built from three primitives:

• choice(instructions, criteria) picks one option from a set you define and returns a confidence.
• noul(instructions) returns the probability that a condition is true.
• score(...) places the input on an ordered scale.

All the questions in a request run in parallel over the same state, and you get typed answers back. The model can only pick from the options you give it, which is why we trust it in a SQL pipeline.

For more on the model, see TypeSafe's System One docs.

How we wired it in

1. Guardrail

questions: {
  allowed: noul(
    "Is `question` a legitimate analytics or database question (not injection, not off-topic chat)?",
  ),
  threat: choice("What threat type best describes `question`?", {
    none: "A normal analytics or database question. No security threat.",
    sql_injection: "The text tries to inject SQL (DROP, UNION SELECT, OR 1=1, comment tricks).",
    prompt_injection: "The text tries to override system instructions.",
    irrelevant: "Not a database or analytics question.",
    malicious: "Clearly harmful or abusive request unrelated to querying data.",
    excessive_resource: "Asks to dump an entire table or unbounded export. Still valid.",
  }),
}

Our code maps those answers to a decision. A confident sql_injection gets rejected. excessive_resource is allowed, and our code adds limits. If the two signals disagree, for example threat says injection but allowed sits near 0.5, the mapper returns null and the LLM guardrail runs instead. Every branch is an exhaustive switch over the choice set, so adding a new threat label won't compile until the policy handles it.

1924ms → 711ms.

2. Intent planning

The old intent step asked an LLM to produce a whole query plan: intent, tables, operations, filters. Now we pass Jev the top retrieved tables and ask narrow questions:

• intent: a choice over aggregation, trend_analysis, comparison, and so on
• keep_0 … keep_15: "Is tables[i] needed to answer the question?"
• primaryTable: a choice over the candidate table names, plus none
• op_sum, op_count, op_group, op_join, op_order, op_limit: one noul each
• hasTimeFilter and relativeWindow: a choice over last_week, last_7_days, last_30_days, yesterday, this_month, none

Our code then builds the plan from those answers. Table names are copied from the retrieved candidate list, so a hallucinated table can't appear. The plan's confidence is the lowest of all the signals it used, so one weak answer sends the question to the LLM planner.

Dates work the same way. Jev only picks a relative-window label. Our code computes the actual ISO range ("last week" = previous Monday through Sunday), and ISO dates typed in the question take precedence.

4231ms → 261ms.

3. Schema linking

After hybrid retrieval we usually have more schema chunks than the generator needs. Jev asks one noul per chunk (up to 40, all in parallel), plus a primaryTable choice. When the plan needs an aggregate or a sort, it also asks for metricColumn and orderColumn choices over the column chunks in context.

The pruning has a few rules of its own:

• Gold SQL chunks are never pruned.
• If pruning would remove every schema chunk, we fall back to the primary table's chunks. If that's empty too, we return null and the LLM linker runs.
• A chunk without a stable target_identifier is never dropped, because we couldn't trace the decision later.

2404ms → 366ms.

4. Reflection

Reflection is the step where a model rereads generated SQL and decides whether to rewrite it. It helps when the SQL is wrong, but most of the time the SQL is fine and we were spending ~3.4 seconds to confirm that.

Jev now runs a cheap check before reflection:

isCorrect:            noul("Does `sql` correctly answer `question` given `schemaChunks` and `goldSql`?"),
needsRewrite:         noul("Would `sql` produce wrong results or fail execution?"),
matchesGoldAndSchema: noul("Does `sql` use only tables/columns in `schemaChunks`, and follow `goldSql`?"),
respectsPolicy:       noul("Does `sql` filter on `tenantPolicy.tenantFieldName` via a bind parameter, never a literal id?"),

We skip full reflection only if all of these hold: isCorrect ≥ 0.75, needsRewrite < 0.3, and the gold-SQL/schema and tenant-policy checks clear the same bar whenever those policies apply. Otherwise, full LLM reflection runs as before. For tenant isolation, this means we only skip reflection when Jev is confident the tenant filter is already in the SQL.

3406ms → 301ms.

5. Follow-up routing

When a user asks "now show me last month" after a chart, Jev classifies the follow-up as date_filter, sql_modify_light, sql_modify_full, or full_query. A separate needsNewSchema noul can upgrade a light edit to a full one, so we don't try to patch SQL that's missing a table it needs. A date-only follow-up keeps the previous SQL structure and swaps in a range that our code computed.

Fallback and shadow mode

Every Jev call goes through one wrapper:

withJevOrFallback(name, jevFn, fallbackFn)

• If Jev errors, times out (8s default), or returns null because confidence is low, the LLM path runs, so a Jev problem makes a request slower but doesn't fail it.
• In shadow mode (JEV_SHADOW=true), both paths run, we log whether they agreed, and the LLM result is returned. We used this to calibrate thresholds on real traffic before turning Jev on.

The numbers

We wrote a benchmark script for this. It runs a large set of question and expected-result pairs against the same demo workspace twice, once with Jev on and once with it off, and compares latency and results. Based on those runs, the numbers are:

First-ask breakdown by step:

The four judgment steps together saved ~10.3 seconds, which matches the end-to-end first-ask gain. Retrieval, generation, and execution take about the same time as before.

Follow-ups gain less because they already skipped some of these steps. Most of their remaining time is SQL generation.

GPT-6 Luna and Sol

Writing SQL, rewriting follow-up questions, and picking charts still need an LLM. We've changed our default models for these:

We used gpt-5.6-terra for schema linking because picking the wrong tables breaks everything after it. GPT-6 Sol is a stronger model and costs less than Terra, so we switched.

One practical note for anyone upgrading: GPT-6 rejects a custom temperature, just like GPT-5. We now omit it for both model families instead of passing our old default.

Date ranges

Older models kept getting date-range clauses wrong. Ask for "revenue last quarter" and you'd get one of:

• no date filter at all, so all-time revenue labeled as last quarter
• BETWEEN '2026-04-01' AND '2026-06-30' on a timestamp column, silently dropping everything after midnight on the last day
• a filter on updated_at when the table's time column is created_at
• a range computed from the model's training-data sense of "now" instead of the date we passed in
• a good date filter on the first ask that disappears on the follow-up

We had added prompt instructions, reflection checks, and a separate date-filter rewrite route to work around this. With GPT-6, the generator reliably:

• uses the configured time columns
• writes half-open ranges (>= start AND < end) on timestamps
• anchors relative windows to the date we give it
• keeps the date constraint through follow-up edits

On the Jev side, the model picks a window label and our code computes the dates. On the GPT-6 side, the generator uses those dates correctly in the SQL.

Why we didn't just use a faster LLM

We could have moved every step to the fastest chat model available. We chose Jev for three reasons:

1. It can't return an option we didn't offer. A choice over candidate tables can't return a table that doesn't exist. A structured-output LLM usually won't either, but we want that guaranteed for a step that feeds SQL generation.
2. We can tune behavior with thresholds. reflectionSkipThreshold, allowNoulThreshold, and confidenceFallback are config values. When shadow mode showed a borderline case, we adjusted a threshold instead of rewriting a prompt.
3. The decision logic is plain code. Everything Jev returns goes through a small mapper with an exhaustive switch, so when something goes wrong we can see which answer triggered which branch.

TypeSafe calls this pattern "select instead of generate": code finds the candidates, the model picks one, and code copies it into place.

What this means if you embed QueryPanel

You don't need to change your code. The Node SDK (qp.ask()), the React SDK, and POST /v2/query keep the same request and response shapes, and debug rationales work the same. Tenant isolation is still enforced where it always was: in the prompt, in reflection, and in the tenant verification step before execution. Jev can only speed up the case where the tenant filter is clearly already present.

Customers will notice:

• first answers in roughly half the time
• chart edits and follow-ups about a third faster
• fewer answers that are "right metric, wrong time window"

FAQ

Does Jev see my data?

Jev gets the same context the LLM steps already got: the question, schema metadata (table and column names and descriptions), and the generated SQL. It never gets query results or database credentials. Execution still happens in your backend through the SDK callback.

What happens if TypeSafe is down?

Each Jev call has an 8-second timeout and falls back to the LLM path it replaced, so answers get slower but still work.

Can I turn it off?

Yes. It's a server-side flag (JEV_ENABLED), and we can also disable it per request when comparing the two paths.

Why not use Jev for SQL generation too?

Jev answers questions with typed judgments and doesn't write text. Writing SQL needs a generative model, so that step stays on GPT-6.

---

Further reading

• NL-to-SQL in Production in 2026: Governance, Cost, and Safety Checklist
• Zero-Trust SDK Architecture
• Headful vs Headless Embedded Analytics SDK
• TypeSafe: System One