<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Automation on vishctl</title><link>https://vishctl.dev/tags/automation/</link><description>Recent content in Automation on vishctl</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Mon, 21 Sep 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://vishctl.dev/tags/automation/index.xml" rel="self" type="application/rss+xml"/><item><title>TypeSafe’s Jev: Early Access to AI Built for Decisions</title><link>https://vishctl.dev/posts/typesafe-jev-system-one-models/</link><pubDate>Sat, 19 Sep 2026 14:00:00 +0800</pubDate><guid>https://vishctl.dev/posts/typesafe-jev-system-one-models/</guid><category>ai</category><category>inference</category><category>ai-infra</category><category>typesafe</category><category>automation</category><description>My early-access look at TypeSafe’s Jev: Noul, Choice, and Score in the playground, the coding-agent skill, and what the performance claims do and do not establish.</description><content:encoded><![CDATA[<p>An incident comes in. Your software needs an owner, an urgency level, and a decision on human review. It needs those answers before the request times out.</p>
<p>That is the part of AI infrastructure that makes TypeSafe interesting to me.</p>
<p>On September 15, 2026, TypeSafe AI <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">announced</a> <strong>Jev</strong>, its first <strong>System One model</strong>, in early access. TypeSafe describes a new architecture and a parallel sampler, built for decisions rather than conversation.</p>
<p>I got early access, and on September 21 I ran the playground&rsquo;s Noul, Choice, and Score examples with <code>jev-latest</code>. The screenshots below come from those runs. They make the interface concrete; they say nothing about production accuracy or the headline speedups.</p>
<p><strong>In simple terms, Jev makes small decisions your software can use immediately.</strong> Give it context and allowed choices; it returns answers with probabilities. Your code decides what happens next, including when to ask a person.</p>
<h2 id="what-does-a-system-one-model-do">What does a System One model do?</h2>
<p><strong>Jev is not a chat model.</strong> It returns typed decisions and probabilities instead of conversational replies, and your application uses those answers to choose actions or hand work to tools, other models, or a person (<a href="https://docs.typesafe.ai/introduction">model introduction</a>).</p>
<figure class="post-screenshot">
    <img loading="lazy" src="https://vishctl.dev/images/posts/typesafe-jev/01-jev-software-workflow.png"
         alt="Three-panel diagram: application state and typed questions enter Jev, which returns decisions and probabilities to application code. The code applies thresholds, selects branches, calls tools or models, routes to a person, and logs results."/> <figcaption>
            <p>Jev supplies narrow judgments. Application code controls execution and escalation; the brain illustration is symbolic.</p>
        </figcaption>
</figure>

<p>You provide <strong>state</strong> and <strong>typed questions</strong>. State is the information to inspect, such as a support conversation and account record. Questions define the judgments to make.</p>
<p>Each question is evaluated independently against the same state, and application code combines the judgments into a workflow.</p>
<p>The three primitives make the idea easier to see:</p>
<div class="table-scroll">
  <table>
    <thead>
        <tr>
            <th scope="col">Primitive</th>
            <th scope="col">Example question</th>
            <th scope="col">Result</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td><strong>Choice</strong></td>
            <td>Which team should receive this ticket?</td>
            <td>A selected option, probabilities over the options, and confidence</td>
        </tr>
        <tr>
            <td><strong>Score</strong></td>
            <td>How severe is this issue on our defined scale?</td>
            <td>A score, probabilities over the levels, and confidence</td>
        </tr>
        <tr>
            <td><strong>Noul</strong></td>
            <td>Does the message report an outage?</td>
            <td>A value from 0 to 1 representing the probability of yes</td>
        </tr>
    </tbody>
  </table>
</div>
<p>Noul has no separate confidence field. Choice picks from predefined alternatives and Score from an ordered scale, so answers are constrained rather than free text (<a href="https://docs.typesafe.ai/primitives">primitives docs</a>).</p>
<p>An illustrative incident workflow could look like this:</p>
<figure class="mermaid-figure">
  <div class="mermaid">flowchart LR
    S["Alert and service context"] --> J["Jev"]
    J --> T["Choice: owning team"]
    J --> U["Score: urgency"]
    J --> O["Noul: outage reported?"]
    T --> P["Application policy"]
    U --> P
    O --> P
    P --> R["Route to a team"]
    P --> H["Request human review"]</div>
  <figcaption>A possible incident-routing design. Jev supplies narrow judgments; application code controls the workflow.</figcaption>
</figure>

<p>Ownership rules, escalation permissions, and audit logging stay explicit in code. A permission change does not depend on the model interpreting a revised prompt correctly.</p>
<h3 id="a-documented-request-and-its-answers">A documented request and its answers</h3>
<p>TypeSafe&rsquo;s <a href="https://docs.typesafe.ai/sdk/python">Python SDK guide</a> includes this three-question support example. Install <code>typesafe-sdk</code> and set <code>TYPESAFE_API_KEY</code> in your environment first. The request below follows that example; the print statements expose the returned probabilities too.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">typesafe_sdk</span> <span class="kn">import</span> <span class="n">Choice</span><span class="p">,</span> <span class="n">Noul</span><span class="p">,</span> <span class="n">Score</span><span class="p">,</span> <span class="n">TypeSafeClient</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">with</span> <span class="n">TypeSafeClient</span><span class="p">()</span> <span class="k">as</span> <span class="n">client</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="n">system_one</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="n">state</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;document&#34;</span><span class="p">:</span> <span class="s2">&#34;I was charged twice. Please fix this ASAP.&#34;</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="n">questions</span><span class="o">=</span><span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="s2">&#34;billing&#34;</span><span class="p">:</span> <span class="n">Noul</span><span class="p">(</span><span class="n">instructions</span><span class="o">=</span><span class="s2">&#34;Is this ticket about billing?&#34;</span><span class="p">),</span>
</span></span><span class="line"><span class="cl">            <span class="s2">&#34;tone&#34;</span><span class="p">:</span> <span class="n">Choice</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">                <span class="n">instructions</span><span class="o">=</span><span class="s2">&#34;What is the customer&#39;s tone?&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="n">criteria</span><span class="o">=</span><span class="p">{</span><span class="s2">&#34;calm&#34;</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s2">&#34;frustrated&#34;</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s2">&#34;angry&#34;</span><span class="p">:</span> <span class="kc">None</span><span class="p">},</span>
</span></span><span class="line"><span class="cl">            <span class="p">),</span>
</span></span><span class="line"><span class="cl">            <span class="s2">&#34;urgency&#34;</span><span class="p">:</span> <span class="n">Score</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">                <span class="n">instructions</span><span class="o">=</span><span class="s2">&#34;How urgent is this ticket?&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="n">criteria</span><span class="o">=</span><span class="p">[</span><span class="s2">&#34;can wait&#34;</span><span class="p">,</span> <span class="s2">&#34;this week&#34;</span><span class="p">,</span> <span class="s2">&#34;today&#34;</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">            <span class="p">),</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">nouls</span><span class="p">[</span><span class="s2">&#34;billing&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">noul</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">choices</span><span class="p">[</span><span class="s2">&#34;tone&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">choice</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">choices</span><span class="p">[</span><span class="s2">&#34;tone&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">probabilities</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">scores</span><span class="p">[</span><span class="s2">&#34;urgency&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">score</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">scores</span><span class="p">[</span><span class="s2">&#34;urgency&#34;</span><span class="p">]</span><span class="o">.</span><span class="n">probabilities</span><span class="p">)</span>
</span></span></code></pre></div><p>One call evaluates the same ticket three ways: a yes probability, a distribution over tones, and a distribution over urgency levels. This is the documented example, not a run from my account.</p>
<h2 id="what-i-saw-in-early-access">What I saw in early access</h2>
<p>The console home brings together the playground, example workflows, API keys, usage, and an agent setup link. In the playground, state and questions sit on the left, with typed answers on the right.</p>
<figure class="post-screenshot"><a href="https://vishctl.dev/images/posts/typesafe-jev/02-early-access-home.png">
    <img loading="lazy" src="https://vishctl.dev/images/posts/typesafe-jev/02-early-access-home.png"
         alt="TypeSafe early-access console home with playground examples, agent setup, API keys, and usage"/> </a><figcaption>
            <p>My early-access console on September 21, 2026. Open any screenshot for the full-size view.</p>
        </figcaption>
</figure>

<h3 id="noul-an-ice-cream-sandwich-gets-35-true">Noul: an ice cream sandwich gets 35% true</h3>
<p>The first example asks whether an ice cream sandwich is a sandwich. The state describes ice cream between cookies, wafers, or cake. The criteria matter: the true condition refers to a filling between structural starch, while the false condition explicitly includes wafers and cookies as non-bread wrappers. Those descriptions leave room for conflicting interpretations.</p>
<p>The result is <strong>35% true</strong>. This is the model&rsquo;s probability for the yes/no question under those definitions. It is not a measure of how much of the dessert is a sandwich, and a single answer cannot tell us whether the probabilities are calibrated.</p>
<figure class="post-screenshot"><a href="https://vishctl.dev/images/posts/typesafe-jev/03-noul-sandwich.png">
    <img loading="lazy" src="https://vishctl.dev/images/posts/typesafe-jev/03-noul-sandwich.png"
         alt="Jev playground showing the ice cream sandwich state, true and false criteria, and a Noul result of 35 percent true"/> </a><figcaption>
            <p>Noul returns a probability for a defined yes/no judgment. The definition needs as much attention as the result.</p>
        </figcaption>
</figure>

<h3 id="choice-richer-descriptions-change-the-leading-option">Choice: richer descriptions change the leading option</h3>
<p>The next example uses a short state about an approaching tornado and asks for the sky&rsquo;s color. With simple option labels, <strong>seafoam green leads at 63%</strong>, followed by gray at 34% and indigo at 3%. The displayed confidence is <strong>56%</strong>.</p>
<p>A second question in the same request uses structured instructions and richer option descriptions. There, <strong>indigo leads at 42%</strong>, followed by seafoam green at 34% and gray at 24%, with <strong>31% confidence</strong>.</p>
<p>Question design is part of the application. Both the instructions and the criteria changed here, so this run can&rsquo;t tell which change caused the shift, or which answer is better. It is a good argument for versioning questions and testing every change against known cases.</p>
<figure class="post-screenshot"><a href="https://vishctl.dev/images/posts/typesafe-jev/04-choice-sky-color.png">
    <img loading="lazy" src="https://vishctl.dev/images/posts/typesafe-jev/04-choice-sky-color.png"
         alt="Two sky-color Choice answers: seafoam green leads at 63 percent with 56 percent confidence, while the question with descriptions selects indigo at 42 percent with 31 percent confidence"/> </a><figcaption>
            <p>Choice exposes the distribution as well as the selected option. The winning probability and confidence are different quantities.</p>
        </figcaption>
</figure>

<h3 id="score-a-distribution-across-five-levels">Score: a distribution across five levels</h3>
<p>The third example describes the Naruto macaque photograph scenario and asks two separate questions about contribution to the image. On five levels from None (0) to Completely (4), the result gives <strong>Naruto 3.46 out of 4</strong> with <strong>55% confidence</strong>, and <strong>the human photographer 1.18 out of 4</strong> with <strong>69% confidence</strong>.</p>
<p>The distributions make the scores easier to read: Naruto&rsquo;s answer concentrates on levels 3 and 4, while the photographer&rsquo;s concentrates on level 1. These are separate judgments against the supplied scenario and rubric; they are not shares that must add up to 100% or determinations of copyright ownership.</p>
<figure class="post-screenshot"><a href="https://vishctl.dev/images/posts/typesafe-jev/05-score-contribution.png">
    <img loading="lazy" src="https://vishctl.dev/images/posts/typesafe-jev/05-score-contribution.png"
         alt="Score playground results showing Naruto at 3.46 out of 4 and the photographer at 1.18 out of 4, with probabilities across five contribution levels"/> </a><figcaption>
            <p>Score can fall between levels. The full distribution preserves information that a single rounded rating would hide.</p>
        </figcaption>
</figure>

<p>TypeSafe <a href="https://docs.typesafe.ai/primitives/score">defines the score</a> as the probability-weighted average of the level indices. The UI rounds the probabilities, so recomputing from the screenshot gives a slightly different number.</p>
<p>The three captures show timing pairs of <code>84ms + 278ms</code>, <code>160ms + 233ms</code>, and <code>158ms + 234ms</code>. The UI doesn&rsquo;t label the two parts, and three runs are not a latency benchmark.</p>
<h2 id="using-the-typesafe-agent-skill">Using the TypeSafe agent skill</h2>
<p>TypeSafe also provides an <a href="https://docs.typesafe.ai/agent-skill">agent skill</a> for coding agents, including Codex and Claude Code. It supplies API context, question types, workflow patterns, and evaluation guidance for writing integrations.</p>
<p>The documented installation command for Codex and other supported agents is:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">npx skills add typesafe-ai/skills --skill typesafe-ai
</span></span></code></pre></div><p>Select your agent when prompted; installation is project-local by default. The guide also covers installing it as a Claude Code plugin. Pick one method, not both.</p>
<p>For an incident-routing experiment, I would start with a prompt like:</p>
<blockquote>
<p>Use the TypeSafe skill to propose an incident-routing experiment in shadow mode. Batch independent questions about the same incident into one request. Keep questions and thresholds together for review, and compare proposed routes against historical labels.</p>
</blockquote>
<p>The skill helps an agent use the API. Questions, criteria, and thresholds still need review and evaluation on the application&rsquo;s own data.</p>
<h2 id="how-is-it-different-from-an-llm-with-structured-outputs">How is it different from an LLM with structured outputs?</h2>
<p>LLMs can already enforce output structure. OpenAI&rsquo;s <a href="https://openai.com/index/introducing-structured-outputs-in-the-api/">Structured Outputs</a> constrains generated tokens to a supplied schema. That guarantees the shape of the answer, not that the values are right. So valid JSON alone is not the breakthrough.</p>
<div class="table-scroll">
  <table>
    <thead>
        <tr>
            <th scope="col">Aspect</th>
            <th scope="col">General text-generating LLM</th>
            <th scope="col">Jev&rsquo;s documented approach</th>
        </tr>
    </thead>
    <tbody>
        <tr>
            <td>Output</td>
            <td>Text, including schema-constrained text where supported</td>
            <td>Typed decisions and distributions</td>
        </tr>
        <tr>
            <td>Generation</td>
            <td>Autoregressive token generation</td>
            <td>Parallel decision outputs</td>
        </tr>
        <tr>
            <td>Interface</td>
            <td>Prompts, messages, tools, and schemas</td>
            <td>State plus focused typed questions</td>
        </tr>
        <tr>
            <td>Application design</td>
            <td>Can generate explanations and proposed plans</td>
            <td>Code combines individual judgments</td>
        </tr>
    </tbody>
  </table>
</div>
<p>The training objective differs too. TypeSafe&rsquo;s <strong>Reinforcement Learning for Calibrated Decisions (RLCD)</strong> targets decisions and calibrated probabilities, where RLHF optimizes for human preferences and RLVR for verifiable rewards (<a href="https://docs.typesafe.ai/introduction/machine-learning-primer">TypeSafe&rsquo;s ML primer</a>). Real models often mix these objectives.</p>
<p>The real question is whether this specialization improves latency, cost, and uncertainty estimates for the same job.</p>
<h2 id="why-could-this-be-a-big-deal">Why could this be a big deal?</h2>
<h3 id="latency-changes-where-ai-can-run">Latency changes where AI can run</h3>
<p>TypeSafe <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">reports</a> <strong>70 to 500 ms</strong> end-to-end latency and <strong>$0.042 per million input tokens</strong> at launch, with no output-token charge. Its headline workflow results are <strong>193.6× faster</strong> and <strong>444.6× cheaper</strong>.</p>
<p>Here is why latency matters. Five dependent 200 ms calls take one second; five 2-second calls take ten. That gap decides whether a workflow fits inside an interactive request or has to become a background job. Independent questions can run in parallel, but dependent decisions still run in stages, and network latency, retries, and queueing still add up.</p>
<h3 id="uncertainty-becomes-something-code-can-use">Uncertainty becomes something code can use</h3>
<p><a href="https://docs.typesafe.ai/introduction/machine-learning-primer">Calibration</a> means outcomes given an 80% probability should happen about 80% of the time across many comparable predictions. It describes groups of predictions, not a single answer.</p>
<p>Jev&rsquo;s <a href="https://docs.typesafe.ai/confidence"><code>confidence</code></a> field for Choice and Score summarizes the shape of the distribution. It is not the measured probability that the answer is correct, so thresholds need testing on your own data.</p>
<p>The practical test is how many decisions you can automate at an acceptable error rate, with good uncertainty estimates flagging the cases that need a person.</p>
<h3 id="the-unit-of-value-becomes-a-completed-decision">The unit of value becomes a completed decision</h3>
<p>For an incident router, I care about decisions per second, cost per correct routing, and the share of cases needing manual review. Token throughput only tells part of that story.</p>
<p>Jev encourages evaluating AI as a software component with a specific job. Constrained outputs simplify inspection, even though the judgments remain probabilistic.</p>
<h2 id="what-the-benchmarks-do-and-do-not-establish">What the benchmarks do and do not establish</h2>
<p><strong>The performance figures are TypeSafe&rsquo;s own. I have not benchmarked Jev myself.</strong> Treat them as a reason to evaluate it, not a production guarantee.</p>
<p>TypeSafe&rsquo;s <a href="https://evals.typesafe.ai/">evaluation</a> covers security incidents, agent trace observability, invoice processing, and customer service, comparing structured workflows with standalone prompts. Reference labels are the average of GPT-6 Astra and Claude Fable 5.1 responses at high thinking settings; other models run at provider-default reasoning settings. So the score measures agreement with other models, not verified business outcomes.</p>
<p>TypeSafe also <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">says</a> the headline gains are at the high end of expectations: its team built the workflows, and the LLM comparison adapter requests probabilities, which adds cost and latency. Real gains will depend on the task, region, input length, and load.</p>
<h2 id="does-type-safety-eliminate-mistakes">Does type safety eliminate mistakes?</h2>
<p><strong>A valid decision can still be the wrong decision.</strong></p>
<p>If the only permitted teams are <code>platform</code>, <code>network</code>, and <code>application</code>, a constrained system can prevent an invented fourth team. It can still send a network incident to <code>application</code>.</p>
<p>The launch&rsquo;s <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">hallucination guarantee</a> is about matching the schema, not about reasoning well or reading the input correctly.</p>
<p>Bad context, ambiguous policies, and incomplete choices still cause problems. An <code>other</code> option and a review path give uncertain cases somewhere to go.</p>
<h2 id="who-is-behind-typesafe">Who is behind TypeSafe?</h2>
<p>The <a href="https://typesafe.ai/team">founders</a> are <strong>Diogo Almeida, CEO; Sasha Sheng, COO; and Erik Gafni, CTO</strong>. Almeida worked at OpenAI and Google Brain, Sheng at Meta/FAIR, and Gafni founded Ravel.</p>
<p>Almeida also co-authored the 2022 <a href="https://arxiv.org/abs/2203.02155">InstructGPT paper</a> on training models to follow instructions with human feedback, which makes this move toward decisions consumed by software more interesting.</p>
<h2 id="what-i-would-test-next">What I would test next</h2>
<p>After these playground examples, my next test would be <strong>incident routing in shadow mode</strong>. Jev proposes a team and urgency; the existing process makes the actual decision.</p>
<p>I would compare Jev, existing rules, and a constrained-output LLM on the same held-out incidents, measuring routing accuracy, missed urgent cases, review volume, p95 and p99 latency, and cost per incident. Break results down by service and incident type: averages can hide weak spots.</p>
<p>Explanations, remediation code, and deeper investigation remain separate tasks. The decision layer can choose when to invoke them.</p>
<p><strong>I would adopt it if it cut latency and cost without increasing missed urgent incidents or manual review. If those errors rose, the speedup would not be worth it.</strong></p>
<h2 id="cheat-sheet">Cheat sheet</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">System One model   = a model built for fast decisions, not conversation
</span></span><span class="line"><span class="cl">Jev                = TypeSafe&#39;s first System One model, in early access
</span></span><span class="line"><span class="cl">State              = the information to inspect, such as a ticket and an account record
</span></span><span class="line"><span class="cl">Typed question     = one judgment to make about the state; each is evaluated independently
</span></span><span class="line"><span class="cl">Choice             = pick one of the predefined options; returns probabilities and confidence
</span></span><span class="line"><span class="cl">Score              = rate on an ordered scale; returns a distribution over the levels
</span></span><span class="line"><span class="cl">Noul               = a yes/no question answered as a probability from 0 to 1
</span></span><span class="line"><span class="cl">Application policy = your code combines the answers and decides what happens next
</span></span><span class="line"><span class="cl">Question design    = wording and option descriptions change results, so version and test them
</span></span><span class="line"><span class="cl">What to verify     = routing accuracy, missed urgent cases, latency, and cost on held-out cases
</span></span></code></pre></div><p><strong>Related posts:</strong> <a href="https://vishctl.dev/posts/ai-agent-basics/">What the heck is an AI Agent?</a> explains the tool loop a decision layer like this would sit inside, and <a href="https://vishctl.dev/posts/sglang-vs-vllm/">SGLang vs. vLLM</a> covers constrained JSON output, the LLM-side way to get structured answers.</p>
]]></content:encoded></item></channel></rss>