For years in data science, if you wanted deterministic reliability for classification tasks, you built a custom BERT model. I've always appreciated BERT because it gave us deterministic, repeatable classifications; it didn't hallucinate outside its label space, and with enough training data it could get remarkably accurate. But getting there meant dealing with the heavy MLOps lift of custom training pipelines, expensive data labelling, and continuous fine-tuning.
Then came Generative AI. Suddenly, we had LLMs capable of complex reasoning out of the box with zero training. But they predict text sequentially, and that comes with a massive operational catch when building automated pipelines.
To be clear: the problem with generative models in 2026 isn't JSON parsing. Modern frontier models have largely solved syntax errors with "structured output" modes. The real, persisting problems in enterprise automation are semantic variance and hallucinated confidence.
When you ask a generative model to classify a customer ticket, it is still generating text under the hood. You might want it to select the canonical category Billing Query, but depending on the prompt or the temperature, it might generate Billing Question, Payment Query or Invoice Issue. This lack of categorical determinism breaks downstream systems that expect exact string matches.
Yes, there are prompt engineering workarounds to force reliable classifications, but they hit a hard ceiling:
More dangerously, we have completely lost the ability to use confidence scores for automated routing. Generative LLMs are notoriously overconfident; if you prompt an LLM to output its probability of being correct, it will happily output 99% on a completely hallucinated category. If a model can classify something correctly 95% of the time but cannot mathematically tell you which 5% it is unsure about, you cannot safely automate around it.
Jev from TypeSafe AI feels like the exact middle ground we've been waiting for. Jev is a specialised "System One" decision model. It abandons string generation entirely, but unlike BERT, you don't have to custom-train it. The model's launch drew significant press coverage at launch for tackling this overconfidence problem head-on [3].
Because the output space is bounded, Jev sidesteps the token-by-token loop. It works seamlessly out of the box by feeding unstructured data alongside a strict schema of API primitives. For example, Jev exposes three core query types:
Jev's hardware-aware parallel sampler evaluates all those predefined options simultaneously in a single forward pass. This maps the context and returns the exact canonical string you requested; eliminating schema/formatting errors by design.
The real shift is how it's trained. Using Reinforcement Learning for Calibrated Decisions, Jev doesn't optimize for what sounds good to a human like standard RLHF models; it optimises for epistemic honesty.
When Jev outputs a result, it looks like this:
{
"choice": "Billing Query",
"confidence": 0.92,
"score": 85
}That confidence score is mathematically calibrated. If Jev says it is 92% confident, it is actually correct 92% of the time. This allows engineering teams to finally build robust routing thresholds: act autonomously on anything above 90% confidence and route the uncertain edge cases to a human.
From my perspective, it is always easier to understand benefits when we look at real business cases, and when you dig into it, the impact of this architecture becomes undeniable when we consider the future potential application against two of the biggest bottlenecks in enterprise automation: high complexity decisioning and high-volume routing.
UK insurers face a growing volume of First Notification of Loss (FNOL) documents, medical reports, and policy submissions, all of which require fast, accurate triage. The goal for AI here is clear: read these submissions to instantly assess, score, and flag risk before it bottlenecks the pipeline.
If underwriting is the most complex AI workload, telephony is undeniably the most expensive and latency-sensitive channel to automate. To put the scale into perspective: recent analysis from the House of Commons Public Accounts Committee found that HMRC received roughly 37 million inbound calls in a single year [1].
For real-time branching logic, whether you are pricing a policy or routing a live voice call, we potentially have an architecture here that is designed to be right, rather than just sound right. As agentic AI becomes more embedded in enterprise workflows, the systems that scale won't be the ones that sound the most impressive — they'll be the ones making these decisions cheaper and faster, at the volume automation actually demands.
These two examples are just the sharpest illustrations of the pattern, not the limit of it. The same architecture applies anywhere a business makes the same kind of bounded decision at volume and needs to trust the confidence score attached to it, retail returns and fraud triage, healthcare intake prioritisation, HR ticket routing, logistics exception handling, legal document classification. None of these are use cases we've built out here; they're simply other places the same underlying problem, semantic variance and hallucinated confidence, shows up once you start looking for it
The fastest way to get a feel for calibrated decisioning is to point Jev at a live call transcript and watch it triage in real time. Below, a banking contact centre's Choice primitive is configured with 51 canonical categories — well within Jev's 255-option ceiling — covering everything from card disputes to trust and estate planning. As each turn comes in, Jev classifies it, scores urgency, and applies the same 90%-confidence routing rule this post has used throughout: clear it automatically, or send it to a human.
The two transcripts below show both outcomes — a routine login issue that resolves entirely through automation, and a genuinely ambiguous fraud dispute that correctly drops below the threshold and escalates to a person.
If you want to move beyond basic classification and analyze live, multi-turn conversations, you can point Jev at a full transcript line-by-line.
By separating conversational filler and holds ("Continuation / Acknowledgment") from substantive banking topics—and using a lookback attribution window so hold times carry forward into the active intent—you can precisely quantify how much time a customer spends on each part of a call. Applying a strict 90% confidence threshold ensures that standard, clear-cut queries are auto-routed, while deeply complex or ambiguous cases (like multi-jurisdictional fraud disputes) automatically trigger a human handover.
Here is the complete implementation script demonstrating realistic telemetry across complex banking transcripts:
import os
import httpx
from datetime import datetime, timedelta
from dotenv import load_dotenv
# Load environment variables
load_dotenv()
API_KEY = os.getenv("TYPESAFE_API_KEY")
API_URL = "https://api.typesafe.ai/v1/systemone"
BANKING_CRITERIA = {
"Continuation / Acknowledgment": "General greetings, pleasantries, polite acknowledgments, conversational filler, brief confirmations, system processing pauses, or customer hold times that do not introduce a new substantive banking topic.",
"Checking & Savings Account Opening": "Applications and onboarding for new personal deposit accounts",
"Account Closure & Liquidation": "Closing existing accounts and transferring remaining balances",
"Overdue Balance & Overdraft Fees": "Queries regarding negative balances, overdraft charges, and grace periods",
"Debit Card Activation & PIN Reset": "Activating new plastic, changing PINs, or unlocking cards",
"Lost, Stolen, or Damaged Cards": "Reporting compromised cards and ordering emergency replacements",
"ATM Cash Withdrawal Disputes": "Investigating failed dispenser cash drops or incorrect dispense amounts",
"Domestic ACH & Faster Payments": "Checking status of local clearing house transfers and direct debits",
"International SWIFT & Wire Transfers": "Initiating, tracking, or investigating global cross-border telegraphic transfers",
"Foreign Exchange & Currency Conversion": "Spot FX trades, multi-currency wallet management, and exchange rates",
"Suspicious Activity & Fraud Alerts": "Reporting unauthorized transactions, phishing scams, and card skimming",
"Account Freeze & AML Compliance Holds": "Unlocking accounts placed on hold due to anti-money laundering reviews",
"Personal Loan Applications": "Inquiries and underwriting status for unsecured instalment loans",
"Mortgage Pre-Approval & Underwriting": "Home loan applications, property valuations, and lending criteria",
"Mortgage Refinancing & Rate Locks": "Locking fixed rates or refinancing existing residential mortgages",
"Home Equity Line of Credit (HELOC)": "Drawing or managing credit secured against property equity",
"Credit Card Rewards & Cashback": "Redeeming loyalty points, travel milestones, or statement cashback",
"Credit Limit Increase Requests": "Applying for higher credit ceilings on revolving credit lines",
"Credit Card Balance Transfers": "Moving balances from external lenders onto promotional 0% APR cards",
"Business Checking & Corporate Accounts": "Commercial banking operations, partnership accounts, and entity setup",
"Merchant Services & POS Terminals": "Credit card processing gateways, POS hardware, and daily merchant payouts",
"Commercial Real Estate Lending": "Financing for office buildings, retail spaces, and industrial properties",
"Corporate Treasury & Cash Management": "Liquidity sweeps, multi-entity pooling, and interest-bearing sweeps",
"Letter of Credit & Trade Finance": "Import/export documentary credits, bills of lading, and guarantees",
"Payroll Processing & Direct Deposit": "Corporate bulk salary payments and automated tax filings",
"Business Line of Credit": "Working capital financing and revolving corporate credit facilities",
"SBA & Government-Backed Small Business Loans": "Small business administration loan application support and servicing",
"Wealth Management & Advisory Services": "High-net-worth portfolio planning, asset allocation, and private banking",
"Brokerage & Equity Trading": "Buying/selling stocks, bonds, ETFs, and options via self-directed accounts",
"Mutual Funds & Index Portfolios": "Long-term investment fund management and dividend reinvestment",
"Retirement Accounts (IRA / 401k / SIPP)": "Tax-advantaged pension contributions, rollovers, and early withdrawal penalties",
"Cryptocurrency & Digital Asset Services": "Buying, selling, or storing supported digital assets via banking portals",
"Trust and Estate Planning Services": "Setting up testamentary trusts, executing estates, and fiduciary care",
"Tax Document Issuance (1099 / Statements)": "Downloading year-end tax certificates, interest statements, and capital gains reports",
"Certificate of Deposit (CD) Maturation": "Opening high-yield fixed term deposits or handling maturity rollovers",
"Joint Account Holder Management": "Adding or removing secondary signatories and mandate updates",
"Power of Attorney & Legal Signatures": "Submitting legal mandates for managing incapacitated account holders",
"Safe Deposit Box Rentals": "Accessing vault lockers, key replacements, and annual rental renewals",
"Online Banking Login & 2FA Issues": "Troubleshooting biometric authentication, browser errors, SMS OTP codes, and login credentials",
"Mobile Banking App Crashes": "Reporting UI bugs, check-deposit scanner failures, and crash dumps",
"Statement & Document Generation": "Requesting official bank reference letters, certified statements, and audits",
"Standing Orders & Direct Debit Mandates": "Setting up, modifying, or cancelling recurring automated bill payments",
"Overdraft Protection Setup": "Linking backup savings accounts to prevent checking account overdrafts",
"Invoicing & Billing Software Integration": "Connecting accounting software (Xero/QuickBooks) to corporate feeds",
"Bank Fee Waivers & Goodwill Refunds": "Requesting reversal of monthly maintenance fees or late penalties",
"Hardship & Debt Forbearance Programs": "Applying for temporary loan payment holidays or structured debt relief",
"KYC / Identity Verification Updates": "Submitting updated proof of address, passports, or visa documentation",
"Insolvency & Bankruptcy Notifications": "Informing the bank of personal bankruptcy filings or company liquidation",
"Student & Youth Banking Accounts": "Managing specialized accounts with fee exemptions for students",
"Green & Sustainable Finance Options": "Inquiring about eco-friendly mortgages, green car loans, or carbon-offset accounts"
}
def classify_turn(speaker: str, text: str, client: httpx.Client):
payload = {
"state": f"Speaker ({speaker}): {text}",
"model": "jev-latest",
"questions": {
"intent": {
"type": "choice",
"instructions": "Which banking topic or conversation phase best classifies this specific statement?",
"criteria": BANKING_CRITERIA
},
"is_urgent": {
"type": "noul",
"instructions": "Does this statement convey an urgent financial threat, fraud risk, or critical banking emergency?"
}
}
}
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json"
}
response = client.post(API_URL, json=payload, headers=headers)
if response.status_code != 200:
return "Unknown", 0.0, 0.0
data = response.json().get("answers", {})
choice_res = data.get("intent", {})
noul_res = data.get("is_urgent", {})
return (
choice_res.get("choice", "Unknown"),
choice_res.get("confidence", 0.0),
noul_res.get("noul", 0.0)
)
def analyze_banking_transcript(transcript_id: int, lines: list):
print(f"\n==================================================")
print(f" BANKING TRANSCRIPT {transcript_id} (Lookback Telemetry)")
print(f"==================================================")
substantive_time_seconds = {}
active_topic = "Continuation / Acknowledgment"
THRESHOLD = 0.90
current_time = datetime(2026, 9, 21, 14, 0, 0)
turn_records = []
with httpx.Client() as client:
for turn in lines:
if not turn.strip():
continue
if ":" in turn:
speaker, text = turn.split(":", 1)
speaker = speaker.strip()
text = text.strip()
else:
speaker = "Unknown"
text = turn.strip()
if "hold" in text.lower() or "pause" in text.lower() or "investigat" in text.lower():
duration_sec = 35
else:
word_count = len(text.split())
duration_sec = max(6, int(word_count / 2.0))
start_timestamp = current_time.strftime("%H:%M:%S")
end_time = current_time + timedelta(seconds=duration_sec)
end_timestamp = end_time.strftime("%H:%M:%S")
category, confidence, urgency = classify_turn(speaker, text, client)
if speaker.lower() == "agent":
status = "[AGENT TURN — NO ROUTING]"
else:
status = "AUTO-ROUTED" if confidence >= THRESHOLD else "HUMAN ESCALATE"
turn_records.append({
"start": start_timestamp,
"end": end_timestamp,
"duration": duration_sec,
"speaker": speaker,
"text": text,
"category": category,
"confidence": confidence,
"urgency": urgency,
"status": status
})
current_time = end_time
accumulated_buffer = 0
for record in turn_records:
cat = record["category"]
dur = record["duration"]
if cat != "Continuation / Acknowledgment":
active_topic = cat
total_dur_to_add = dur + accumulated_buffer
substantive_time_seconds[active_topic] = substantive_time_seconds.get(active_topic, 0) + total_dur_to_add
accumulated_buffer = 0
else:
if active_topic != "Continuation / Acknowledgment":
accumulated_buffer += dur
print(f"[{record['start']} - {record['end']}] ({record['duration']}s) [{record['speaker']}]")
print(f" \"{record['text']}\"")
print(f" └─> Topic: {record['category']} (Conf: {record['confidence']:.2f}) | Urgency: {record['urgency']:.2f} | Routing: {record['status']}\n")
print(f"--- Transcript {transcript_id} Cumulative Time-on-Topic (with Lookback Attribution) ---")
for topic, secs in sorted(substantive_time_seconds.items(), key=lambda x: x[1], reverse=True):
print(f" • {topic}: {secs} seconds ({secs / 60:.1f} mins)")
if __name__ == "__main__":
if not API_KEY:
print("Error: TYPESAFE_API_KEY environment variable is not set.")
exit(1)
banking_transcripts = [
# Transcript 1: Clean, unambiguous intent mapping with high confidence
[
"Agent: Thank you for calling Northbridge Banking Group, my name is Marcus. How may I assist your accounts today?",
"Customer: Hi, I'm calling because my online banking login has been locked out and my two-factor authentication SMS codes are not coming through.",
"Agent: I can certainly look into your profile status right now. Please hold for just a moment while I pull up your secure records.",
"Agent: Thank you for holding. Are you currently experiencing these login issues with our web portal or through the mobile application?",
"Customer: It's definitely the web portal; my 2FA authentication SMS code isn't arriving when I try to authorize our international SWIFT wire transfer to Frankfurt.",
"Agent: Let me trigger a secondary authentication reset for your online profile right now.",
"Customer: Thanks. Once that wire clears, I also want to move £250,000 from my commercial savings into our managed equity portfolio."
],
# Transcript 2: Complex Fraud Dispute triggering a valid Human Escalation due to multi-jurisdictional ambiguity
[
"Agent: Welcome to Northbridge Banking Group, how can I help you today?",
"Customer: I need urgent help with a multi-party international dispute involving compromised crypto wallet linkages and cross-border merchant chargebacks that span three different legal jurisdictions.",
"Agent: That sounds extremely complex and sensitive. I am going to place you on an extended security hold while I escalate your case directly to our Senior Fraud and Legal Arbitration Directorate.",
"Agent: Thank you for holding. Our fraud directors have reviewed the preliminary notes, and a specialized human agent is taking over the line to walk you through the evidentiary affidavit.",
"Customer: Thank you, please also make sure nobody touches my mortgage direct debit scheduled for tomorrow morning while this is sorted."
]
]
for idx, t_lines in enumerate(banking_transcripts, start=1):
analyze_banking_transcript(idx, t_lines)Output — Transcript 1 (routine login issue, resolves entirely through automation):
==================================================
BANKING TRANSCRIPT 1 (Lookback Telemetry)
==================================================
[14:00:00 - 14:00:09] (9s) [Agent]
"Thank you for calling Northbridge Banking Group, my name is Marcus. How may I assist your accounts today?"
└─> Topic: Continuation / Acknowledgment (Conf: 1.00) | Urgency: 0.01 | Routing: [AGENT TURN — NO ROUTING]
[14:00:09 - 14:00:20] (11s) [Customer]
"Hi, I'm calling because my online banking login has been locked out and my two-factor authentication SMS codes are not coming through."
└─> Topic: Online Banking Login & 2FA Issues (Conf: 1.00) | Urgency: 0.27 | Routing: AUTO-ROUTED
[14:00:20 - 14:00:55] (35s) [Agent]
"I can certainly look into your profile status right now. Please hold for just a moment while I pull up your secure records."
└─> Topic: Continuation / Acknowledgment (Conf: 1.00) | Urgency: 0.02 | Routing: [AGENT TURN — NO ROUTING]
[14:00:55 - 14:01:30] (35s) [Agent]
"Thank you for holding. Are you currently experiencing these login issues with our web portal or through the mobile application?"
└─> Topic: Online Banking Login & 2FA Issues (Conf: 0.99) | Urgency: 0.01 | Routing: [AGENT TURN — NO ROUTING]
[14:01:30 - 14:01:42] (12s) [Customer]
"It's definitely the web portal; my 2FA authentication SMS code isn't arriving when I try to authorize our international SWIFT wire transfer to Frankfurt."
└─> Topic: Online Banking Login & 2FA Issues (Conf: 0.90) | Urgency: 0.19 | Routing: AUTO-ROUTED
[14:01:42 - 14:01:48] (6s) [Agent]
"Let me trigger a secondary authentication reset for your online profile right now."
└─> Topic: Online Banking Login & 2FA Issues (Conf: 1.00) | Urgency: 0.06 | Routing: [AGENT TURN — NO ROUTING]
[14:01:48 - 14:01:58] (10s) [Customer]
"Thanks. Once that wire clears, I also want to move £250,000 from my commercial savings into our managed equity portfolio."
└─> Topic: Wealth Management & Advisory Services (Conf: 0.95) | Urgency: 0.03 | Routing: AUTO-ROUTED
--- Transcript 1 Cumulative Time-on-Topic (with Lookback Attribution) ---
• Online Banking Login & 2FA Issues: 99 seconds (1.6 mins)
• Wealth Management & Advisory Services: 10 seconds (0.2 mins)
Output — Transcript 2 (ambiguous fraud dispute, correctly escalates to a human):
==================================================
BANKING TRANSCRIPT 2 (Lookback Telemetry)
==================================================
[14:00:00 - 14:00:06] (6s) [Agent]
"Welcome to Northbridge Banking Group, how can I help you today?"
└─> Topic: Continuation / Acknowledgment (Conf: 1.00) | Urgency: 0.01 | Routing: [AGENT TURN — NO ROUTING]
[14:00:06 - 14:00:18] (12s) [Customer]
"I need urgent help with a multi-party international dispute involving compromised crypto wallet linkages and cross-border merchant chargebacks that span three different legal jurisdictions."
└─> Topic: Suspicious Activity & Fraud Alerts (Conf: 0.67) | Urgency: 0.93 | Routing: HUMAN ESCALATE
[14:00:18 - 14:00:53] (35s) [Agent]
"That sounds extremely complex and sensitive. I am going to place you on an extended security hold while I escalate your case directly to our Senior Fraud and Legal Arbitration Directorate."
└─> Topic: Suspicious Activity & Fraud Alerts (Conf: 0.77) | Urgency: 0.72 | Routing: [AGENT TURN — NO ROUTING]
[14:00:53 - 14:01:28] (35s) [Agent]
"Thank you for holding. Our fraud directors have reviewed the preliminary notes, and a specialised human agent is taking over the line to walk you through the evidentiary affidavit."
└─> Topic: Suspicious Activity & Fraud Alerts (Conf: 0.96) | Urgency: 0.33 | Routing: [AGENT TURN — NO ROUTING]
[14:01:28 - 14:01:38] (10s) [Customer]
"Thank you, please also make sure nobody touches my mortgage direct debit scheduled for tomorrow morning while this is sorted."
└─> Topic: Standing Orders & Direct Debit Mandates (Conf: 1.00) | Urgency: 0.24 | Routing: AUTO-ROUTED
--- Transcript 2 Cumulative Time-on-Topic (with Lookback Attribution) ---
• Suspicious Activity & Fraud Alerts: 82 seconds (1.4 mins)
• Standing Orders & Direct Debit Mandates: 10 seconds (0.2 mins)
If you're already building with Claude, Codex (or any other supported CLI), TypeSafe's plugin is designed to drop into your existing setup with ease.
From agentic workflows to parallel sampling, our team brings deep generative AI expertise to every engagement - helping you stop wrestling with semantic variance and hallucinated confidence, and start building more powerful, efficient, and cost-effective automation from the moment we sit down.
Get in touch to explore what parallel sampling could do for your data pipelines with a technical discovery call.
[1] House of Commons Committee of Public Accounts (2025) HMRC Customer Service and Accounts (Twenty-Ninth Report of Session 2024–25). UK Parliament, 22 January 2025. Available at: https://committees.parliament.uk/publications/46301/documents/233200/default/ (Accessed: 21 September 2026).
[2] ContactBabel (2024) UK Contact Centre Decision-Makers' Guide. ContactBabel, March 2024. Available at: https://www.contactbabel.com/the-uk-contact-centre-decision-makers-guide/ (Accessed: 21 September 2026).
[3] Shrivastava, R. (2026) 'This $200 Million Startup Wants To Fix AI's Overconfidence Problem', Forbes, 15 September 2026; Fernholz, T. (2026) 'A new kind of AI model from a ChatGPT inventor is thrilling developers', TechCrunch, 18 September 2026. Available at: https://www.forbes.com/sites/the-prompt/2026/09/15/this-200-million-startup-wants-to-fix-ais-overconfidence-problem/ & https://techcrunch.com/2026/09/18/a-new-kind-of-ai-model-from-a-chatgpt-inventor-is-thrilling-developers/ (Accessed: 21 September 2026).
[4] TypeSafe AI (2026) API reference. TypeSafe AI. Available at: https://docs.typesafe.ai/api (Accessed: 21 September 2026).