기본 콘텐츠로 건너뛰기

When a Small Decider Beats a Frontier LLM: Strands Decider

In short: Two Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, a small open-source "decision model", for labeling tickets into 20 categories, and report the same quality at about 98% lower cost and a 78% shorter call. The trick is not that a small model is smarter; it is that ticket labeling is a closed choice from a known list, so a model that only picks options (and says how sure it is) is enough. This playbook explains when that swap makes sense for your own workload, how to test it without risk, and where the math stops working.

What a decision model is, in one paragraph

Strands Decider, published by the Strands Agents team under the Apache-2.0 license, is not a chatbot. According to its docs, it "answers typed questions about a state instead of generating text." You give it some text (the state, for example a ticket) and one or more questions, each of a fixed type:

  • noul: a yes-or-no question, returned as the probability that the answer is true.
  • choice: pick one option from a set, returned as the selected option, every option's probability and a confidence.
  • score: rate something on an ordered scale, returned as an expected score, the distribution and a confidence.

Under the hood, the team takes a pre-trained LLM torso, removes the part that writes text, and adds a small "pointer head" (just over a million parameters, per the launch post) that scores each option. The upside is speed, a single-token-sized output and a calibrated confidence number. The downside, which the launch post states plainly, is that it is much worse than reasoning models at complex problems and cannot do coding, chat or summarization at all.

What Amazon actually changed

The case study on the Strands blog (November 9, 2026) describes an internal ticketing system where Sonnet 4.6 ran a long system prompt on every ticket and produced JSON with reasoning, choosing among 20 labels. The tickets ranged from customer escalations and capacity requests to automated alarms and security findings.

They first tried the original 2B Decider, then switched to the Gemma 4 based E4B build because it did clearly better on their test tickets, and tuned how confident the model had to be before a label was applied. The reported numbers:

  • Average time per call: 4,215 ms for Sonnet 4.6 vs 897 ms for Decider E4B (78% less).
  • Tokens per ticket: 6,724 vs 2,010 (70% less). Output fell from 170 tokens to 1, because the model returns a choice, not JSON.
  • Cost per million tickets: about $22,200 vs about $500, at published us-east-1 on-demand prices.

The serving setup was ordinary AWS: tickets land in SQS, a Lambda function reads them and calls a SageMaker real-time endpoint running Decider E4B on ml.g5.xlarge (one A10G GPU, 24 GB). The model used about 19 GiB and a single request took about 640 ms. You can see the measured and cost tables from the case study in the official screenshots at the end of this post.

Two details most summaries skip

  • The prompt had to shrink. The E4B model is trained and served at a 4,096-token window, and Amazon's existing label descriptions alone were longer than that. They rewrote each of the 20 labels as a one- or two-sentence criterion, added a "none" option, and on long tickets dropped the oldest comments. Part of the token saving comes from that rewrite, not from the model.
  • JSON key order changed the answer. Sending the same content with keys in a different order moved the output probabilities by up to 0.19. They now sort keys on every request. If you compare runs, do the same.

The cost math, redone for smaller teams

Amazon's $500 per million tickets assumes enough traffic to keep GPU instances busy. Most teams do not have that. I reran the numbers using only the figures in the case study and Anthropic's published price list, to show where the swap starts to pay off.

  • Sonnet 4.6 per ticket: 6,554 input tokens at $3 per million plus 170 output tokens at $15 per million comes to $0.0222, which matches the case study.
  • Decider at full use: at about 640 ms per request, one instance handles roughly 5,600 tickets an hour. $1.408 per hour divided by 5,600 is about $0.00025 per ticket, or $250 per million. Amazon keeps headroom for bursts, which is why its real figure is about $500.
  • One instance left on all month: 730 hours at $1.408 is $1,027.84. Divide that by Sonnet's $0.0222 and the break-even point is roughly 46,300 tickets a month. Below that, a single always-on GPU endpoint costs more than just calling Sonnet.

So the headline "98% cheaper" is a high-volume result. At a few thousand decisions a month, the cheaper move may be a smaller LLM, prompt caching for the repeated system prompt, or running the 2B Decider on a machine you already own. My own cost table is the last image at the end of this post.

Options compared

Option Price basis Cost per 1M tickets* Speed (as published) Good fit
Claude Sonnet 4.6 (what Amazon replaced) $3 input / $15 output per 1M tokens ~$22,200 4,215 ms average call (case study) Labels that need an explanation, catalogs that change often, long context
Claude Haiku 4.5 (cheaper LLM, my estimate) $1 input / $5 output per 1M tokens ~$7,400 at the same token counts Not measured in the case study Teams that want a cheaper LLM without new infrastructure
Strands Decider E4B on SageMaker ml.g5.xlarge $1.408 per instance-hour ~$250 at full use, ~$500 with headroom 897 ms average call; ~640 ms per request on the instance High-volume closed-choice decisions, latency-sensitive paths
Strands Decider 2B on your own GPU or Mac Open weights, no per-call fee (your hardware and power) Depends on your hardware ~115 ms median on an RTX 3090; ~153 ms on an M3 MacBook for small tasks (launch post) Experiments, internal tools, gating agent tool calls

*Sonnet and Haiku rows use the case study's 6,554 input and 170 output tokens per ticket. The Haiku row is my arithmetic, not a measured run; a real Haiku setup could need a different prompt.

Is your workload a decider job? A five-question check

The case study's own "lessons" section gives the conditions. I turned them into a checklist you can run against a real task:

  1. Is the output a fixed list? Labels, queues, teams, yes/no, or a 1-5 score. If you need free text back, stop here.
  2. Does the list stay stable? Amazon warns that a catalog that changes weekly is a bad fit.
  3. Does the useful context fit in a few thousand tokens? The Gemma 4 builds are trained at 4,096 tokens. If you need whole documents, a decider probably cannot replace the LLM.
  4. Is the volume high enough? Use the break-even math above with your own token counts and instance price.
  5. Does latency matter? If a person or another system waits on the decision, a sub-second answer is worth more than if the job runs overnight.

Three or more "yes" answers make a pilot worthwhile. Good candidates the Strands team lists include routing a request to a model, tool, team or queue; checking tool arguments, grounding or policy; and scoring model responses during evaluation.

How to test the pattern on your own data

Step 1: ask one question locally

The docs show the quickest start: install the package from PyPI and point it at the published 2B model.

pip install strands-decider

strands-decider ask StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 \
  --state "Help! My payouts have been failing for 3 days." \
  --choice "Which team should handle this?=billing,technical,sales"

Pass --device cuda, mps or cpu explicitly; otherwise the CLI picks CUDA, then MPS, then CPU. The Gemma 4 models (E2B, E4B, 12B, 26B-A4B) need the code from the GitHub main branch, because the PyPI package does not include Gemma 4 support yet: pip install "strands-decider @ git+https://github.com/strands-labs/strands-decider".

Step 2: rewrite your labels as short criteria

Copy Amazon's move: one or two sentences per label, plus an explicit "none" option. The HTTP API accepts a description for each option under criteria, so the label name stays short and the criterion carries the meaning.

Step 3: serve it and send a few hundred real items

strands-decider serve <model> --port 8000 exposes POST /v1/systemone plus /health. Note the docs' warning: the server binds to 127.0.0.1, has no authentication, and its behavior under concurrent requests is not verified, so treat it as a local experiment until you put it behind your own gateway. For hosting, the repo's deploy/ folder includes a SageMaker guide, and the server also answers on /ping and /invocations, the paths SageMaker calls.

Step 4: run both models side by side

This is the part Amazon says it would copy regardless of model. Their Lambda read a three-way switch from AWS AppConfig on every message: BEDROCK (LLM only), DUAL (both run, the LLM's answer is applied, the two are compared) and DECIDER (decider only). Running DUAL on real traffic and judging every disagreement told them more than offline evals, and rolling back was a 45-second config change. You can build the same switch with any feature-flag tool; an offline version is simply logging both answers to a table. If you already use a prompt test suite, see my Promptfoo tutorial for a quick way to score the disagreements.

Step 5: set confidence bands before you switch

The architecture docs suggest a starting convention: at 0.9 confidence or above, act without a check; from 0.5 to 0.9, confirm first; below 0.5, send it to a person. They also say the right thresholds depend on what each action costs when wrong, and should be measured on your own traffic. In the docs' own recorded example, "failing payouts" was routed to technical at 0.622 confidence, which lands in the confirm band, exactly the kind of borderline case a human or an LLM fallback should see.

Editor's note

Who it's for: teams already paying for a frontier LLM to classify, route or gate things at high volume, with a stable label set; agent builders who want a cheap check before a tool call runs.

Who it's not for: anyone who needs explanations, summaries or generated replies; low-volume workflows where one always-on GPU costs more than the LLM calls; label sets that change every week.

Gotchas I found in the docs:

  • Gemma 4 model ids not working after pip install strands-decider: the PyPI release lacks Gemma 4 support; install from the GitHub main branch.
  • HTTP 422 on long inputs: a prompt longer than the checkpoint's window is refused, with a message naming the window. Trim old comments or shorten label criteria.
  • Out-of-memory on a busy GPU: without --max-concurrency, up to 40 requests can run at once, and a burst of large ones can exhaust GPU memory. Set a small value.
  • Health checks failing at startup: the server warms up before accepting requests and /health fails until it finishes, so allow for it in startup timeouts.
  • Unstable results between runs: sort JSON keys in structured states; Amazon saw probabilities move by up to 0.19 from key order alone.

Where this fits with the rest of your stack

A decider does not replace your LLM layer; it sits in front of it. A common shape is decider first, LLM only when confidence is low. If you route between several providers, a proxy like the one in my LiteLLM tutorial keeps the LLM fallback simple, and tracing with Langfuse shows which calls were expensive enough to be worth moving. If you plan to host the model yourself, GPU hourly prices vary a lot by provider; my Modal vs Runpod vs Lambda comparison covers that side.

FAQ

Is Strands Decider free to use?

The code is released under the Apache-2.0 license and the weights are published on Hugging Face (the StrandsAgents organization, with the five v1 models also mirrored under the amazon organization). There is no per-call fee from the project; you pay for whatever hardware or cloud instance runs it.

Which Decider size should I start with?

The README's guidance: the 12B and 26B-A4B are the most accurate, the 26B-A4B is better calibrated but needs about twice the memory and latency of the 12B, the E4B suits a GPU shared with other work, and the E2B fits tight memory or CPU. For a first local test, the 2B Qwen3.5 model is the documented quickstart. Amazon picked E4B after it beat the 2B on their own tickets.

Can a decider handle my chatbot or support replies?

No. It only picks options, answers yes/no or returns a score. It can decide which team or template a reply should use, but an LLM (or a person) still has to write the text.

If your LLM bill is mostly spent picking from a list, a one-week DUAL run on real traffic will tell you more than any benchmark.

Last verified: 2026-11-13. Images: screenshots of the Strands Agents blog case study, the Strands Decider docs and the strands-decider GitHub README, plus one table made by lightmind24 (see the official screenshots at the end of this post).
lightmind24 is an independent review site, not affiliated with Amazon Web Services, Strands Agents or Anthropic.
Sources: Strands blog: Amazon replaces Sonnet 4.6 with Strands Decider · Introducing Strands Decider · Decider overview · Decider inference docs · Decider architecture · strands-decider on GitHub · Claude API pricing

Uploaded Image Uploaded Image Uploaded Image Uploaded Image Uploaded Image

댓글

이 블로그의 인기 게시물

브루(Vrew) 요금제 정리: 무료 200크레딧으로 어디까지 쓸 수 있나

요약: 영상 자막 편집기 브루(Vrew)는 2026년 4월 22일부터 기능별 사용량 대신 '통합 크레딧' 하나로 요금을 매겨요. 무료 플랜은 매달 200크레딧이 들어오는데, 음성 분석(자동 자막)만 쓰면 최대 200분 분량이라 짧은 영상 몇 편은 충분하지만, 영상 초반의 Vrew 마크를 지우려면 유료 플랜이 필요해요. 이 글은 브루 공식 가격 페이지와 고객센터 공지를 2026년 10월 23일에 다시 확인해, 크레딧 계산법과 플랜 고르는 기준을 정리했어요. 목차 브루 요금 구조: 통합 크레딧이란 1크레딧으로 할 수 있는 일 무료 플랜으로 되는 것과 안 되는 것 유료 플랜 4종 비교표 월 구독·연 구독·기간 이용권, 무엇이 이득일까 상황별 추천: 학생·직장인·유튜버 크레딧이 모자랄 때: 사용 기록 지우개 업그레이드·환불 전에 알아둘 규칙 운영자 메모, 자주 묻는 질문 1. 브루 요금 구조: 통합 크레딧이란 예전 브루는 플랜마다 음성 분석 몇 분, AI 목소리 몇 자, 번역 몇 자처럼 기능별 한도를 따로 줬어요. 그런데 보이저엑스(브루 운영사)가 실제 이용 데이터를 보니 한두 가지 기능만 집중해서 쓰는 사람이 훨씬 많았다고 해요. 그래서 2026년 4월 22일 업데이트부터는 플랜마다 정해진 크레딧을 받고, 어떤 기능에 쓸지는 내가 정하는 방식으로 바뀌었어요. 플랜별 매달 지급량은 Free 200 · Light 2,000 · Standard 10,000 · Business 12,000크레딧 이에요. 공식 공지에 따르면 무료 사용자는 매달 1일에, 유료 사용자는 매월 결제일에 충전되고, 남은 크레딧은 다음 달로 넘어가지 않아요. 바뀐 뒤 무엇이 좋아졌나 공지에 나온 예시가 이해하기 쉬워요. Standard 플랜에서 AI 목소리만 쓰면 예전에는 최대 50만 자였지만, 지금은 1만 크레딧을 모두 AI 목소리에 몰아 쓸 수 있어 최대 100만 자가 돼요. 반대로 여러 기능을 골고루 쓰는 사람은 크레딧이 기능끼리 ...

에이닷 전화 vs 익시오: 통화 녹음·AI 요약, 내 통신사·폰에 맞는 앱 고르기

요약: 통화 녹음과 AI 요약을 쓰고 싶다면 먼저 내 통신사와 휴대폰 종류부터 확인해야 해요. LG유플러스 고객은 무료 앱 익시오, SKT 고객은 에이닷 전화가 가장 기능이 많고, 다른 통신사의 아이폰 사용자는 iOS에 들어 있는 기본 통화 녹음·전사 기능이 현실적인 선택이에요. 이 글은 2026-10-30에 SKT·LG유플러스·Apple 공식 페이지와 도움말을 직접 확인해 세 가지를 비교했어요. 왜 '앱 기능'보다 '내 조건'을 먼저 봐야 할까요 통화 요약 앱 비교 글을 보면 기능 목록부터 나열하는 경우가 많아요. 그런데 실제로 써 보려는 사람에게 가장 먼저 걸리는 건 기능이 아니라 이용 조건 이에요. 익시오는 LG유플러스 고객 전용이고, 에이닷 전화는 누구나 설치할 수 있지만 통역콜·baro 같은 기능은 SKT 고객에게만 열려 있어요. 아이폰은 운영체제 구조상 앱이 통화를 마음대로 녹음하기 어려워서, 통신사 앱이 안 되면 iOS 기본 기능에 기대야 해요. 그래서 이 글은 "어떤 앱이 제일 좋은가"보다 "내 통신사·내 폰에서 무엇을 쓸 수 있나" 순서로 정리했어요. 기능 차이는 그다음이에요. 한눈에 비교: 에이닷 전화 vs 익시오 vs 아이폰 기본 통화 녹음 구분 에이닷 전화 (SKT) 익시오 ixi-O (LG유플러스) 아이폰 기본 통화 녹음 (Apple) 누가 쓸 수 있나 통신사 구분 없이 앱 설치 가능. 단, 통역콜·baro·그룹통화 발신 등은 SKT 고객 전용 U+ 모바일 고객. 타 통신사, 만 14세 미만, 선불요금제 고객은 제한 (공식 사이트 FAQ상 알뜰폰 중 KB리브엠은 예외) 지원 국가·언어의 iPhone 사용자. 1명과 통화할 때만 녹음 가능 지원 OS·기기 Android 10 이상, iOS 16 이상 iOS 17 이상 iPhone 12 이상 / Android...

타입캐스트 vs 클로바더빙: 수익 나는 채널에 쓸 때 무료·유료 조건 비교

요약 타입캐스트와 클로바더빙은 둘 다 글만 입력하면 한국어 AI 목소리를 만들어 주는 서비스지만, 무료로 쓸 수 있는 범위가 꽤 다릅니다. 클로바더빙 무료는 매달 15,000자까지 쓸 수 있는 대신 수익이 생기는 채널에는 쓸 수 없고, 타입캐스트 무료는 처음에 주는 약 5분 분량의 다운로드로 끝나는 체험판에 가깝습니다. 광고 수익이 붙은 유튜브나 쇼핑몰 영상에 넣을 목소리라면 처음부터 유료 플랜 조건을 보고 고르는 편이 안전합니다. 왜 이 두 서비스를 같이 비교하나 쇼츠 내레이션, 수업 자료 더빙, 상품 소개 영상처럼 "내 목소리 대신 AI 목소리"가 필요할 때 한국 사용자들이 가장 많이 찾는 곳이 네오사피엔스의 타입캐스트 와 네이버의 클로바더빙 입니다. 기능 소개만 보면 비슷해 보이지만, 실제로 문제가 되는 건 음질보다 "이 목소리를 내 채널에 써도 되나" 입니다. 그래서 이 글은 기능 나열보다 무료 사용 범위, 출처 표기 의무, 유료 플랜 가격과 다운로드 한도를 중심으로 정리했습니다. 모든 수치는 2026년 10월 25일에 두 서비스의 공식 요금 페이지와 고객센터 도움말에서 직접 다시 확인한 내용입니다. 한눈에 보는 비교표 항목 타입캐스트 무료 타입캐스트 베이직 클로바더빙 Free 클로바더빙 Standard 월 요금 0원 9,900원 (연간 결제 시 10% 할인) 0원 19,900원 (네이버 클라우드 플랫폼에서 결제) 무료·포함 사용량 음성 생성·재생 무제한, 다운로드는 최초 3,000크레딧(약 5분) 1회 지급 월 30,000크레딧(약 60분) 월 15,000자, 다운로드 월 20회, 프로젝트 누적 5개 (매월 1일 갱신) 월 30,000자, 다운로드 월 40회, 프로젝트 누적 20개 쓸 수 있...