supportdesk/extract.py:21

extract_order_info

found by astmediumjson_fieldsjsonmax_tokens 60temperature 0

Extract order ID and order date from a support ticket body.

Output contract

JSON object with exactly two keys: `order_id` (string matching ORD-\d+ or null) and `order_date` (string in YYYY-MM-DD format or null); no additional keys; values must be null when not explicitly stated. Graded fields: order_id, order_date.

downgradeqwen2.5:7b → qwen2.5:3b

keeps 105% of baseline quality and costs less

$335.00

Before / month

$52.96

After / month

$282.04

Savings (84.2%)

20,000

Calls / day

Dollar figures are projections: measured token counts x illustrative per-model prices x an assumed call volume, all set in downshift.yaml. They are not a real bill.

Models

How each model did

Model	Pass rate	Tokens in / out	Latency	Cost / call	vs baseline	Check
qwen2.5:7bbaseline premium tier	95.2% (20/21)	105 / 30	2.15 s	$0.000558	n/a	n/a
qwen2.5:3bchosen standard tier	100.0% (21/21)	105 / 29	0.92 s	$0.000088	105.0%	pass
qwen2.5:1.5b budget tier	81.0% (17/21)	105 / 30	0.71 s	$0.000034	85.0%	fail
qwen2.5:0.5b nano tier	71.4% (15/21)	105 / 28	0.33 s	$0.000022	75.0%	fail

3b keeps 105% of baseline quality and costs less
1.5b keeps 85% of baseline quality, needs 95%
0.5b keeps 75% of baseline quality, needs 95%

Prompt

What the call sends

Model in code: qwen2.5:7b

system

Extract the order ID and the order date from the support ticket. Order IDs look like ORD-12345. Return JSON with exactly two keys: order_id and order_date. order_date must be YYYY-MM-DD. Use null for anything not stated explicitly.

user

{ticket_text}

Eval examples

3 of 21 cases, every model side by side

›ext-017b3b1.5b0.5b

Inputs

ticket_text: Refund request for ORD-10421 Hi, I ordered a pair of wireless headphones on 2026-09-02 (order ORD-10421). They're unused and still in the box, but I changed my mind. Can I get a refund?

Expected