supportdesk/triage.py:34
detect_sentiment
Classify the customer's overall sentiment in a support ticket as positive, neutral, or negative.
Output contract
Exactly one lowercase word from the set {positive, neutral, negative}; caller strips trailing period and lowercases; max_tokens is 3.
keeps 100% of baseline quality and costs less
Dollar figures are projections: measured token counts x illustrative per-model prices x an assumed call volume, all set in downshift.yaml. They are not a real bill.
Models
How each model did
| Model | Pass rate | Tokens in / out | Latency | Cost / call | vs baseline | Check |
|---|---|---|---|---|---|---|
qwen2.5:7bbaseline premium tier | 90.9% (20/22) | 86 / 2 | 0.39 s | $0.000236 | n/a | n/a |
qwen2.5:3bchosen standard tier | 90.9% (20/22) | 86 / 2 | 0.17 s | $0.000038 | 100.0% | pass |
qwen2.5:1.5b budget tier | 63.6% (14/22) | 86 / 2 | 0.11 s | $0.000014 | 70.0% | fail |
qwen2.5:0.5b nano tier | 81.8% (18/22) | 86 / 2 | 0.04 s | $0.000009 | 90.0% | fail |
- 3b keeps 100% of baseline quality and costs less
- 1.5b keeps 70% of baseline quality, needs 95%
- 0.5b keeps 90% of baseline quality, needs 95%
Prompt
What the call sends
Model in code: qwen2.5:7b
Eval examples
3 of 22 cases, every model side by side
›sen-017b3b1.5b0.5b
Inputs
- ticket_text
- Thank you! Just wanted to say the support team was amazing last week. My issue was fixed in minutes. Keep it up!
Expected
genuine praise, clearly positive (T009)
Outputs
match
match
match
match
›sen-027b3b1.5b0.5b
Inputs
- ticket_text
- Arrived early! Order ORD-11533 arrived two days early, great packaging. One question: can I get the tracking history for my records?
Expected
happy with delivery, minor informational request (T032)
Outputs
match
match
match
match
›sen-037b3b1.5b0.5b
Inputs
- ticket_text
- Update email Hi! Could you help me change the email address on my account? Thanks so much :)
Expected
polite, friendly tone with smiley (T024)
Outputs
match
expected 'positive', got 'neutral'
match
match
All cases