Why do AI features surprise people on cost?

Because a model call costs real money per use, an AI feature is usually the first part of a product with a marginal cost. Refractr bills in credits and OsceSetGo enforces per-plan limits — both designed alongside the feature, not after the first surprising invoice.

Should the feature be a chat box?

Usually not. Users want the job done, not a conversation — constraining the model to structured output in the shapes your product already uses makes results predictable, testable and cheap to render. Save open-ended chat for the rare product where conversation is the point.

Which model should we build on?

Whichever wins for the task this quarter. We run OpenAI and Gemini across our products and pick per job — and keep the seam between product and model thin, because prices and quality move every few months and you want to be able to swap.

Adding AI to an existing product without burning money

Every product roadmap has an AI feature on it this year — usually with a deadline, rarely with a plan for what it costs to run. The demo takes a weekend. Production is where it gets interesting.

We've shipped AI features in our own products — strategy briefs in Refractr AI, voice and text practice in OsceSetGo — and the lessons repeat. Here they are.

Meter it from day one

Model calls cost real money per use, which makes an AI feature the first part of your product with a marginal cost. Refractr bills in credits; OsceSetGo enforces usage limits per plan. Both were designed alongside the feature, not bolted on after the first surprising invoice. Decide how usage maps to revenue before you ship — or your most enthusiastic users become your most expensive.

Structured beats chatty

Users don't want a chat box; they want the job done. Refractr doesn't hand marketers a conversation — it generates a strategy brief in the shape the product already uses. Constraining the model to structured output makes results predictable, testable and cheap to render into your existing screens. Save open-ended chat for the rare product where conversation is the point.

Model choice is a product decision

We run OpenAI and Gemini across our products and pick per task — different jobs justify different cost and quality trade-offs. Build the thinnest possible seam between your product and the model, so you can swap when prices or quality move. They do, every few months.

Ship one seam, then widen it

The pattern that works is boring: one well-instrumented feature, real usage data, then expand. The pattern that fails is a platform-wide “AI layer” announced before anyone has checked what a month of tokens costs. If you're planning the first kind, we can usually tell you in one call what it will take.

The short answers

Why do AI features surprise people on cost?: Because a model call costs real money per use, an AI feature is usually the first part of a product with a marginal cost. Refractr bills in credits and OsceSetGo enforces per-plan limits — both designed alongside the feature, not after the first surprising invoice.
Should the feature be a chat box?: Usually not. Users want the job done, not a conversation — constraining the model to structured output in the shapes your product already uses makes results predictable, testable and cheap to render. Save open-ended chat for the rare product where conversation is the point.
Which model should we build on?: Whichever wins for the task this quarter. We run OpenAI and Gemini across our products and pick per job — and keep the seam between product and model thin, because prices and quality move every few months and you want to be able to swap.

Related: Refractr AI case study