Small models, big jobs, and the tasks that need neither

There's a reflex in software right now: a feature needs to be smart, so it calls an API, so it needs a network, an account, a key, and a cost that grows with every user. Sometimes that's the right architecture. Often the task at hand doesn't need general intelligence at all — it needs a narrow answer, fast, with the data it's already looking at.

What a frontier model is actually for

Open-ended reasoning. Conversation that could go anywhere. Reading a messy document nobody anticipated. Writing code. These are jobs where the space of valid inputs is enormous and the right response can't be enumerated in advance. A large general model is worth its latency and its cost there, because nothing else does the job.

The mistake is assuming everything downstream of "AI" inherits those requirements.

The jobs a small local model does better

Counting reps from a camera feed. Recognizing text on a page. Estimating where a person's joints are. These are narrow, well-defined perception problems, and a small model running on the phone beats a big one over the network on every axis that matters to the person holding it.

Latency: feedback about a push-up has to arrive during the push-up. Cost: a local model costs nothing per inference, so a feature can run continuously instead of being rationed. Offline: a basement gym, a plane, a national park with no bars. Privacy: the frames are analyzed and discarded, so there's no copy anywhere to leak.

Form Coach is our clearest example. The phone sits on the floor, watches you, draws a skeleton over your body, counts reps and scores each one on form — on iPhone and Android, offline, with no account. A cloud version of that feature would be worse in every way except the ease of building it.

Sometimes the honest answer is no model at all

The underrated case is the one where machine learning is the wrong tool entirely, and a well-designed rules engine is both better and more explainable.

Dialin logs an espresso shot — dose, grind, yield, time — plus how it tasted, and then recommends exactly one change for the next pull. Under the hood that's a deterministic rules engine reading shot time, taste verdict and brew ratio. It might tell you "sour and ran in 22s, grind one step finer," with a plain-language reason attached. No model, no network, and no black box: the logic is the same cause-and-effect reasoning a barista uses, written down.

If you can write the rule down and explain it to the user in one sentence, you probably shouldn't be renting a model by the token to guess it.

That transparency is a product feature, not an engineering shortcut. A recommendation you can argue with is a recommendation you can learn from.

Deciding, feature by feature

The question we ask is not "cloud or on-device." It's what this specific task needs: general reasoning, narrow perception, or plain deterministic logic. Answer that honestly and the architecture picks itself, along with the privacy posture and the cost curve.

Small models keep getting more capable, which means the set of jobs that need a network keeps shrinking. That's the direction we're building toward. See how it plays out in Dialin.