Writing an Answer or Choosing One

Some AI jobs need new writing. Others need a choice from a short list. Telling them apart shows you what to check, and whether a language model belongs in the job at all.

In this authored illustration, a fictional bike shop receives this message:

I was charged twice for the tune-up on my daughter's bike. Can you fix that? And is the bike ready yet?

Answering it takes several jobs.

Generation writes or explains something: a reply to the customer, or a plain summary of the refund policy. How does an AI answer get made? follows this job.

Retrieval finds relevant material: the refund policy, the order record, the repair ticket. A reply without them has nothing to stand on.

Ranking puts candidates in order. If a search returns ten policy passages, ranking decides which to read first.

Bounded selection chooses from allowed outcomes. Does this message go to billing, to repairs, or to a person for review? A poem about tune-ups is not a fourth option.

Whether the card was charged twice, and how much to return, are exact questions. Code can compare the two charges and subtract, with the same answer every time.

Issuing the refund is an action. Actions belong to a workflow with permissions, under rules about who approves each step. When a model decides some steps along the way, people often call the system an agent. An agent is a system built around models and tools, not a separate kind of model.

Likewise, an assistant application is more than the model inside it. The shop's assistant might include the inbox, the policy search, the payment connection, and the approval rules. The model is one part.

A general language model can also choose. Ask, "Is this message about billing, repairs, or review? Answer with one word," and it classifies. Ask it to order ten passages, and it ranks. The answer still arrives as generated text, so software must check that it is one of the allowed words.

Other models are built for choosing. A classifier learns fixed labels from labeled examples. A decision model takes the material and the allowed answers and returns a choice. Some are built on a language model underneath. The name describes the job, not the construction.

Jev, from TypeSafe, is one current decision model. You supply the material to judge, which TypeSafe calls state: here, the message and the order record. You also supply the allowed answers: billing, repairs, review. According to TypeSafe's description of its question types, Jev returns one of those answers with a probability for each, never a value outside the list.

TypeSafe calls this class of model "System One," a name borrowed from a book about fast and slow human thinking. That is the vendor's label, not evidence that the model thinks like a person.

An answer in the right shape can still be wrong. TypeSafe's confidence documentation describes its confidence number as a summary of how concentrated the probabilities are, and advises testing thresholds on your own data. Confident is not the same as correct; only checked examples show the second.

Channel3, which builds a large catalog of products and sorts each into a category, wrote about trying Jev. Its first approach walked down the category tree one level at a time, keeping a few of the best-scoring paths. For a youth bicycle, the right branch dropped out early, and the product ended up as a motor vehicle.

Channel3's existing pipeline already retrieved 30 likely categories, and the company reports the right one was among them more than 99% of the time. With Jev making the final choice from those 30, it reported better accuracy than the open model it had used for that step, at about the same cost. These are its own figures on its own catalog, given without sample details.

The lesson travels better than the numbers. If an early step throws away the right answer, no later choice can recover it. Check what each stage keeps, as well as what the last one picks.

Sort these tasks into generation, retrieval, ranking, bounded selection, exact rule, or action:

1. Find last month's invoice for this customer.
2. Decide whether a new message goes to billing, repairs, or review.
3. Work out the refund for a $60 tune-up charged twice.
4. Write an apology that explains the next step.
5. Send the refund to the customer's card.
6. Put five past messages in order of how much they resemble this one.

Answer: 1 is retrieval. 2 is bounded selection. 3 is an exact rule: return one $60 charge, using code or a calculator. 4 is generation. 5 is an action for a workflow allowed to issue refunds once a person or rule approves. 6 is ranking.

Putting 2 under generation is a reasonable guess, since a language model could answer in one word. The fixed list changes it: with known outcomes, you can measure how often the choice is right.

Read finding before answering for the retrieval step, or checking an AI answer to test an answer against its sources.

For the technical version, should a model make this decision? compares code, classifiers, and decision models against an error limit. The Jev workshop lets you route requests and compare methods on the same cases.