> ## Content Index
> Fetch the complete content index at: https://kudithipudi.org/llms.txt
> Use this file to discover other available public pages before exploring further.

# HOW TO : Build a validated dataset with a decision model
- URL: https://kudithipudi.org/how-to-build-a-validated-dataset-with-a-decision-model/
- Published: 2026-10-21T04:05:51.000Z
- Updated: 2026-10-21T04:05:51.000Z
- Description: I used Jev, a decision model, to sort 43,202 Texas food permits by cuisine in six minutes for about $1.30. Fast, cheap, no made-up answers, but a confident answer still isn’t a correct one.
- Author: Vinay Kudithipudi
- Tags: Technology, AI

I wanted a definitive list of Indian restaurants in Dallas-Fort Worth. Not a "top 10" list, but every one of them, with a source behind each entry. The obvious move is to ask a chatbot or scrape a maps API. Neither gives you data you can defend row by row.

### Start from records you can cite

Every restaurant in Texas needs a sales tax permit, and the Texas Comptroller publishes all of them as [open data](https://data.texas.gov/d/3kx8-uryv?ref=kudithipudi.org). Filtering to the 11 DFW counties and the food-service business codes gives 43,202 outlets, each with a name, an address, and open and close dates.

What the permits don't say is the cuisine. "SAI BHAVAN LLC" is obviously Indian to a person. "KK BUSINESS GROUP LLC" is not obviously anything. That one question, "what does this place serve?", is the only part I handed to AI.

### Ask a decision model, not a chat model

A chat model returns text. I would have had to prompt it for JSON, parse it, and hope the confidence it wrote down meant something. For a classification over a fixed set of answers, I used [Jev](https://openrouter.ai/typesafe/jev-1.13?ref=kudithipudi.org), a decision model from TypeSafe, through OpenRouter. You give it the facts and a list of allowed answers, and it returns a probability for each one.

```python
# app/services/llm.py (trimmed)
body = {
    "model": "typesafe/jev-1.13",
    "state": {"outlet_name": "BABAI BANDI", "registered_llc": "SLOKAM FOODS LLC",
              "city": "DENTON", "business_type": "mobile food service (food truck)"},
    "questions": {"cuisine": {
        "type": "choice",
        "instructions": "Decide what cuisine this food business primarily serves...",
        "criteria": {"indian": "biryani, dosa, chaat; bhavan, vilas, bandi (food cart)...",
                     "pakistani": "...", "nepali": "...", "not_south_asian": "...", "unclear": "..."}}},
}
r = await client.post("https://openrouter.ai/api/alpha/decisions", json=body)
r.json()["answers"]["cuisine"]["probabilities"]   # {"indian": 0.98, "unclear": 0.02, ...}

```

There's no parsing step and no free text, and the probabilities are what the rest of the pipeline is built on.

### What it costs and how fast it is

Each call cost about $0.00003\. A full pass over all 43,202 outlets cost $1.20 to $1.33 and took about six minutes with 16 requests in flight, roughly 130 decisions a second. Later refreshes only re-classify outlets whose details changed, so they cost cents.

For comparison, here's the same pass on general chat models, estimated from OpenRouter list prices at roughly 400 tokens in and 70 out per business:

| Model                        | Per business | All 43,202 |
| ---------------------------- | ------------ | ---------- |
| Jev (measured)               | \~$0.00003   | $1.20–1.33 |
| Claude Haiku 5.5 (estimate)  | \~$0.000075  | \~$3       |
| Claude Sonnet 5.5 (estimate) | \~$0.0015    | \~$65      |

Price is only part of it. A chat model has to write its answer out token by token, and then I'd have to parse it. Jev returns probabilities directly, which is why a whole metro area fits in a coffee break. It also can't hallucinate in the usual sense: it can only choose from the answers I gave it, so it never invents a cuisine, a fact or a malformed reply.

That doesn't make its answers right. Picking from a list is not the same as being correct. My first version of the question scored "Pak-A-Pocket", a pita sandwich shop, 91% South Asian, so it went straight onto the list. The current version still puts "Roti Restaurants", a Mediterranean chain, at 84%, one point under the bar for listing. A confident answer to a narrow question is still a guess from a name.

### What made it trustworthy

- **Probabilities decide, people confirm.** At 85% or more South Asian an outlet is listed; between 30% and 85% it waits for a human review with an evidence link; below 30% it's out. Every answer is stored with its exact input and question version, and a public [methodology page](https://lab.kudithipudi.org/dfw-indian-restaurants/methodology?ref=kudithipudi.org) shows the whole process.
- **Cheap calls make testing cheap.** I compared question changes side by side on 3,658 borderline outlets for about $0.09 before adopting one. A long keyword list made Jev match words ("Swig N Sweets" looked South Asian), while moving the same examples into each answer's description fixed the bhavan, vilas and bandi cases.

The live site is at [lab.kudithipudi.org/dfw-indian-restaurants](https://lab.kudithipudi.org/dfw-indian-restaurants/?ref=kudithipudi.org).