Curating a Genie Agent that earns trust

Pull-quote: “You get 100 instructions. Spend them on rules no schema could express, and nothing else.”

A Genie Agent is curated by an analyst, and the four things that analyst supplies decide whether anyone trusts the answers. This is the least glamorous work in the Databricks Genie family and the work that determines everything else, because a question arriving in Genie One is answered by an agent somebody scoped, instructed, and gave examples to.

The name changed in July 2026. What was called a Genie space is now a Genie Agent, with the same capabilities and the same meaning: a domain-scoped natural-language interface over datasets registered in Unity Catalog. Genie Agents are generally available.

This post is for the analyst who owns one, or is about to be handed one. It assumes you write SQL, know your own data model, and have a Unity Catalog metastore. By the end you will know what to supply and in what order, how to diagnose a wrong answer down to a stage rather than blaming the model, how to spend a hard instruction limit, and how to tell when the agent is ready to share.

What you will be able to do

• Name the four inputs a Genie Agent reads, and what each one is responsible for.
• Diagnose a wrong answer to one of four stages, and change the input that actually governs that stage.
• Spend the 100-instruction budget on rules rather than on descriptions that belong in metadata.
• Explain why structured data only is a design constraint that makes answers verifiable.
• Promote a real Genie One conversation into a curated, shareable agent instead of guessing at questions.
• Decide, with evidence, when an agent is ready to be embedded in Microsoft Teams or Slack.

What is a Genie Agent, and who is supposed to curate it?

A Genie Agent is a domain-specific natural-language analytics interface, curated by data analysts over datasets registered in Unity Catalog, and it is generally available. It was called a Genie space until Databricks renamed it in July 2026, so older documentation and your colleagues’ habits will both use the old term for a while.

The curator should be the analyst who knows the model, not the platform team and not the business sponsor. Every one of the four inputs is a judgment about meaning: which dataset is authoritative, what a phrase in a question refers to, how a measure is calculated, which comparison a user implies when they say “versus last year”. A platform engineer can attach a warehouse and grant access. Only someone who has argued about the numbers can say which table is the right one.

One thing curation is not is a permission grant. Access is enforced by Unity Catalog, so adding a dataset to an agent makes it available to the agent’s SQL generation, not to a user who lacks SELECT on it. Two people asking the same question can correctly receive different results, and that is the mechanism working rather than failing.

What does an analyst actually supply to a Genie Agent?

Four inputs: datasets registered in Unity Catalog, example SQL, SQL expressions that encode business semantics, and text instructions written in your organisation’s own terminology. They are not interchangeable, and most disappointing agents have three of them thin and one of them overloaded.

Input	What it is responsible for	The failure it prevents
Registered datasets	The scope boundary the agent can query	Answers drawn from a table nobody meant to expose
Example SQL	Question and query pairs that fix the shape of an answer	Invented join paths and silent double counting
SQL expressions	Business semantics expressed as calculation, not prose	Three different figures for the same measure
Text instructions	Rules and vocabulary no schema can carry	Fiscal calendars, status codes, and defaults being guessed

Supply them in that order, because each one is cheaper once the previous is settled. Registering a narrow, well-named set of datasets first means fewer instructions later. Writing example SQL before instructions often removes the need for the instruction, since the query demonstrates the rule better than a sentence describing it.

Why does it matter that a Genie Agent is a compound AI system?

Because the agent runs four distinct stages rather than one text-to-SQL call, a wrong answer has four possible causes and each is fixed by a different input. Databricks describes it as a compound AI system: classify the question, select the dataset, identify metrics and dimensions, then generate and evaluate SQL.

That structure is the most useful debugging tool you have. Ask a failing question, read the generated SQL, and locate the stage that broke.

• The answer addresses a different question than the one asked, which is a classification problem, usually vocabulary. Fix the instructions that carry your terminology.
• The SQL is correct but runs against the wrong table, which is dataset selection. Narrow the scope or remove the near-duplicate table.
• The right table is used with the wrong measure or the wrong grain, which is metric identification. Encode the measure as a SQL expression rather than describing it.
• The SQL invents a join path, or sums across grains, which is generation. Add example SQL for that shape of question.

Notice that “the model got it wrong” is not on the list. Prompt rewording moves an answer, but it does not move the input that produced the error, so the same question in a different phrasing brings the problem back a week later.

How should you spend the 100 instruction ceiling?

A Genie Agent supports up to 100 instructions, and the general instructions block counts as one of them, so treat instructions as a budget rather than a text box. The limit is a useful constraint: it forces the question of whether a piece of guidance belongs in prose at all.

Spend an instruction when the rule cannot be expressed anywhere else. A fiscal calendar that starts in February. Which of two tables is authoritative for revenue. The status codes that mean active. The default time window when a question names none. A word your business uses that appears nowhere in the schema.

Do not spend an instruction on anything metadata or SQL can carry better. Column meanings belong in column comments. A contested calculation belongs in a SQL expression or a governed measure, where dashboards inherit the same definition. A join path belongs in declared keys plus one example query. A synonym you could fix by renaming a column in a view belongs in the view.

Here is the checkpoint that tells you the budget is being misused: the instruction count keeps rising while benchmark accuracy stays flat. That pattern means prose is standing in for structure, and the fix is upstream in the model you are exposing rather than in the sentence you are about to add.

Why is structured data only a design constraint rather than a gap?

Because it is what makes every answer checkable: the documentation is explicit that a Genie Agent cannot answer questions about PDFs, Word documents, or other file content. Every answer reduces to SQL over governed tables, which means you can read the query, re-run it, and reconcile the number against a report.

Consider what a blended answer would cost you. If an agent could combine a figure from your Gold layer with a sentence from a policy document, the answer would have two provenances and only one of them would be re-runnable. When somebody challenges the number six weeks later, “here is the query” becomes “here is the query, and also a paragraph a model selected from a document that has since been edited”. The boundary is what keeps an analytics answer auditable.

Document questions are a real need, and they belong in a retrieval system with its own evaluation, its own citations, and its own failure modes. Summit messaging describes agents grounded in documents as a direction of travel, so do not plan a rollout on it, and do not tell your users that it works today. Where a product needs both, keep the paths separate and label which is which, a pattern covered in building applications and agents on Genie.

How do you promote a real conversation into a curated agent?

Build the agent from questions people actually asked, using the supported path: an agent can be created from a Genie One conversation, inheriting that conversation’s memory, sources, and instructions, and can then be shared and embedded in Microsoft Teams or Slack. This is a better starting point than a workshop, because a workshop produces the questions people think they ask.

1. Watch a real conversation. Let a small group use Genie One against a broad, unpolished scope for a week, and collect what they typed, including the phrasings that failed.
2. Create the agent from the conversation. It arrives carrying the memory, sources, and instructions already established, so you start from context rather than from an empty configuration.
3. Curate down, not up. Remove datasets nobody needed. Rename what was ambiguous. Add example SQL for the ten questions that recurred.
4. Encode the contested measures. Every figure two people argued about becomes a SQL expression or a governed measure, not a sentence in the instruction block.
5. Benchmark, then share. Only once the benchmark set passes do you embed the agent in Teams or Slack, where the audience is wider than the people who can check it.

How do you know the agent deserves trust?

By having a fixed set of questions with known-correct answers that you re-run after every change, and by reading the SQL rather than the number. Trust is a measured property here, not a judgment call after a good demo.

Four practices carry most of the weight. Keep a benchmark question set that includes the common questions, at least one question with a known trap such as an invitation to double count, one that requires a governed measure, and one the agent should decline. Read the generated SQL for every benchmark question, because a correct number produced by a wrong query will diverge as soon as the data shifts. Verify permissions by asking as a restricted user rather than by reading the grants. And treat the agent’s configuration as reviewable: the datasets, the examples, the expressions, and the instructions all change behaviour, so they deserve the same review a query change would get. What a Genie Agent needs to give answers you can defend sets out the benchmark mechanics, and the agent evaluation harness covers the same discipline for agent systems generally.

Where this goes wrong

One agent for the whole company. The symptom is the agent choosing the wrong dataset for a routine question. The cause is scope: several datasets could answer it and nothing declares which is authoritative. Split by subject area, and accept that breadth comes from having several focused agents.

Instructions used as a patch kit. The symptom is an instruction block that grows every week while accuracy stays flat, eventually approaching the 100 limit. The cause is prose compensating for a model that was never fixed. Move meanings into comments, calculations into expressions, and join paths into examples.

Example SQL written by the curator’s imagination. The symptom is good coverage of questions nobody asks and no coverage of the question everyone asks. The cause is inventing the question set instead of harvesting it. Take the examples from a real conversation, in the users’ phrasing, including the failures.

Curation confused with entitlement. The symptom is a governance review concluding that an agent exposed sensitive data. The cause is treating the dataset list as a security boundary. Unity Catalog grants, row filters, and column masks decide what any user sees, and the check is to ask a question as a restricted user.

Shared before a benchmark existed. The symptom is a wrong figure reaching a wide audience in week two, which ends the project regardless of what you fix afterwards. The cause is embedding into Teams or Slack because it was easy, before anything measured accuracy.

Common questions

What is the difference between a Genie Agent and a Genie space?

They are the same thing under two names. Databricks renamed Genie Spaces to Genie Agents in July 2026 without changing the capabilities, so documentation, blog posts, and internal wikis written before that date describe an identical object. Use the current name in anything you write now.

How many datasets should one Genie Agent contain?

Few enough that you could explain every join path out loud without checking. One subject area per agent is the working rule, because dataset selection is one of the four stages where an answer can go wrong, and every near-duplicate dataset you add makes that stage harder.

Can a Genie Agent answer questions about documents or PDFs?

No. The documentation is explicit that a Genie Agent cannot answer questions about PDFs, Word documents, or other file content, because it works over structured data registered in Unity Catalog. Document question answering needs a retrieval system with its own citations and evaluation, and it should be presented to users as a separate capability.

What counts as an instruction against the 100 limit?

Each instruction counts as one, and the general instructions block counts as a single instruction however many rules you pack into it. So you have 99 individual slots plus one general block, which is ample room if meanings live in column comments and calculations live in SQL expressions rather than in prose.

Who should own a Genie Agent day to day?

The analyst who knows the data model, with a named backup. Ownership includes the benchmark set, the example queries, and the decision about when the agent is ready to widen. Platform teams own the warehouse and the grants, business sponsors own the questions, and neither can settle which dataset is authoritative.

Next steps

Take the ten questions your users asked last week, in their own words, and answer each one in SQL by hand. That exercise produces your example query set, your first benchmark set, and a short list of modelling problems, usually two or three, that would otherwise have surfaced in front of an audience. Then create the agent from a real conversation rather than from a blank configuration.

Read Genie Ontology, the context layer under every Genie answer next, because it explains where the meaning you are hand-curating today is heading, and why authored definitions still matter when a ranking algorithm is assembling context. If your agent is meant to serve a business team rather than analysts, Genie One for business teams covers the scheduled tasks and alerts that will be built on top of your curation work.

Zorost curates these agents on client lakehouses as a trusted Databricks partner, where an agent does not reach a Teams channel until its benchmark set passes and someone who knows the model has read the generated SQL.