Groundtruth · verification report

The same prompt, generated twice — then checked against the real warehouse

Every table and column in both models was graded against the live DataHub metadata graph: 67 datasets, 816 columns, captured 2026-08-10 13:32 UTC.

the prompt, given to bothWrite a dbt model that reports on orders that were returned or refunded. For each such order, include the order id, order total, and the date it was returned/refunded. Save it as dbt_project/models/marts/orders_returns.sql.

The ungrounded run invented 5 things that do not exist. The grounded run invented 0.

Ungrounded scored 0% grounded; the grounded run scored 100% across 7 checked references.

The two models, annotated

verified against DataHub does not exist read from a table that does not exist

Ungrounded — no DataHub access

HALLUCINATED 0%
references verified
0 / 3
invented tables
3
unresolvable columns
8
DataHub queries made
0

Provenance: no DataHub queries were made. Nothing here was looked up.

{{
    config(
        materialized = 'table'
    )
}}
 
-- Orders that were returned and/or refunded, one row per order.
-- An order may have multiple return lines and/or multiple (partial) refunds;
-- we report the earliest return/refund event date for the order.
 
with orders as (
 
    select
        order_id,
        order_total
    from {{ ref('stg_orders') }}
 
),
 
returns as (
 
    select
        order_id,
        returned_at
    from {{ ref('stg_order_returns') }}
    where returned_at is not null
 
),
 
refunds as (
 
    select
        order_id,
        refunded_at
    from {{ ref('stg_refunds') }}
    where refunded_at is not null
 
),
 
return_refund_events as (
 
    select
        order_id,
        'returned' as event_type,
        returned_at as event_at
    from returns
 
    union all
 
    select
        order_id,
        'refunded' as event_type,
        refunded_at as event_at
    from refunds
 
),
 
events_by_order as (
 
    select
        order_id,
        min(event_at) as returned_or_refunded_at,
        max(case when event_type = 'returned' then 1 else 0 end) = 1 as is_returned,
        max(case when event_type = 'refunded' then 1 else 0 end) = 1 as is_refunded
    from return_refund_events
    group by order_id
 
),
 
final as (
 
    select
        orders.order_id,
        orders.order_total,
        events_by_order.returned_or_refunded_at,
        events_by_order.is_returned,
        events_by_order.is_refunded
    from orders
    inner join events_by_order
        on orders.order_id = events_by_order.order_id
 
)
 
select * from final
 

Grounded — queried DataHub first

GROUNDED 100%
references verified
7 / 7
invented tables
0
unresolvable columns
0
DataHub queries made
9

Provenance: 9 live DataHub queries returned 122.3 KB of real metadata before a line of SQL was written.

-- orders_returns
--
-- WHAT WAS ASKED FOR VS. WHAT THE WAREHOUSE HAS
--
-- The request was for orders that were "returned or refunded". Verified against
-- DataHub, this warehouse tracks RETURNS ONLY. There is no refund concept in the
-- catalog: no refunds / credit-memo / chargeback dataset exists, and no
-- refund_amount, refund_date or refund_status field appears on any dataset. The
-- catalog's own "Returns & Refunds" data product (doc: E-Commerce Data Catalog)
-- resolves to exactly two assets -- order_items (return_date) and order_details --
-- confirming refunds are not modelled separately upstream. This model therefore
-- reports returns. The refunded half of the request cannot be answered from the
-- warehouse as it stands; it would need a new source feed from the payments system.
--
-- The requested "date it was returned/refunded" is also not a single value per
-- order. Returns are recorded per LINE ITEM (order_items.return_date), so an order
-- whose items came back on different days has several return dates. This model is
-- one row per order and exposes first/last return date plus the line-item counts
-- needed to distinguish a fully returned order from a partial one.
--
-- SOURCE (fields below all confirmed present in DataHub)
--   order_entry_db.analytics.order_details -- Snowflake table, also a dbt model in
--   the order_entry_db project. Grain: one row per order line item.
--     order_id      NUMBER
--     order_total   FLOAT    order-header value, repeated on every line of the order
--     line_item_id  NUMBER
--     line_total    FLOAT    unit_price * quantity
--     return_date   VARCHAR  date item was returned; NULL = not returned
--     return_status VARCHAR  'Returned' / 'Not Returned', derived from return_date
--
--   order_details is the documented default for analytics. order_details_replica is
--   an exact copy for query load balancing and is not used here. The Postgres
--   order_entry tables are the operational source and are not queried directly.
--
-- NOTES ON THE COLUMNS BELOW
--   order_total is GMV at point of sale. Per the Orders Table doc it is captured at
--   order creation and does NOT deduct returns, so it is the value of the order as
--   placed, not the amount left after the return. returned_item_value (sum of
--   line_total over returned lines) is the closest confirmed proxy for the value
--   sent back, since no refund amount is recorded anywhere.
--
--   return_date is a VARCHAR in both order_items and order_details, so it is cast
--   with try_to_date. Rows with an unparseable value still count as returned, but
--   surface as a NULL date rather than failing the build.
--
--   order_status is deliberately not filtered on. Its schema type (NUMBER, with
--   "1=Pending, 2=Processing, 3=Shipped") disagrees with the Orders Table doc,
--   which describes string statuses ('Cancelled', 'On Hold'). The encoding could
--   not be confirmed, so no cancellation filter is applied. Add one here once the
--   status domain is settled if cancelled orders should be excluded.
 
with order_lines as (
 
    select
        order_id,
        line_item_id,
        order_total,
        line_total,
        return_date,
        try_to_date(return_date) as returned_on
 
    from {{ source('analytics', 'order_details') }}
 
),
 
returns_by_order as (
 
    -- Aggregate every line of the order, not just the returned ones, so that
    -- total_line_items is a true denominator for the return.
    select
        order_id,
 
        -- order_total is constant across the order's lines; max() just picks it.
        max(order_total) as order_total,
 
        min(case when return_date is not null then returned_on end) as first_return_date,
        max(case when return_date is not null then returned_on end) as last_return_date,
 
        count(distinct case when return_date is not null then line_item_id end) as returned_line_items,
        count(distinct line_item_id) as total_line_items,
 
        sum(case when return_date is not null then line_total else 0 end) as returned_item_value
 
    from order_lines
    group by order_id
 
)
 
select
    order_id,
    order_total,
    first_return_date,
    last_return_date,
    returned_line_items,
    total_line_items,
    returned_item_value
 
from returns_by_order
where returned_line_items > 0
order by first_return_date desc, order_id
 

Every reference, checked

Ungrounded

linereferenceverdictwhat DataHub says
14 order_id orphaned exists elsewhere, but not on any table read here
15 order_total orphaned exists elsewhere, but not on any table read here
16 ref('stg_orders') hallucinated no such dataset in DataHub did you mean orders / order_history?
23 order_id orphaned exists elsewhere, but not on any table read here
24 returned_at orphaned invented — on 0 of 67 datasets did you mean return_date / return_status / updated_at?
25 ref('stg_order_returns') hallucinated no such dataset in DataHub did you mean order_details?
26 returned_at orphaned invented — on 0 of 67 datasets did you mean return_date / return_status / updated_at?
33 order_id orphaned exists elsewhere, but not on any table read here
34 refunded_at orphaned invented — on 0 of 67 datasets did you mean updated_at?
35 ref('stg_refunds') hallucinated no such dataset in DataHub
36 refunded_at orphaned invented — on 0 of 67 datasets did you mean updated_at?

Grounded

linereferenceverdictwhat DataHub says
55 order_id verified on analytics.order_details (dbt, snowflake)
56 line_item_id verified on analytics.order_details (dbt, snowflake)
57 order_total verified on analytics.order_details (dbt, snowflake)
58 line_total verified on analytics.order_details (dbt, snowflake)
59 return_date verified on analytics.order_details (dbt, snowflake)
60 return_date verified on analytics.order_details (dbt, snowflake)
62 source('analytics', 'order_details') verified analytics.order_details (dbt, snowflake)

What the grounded run actually asked DataHub

  1. ToolSearch select:mcp__datahub__search,mcp__datahub__list_schema_fields,mcp__datahub__get_entities,mcp__datahub__get_lineage
  2. Bash command="ls -la \"C:/Users/moksh/AppData/Local/T 150 B
  3. Bash command="find \"C:/Users/moksh/AppData/Local/Tem 88 B
  4. search /q order* 7.7 KB
  5. Read dbt_project.yml 288 B
  6. search /q return OR returns OR refund OR refunds OR rma OR chargeback 5.6 KB
  7. get_entities 3 entities 30.7 KB
  8. search /q status OR state OR cancel* 6.1 KB
  9. list_schema_fields snowflake:analytics.order_details 5.0 KB
  10. get_entities dbt:analytics.order_details 41.2 KB
  11. get_entities postgres:order_entry.order_items 4.1 KB
  12. ToolSearch select:mcp__datahub__grep_documents,mcp__datahub__search_documents
  13. search_documents /q refund OR return OR returns 3.2 KB
  14. grep_documents 5 entities 18.7 KB
  15. Write orders_returns.sql 200 B
  16. Write _sources.yml 194 B