validation

Overview

This skill is MANDATORY after creating or modifying any Honeydew object.

Validation ensures:

• The object compiles and executes without errors
• Results are returned (not NULL or empty)
• Values make business sense
• Related objects are consistent with each other

---

Validation by Object Type

Metrics

Step 1: Execute the metric

Call get_data_from_fields with:

• metrics: ["<entity>.<metric_name>"]

If the tool call fails (API error, permission denied, timeout), report the error to the user before proceeding. Do not confuse a tool error with suspicious data.

Step 2: Sanity checks

| Check | What to Look For | Action if Failed |

| --------------- | ------------------------------------- | ----------------------------------------------------- |

| Returns data | Not NULL, not empty | Check SQL syntax, entity references |

| Magnitude | Reasonable for business context | Verify calculation logic |

| Sign | Positive for revenue/counts (usually) | Check for inverted logic |

| Related metrics | Parts sum to whole | Query related metrics together and verify consistency |

| Ratios | Between 0-100% (usually) | Check numerator and denominator metrics independently |

Step 3: Cross-validation

If the new metric is a filtered subset of an existing metric, or a ratio of existing metrics, query them together to verify consistency:

• For filtered metrics: query the filtered metric alongside the unfiltered total — the filtered value should be less than or equal to the total.
• For ratio/derived metrics: query the numerator and denominator independently to confirm they return sensible values before checking the ratio.

Call get_data_from_fields with both metrics:

• metrics: ["<entity>.<filtered_metric>", "<entity>.<total_metric>"]

Alert user if:

• Metric returns $0 or NULL unexpectedly
• Revenue/count is negative
• Ratio exceeds 100% or is negative (unless expected)
• Magnitude seems off by orders of magnitude

---

Attributes

Step 1: Execute the attribute

If the attribute references a related entity (multi-entity attribute), first verify the relation exists using get_entity on the source entity.

If the relation is missing, report that before attempting to query the attribute.

Sample rows — call get_data_from_fields with:

• attributes: ["<entity>.<attribute_name>"]

For boolean attributes, check distribution — call get_data_from_fields with:

• attributes: ["<entity>.<boolean_attribute>"]
• metrics: ["<entity>.count"]

Step 2: Sanity checks by attribute type

| Attribute Type | Valid Range | Red Flags |

| ------------------- | ------------------------ | ----------------------------------- |

| Age/Duration | 0 to reasonable max | Negative values, >150 years |

| Boolean | TRUE/FALSE mix | All TRUE or all FALSE |

| Percentage | 0-100 (usually) | Negative, >100 (unless growth rate) |

| Date | Past to near future | Year 1900, year 2099, all NULLs |

| Category/Bucket | Expected labels | 95%+ in "Unknown" or NULL |

| Rank | Starts at 1, consecutive | Starts at 0, gaps, all same value |

| Running total | Monotonically increasing | Decreases, resets unexpectedly |

| Any type | Mix of values | >50% NULLs warrants investigation |

Alert user if:

• All values are NULL or >50% NULLs
• Boolean is 100% one value
• Dates are invalid or in wrong century
• Buckets are mostly "Unknown"

---

Entities

Step 1: Verify entity was created

Use list_entities and filter results for the new entity name.

Step 2: Verify data flows

Call get_data_from_fields with:

• metrics: ["<entity>.count"]

Also call with a list of attributes to verify they are accessible:

• attributes: ["<entity>.<attribute1>", "<entity>.<attribute2>", "<entity>.<attribute3>"]

Step 3: Sanity checks

| Check | What to Look For | Action if Failed |

| --------------------- | -------------------------------- | ---------------------------- |

| Entity exists | Shows in list_entities results | Check create_entity call |

| Has rows | Count > 0 | Verify source table path |

| Key is unique | Count = count distinct of key | Fix key or add composite key |

| Attributes accessible | Can query attributes | Check dataset definition |

Alert user if:

• Entity not found after creation
• Zero rows returned
• Key column has duplicates (for non-fact tables)

Validating Relations (part of entity validation)

Relations are not standalone objects — they are defined within an entity's YAML. Validate them as part of the entity that contains them.

Step 1: Verify relation exists

Use get_entity on the source entity and check its relations list for the new relation.

Step 2: Test the join works

Call get_data_from_fields with a cross-entity query:

• attributes: ["<target_entity>.<attribute>"]
• metrics: ["<source_entity>.<metric>"]

Step 3: Sanity checks

| Check | What to Look For | Action if Failed |

| ---------------- | ----------------------------- | ----------------------------------------------- |

| Relation exists | Shows in get_entity results | Check update_object call |

| Join works | Cross-entity query succeeds | Verify join keys match |

| No fan-out | Counts don't explode | Check cardinality (many-to-one vs many-to-many) |

| NULLs reasonable | Some NULLs OK for left join | Too many NULLs = bad join key |

Alert user if:

• Relation not found on the entity
• Cross-entity query fails
• Row count explodes (indicates wrong cardinality)
• All joined values are NULL (bad join condition)

---

Domains

Step 1: Verify domain exists

Use search_model (with search_mode: EXACT) to find the new domain by name.

Step 2: Test with a scoped query

Call get_data_from_fields with the domain parameter to verify entities are accessible and filters apply:

• metrics: ["<entity>.count"]
• domain: "<domain_name>"

Step 3: Verify filters apply

If the domain has semantic or source filters, compare results with and without the domain to confirm filters reduce the data as expected:

• Query a metric with the domain set — note the result.
• Query the same metric without the domain — note the result.
• The domain-scoped result should be less than or equal to the unscoped result (for filters that restrict rows).

Step 4: Sanity checks

| Check | What to Look For | Action if Failed |

| -------------------- | ------------------------------------------- | --------------------------------------------- |

| Domain exists | Found via search_model | Check create_object call and YAML syntax |

| Entities accessible | Scoped query returns data | Verify entity names match existing entities |

| Filters apply | Scoped count <= unscoped count | Check filter SQL and entity.field references |

| Field selectors work | Excluded fields not returned in query | Verify selector patterns and order |

| No errors | Query executes without compilation errors | Check filter SQL syntax, fully qualified refs |

Alert user if:

• Domain not found after creation
• Scoped query returns an error (likely bad filter SQL or missing entity)
• Filters have no effect (scoped count equals unscoped count when a filter is expected to reduce rows)
• Excluded fields are still accessible (field selector not applied correctly)

---

Validating Updates (update_object)

When modifying an existing object, compare before and after:

1. Before altering, query the object and note the current result.
2. After altering, query again and compare.
3. Report the difference to the user: "The metric revenue changed from $1,234,567 to $1,198,432 after applying the discount logic."

This confirms the change had the intended effect and helps catch unintended regressions.

---

Error Handling

If a tool call fails, distinguish between:

| Situation | Meaning | Action |

| ------------------------------ | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |

| Tool returns an error | API error, permission denied, timeout, invalid YAML | Report the error message to the user. Do not treat as a validation failure — the object may not have been created. |

| Tool returns empty/NULL data | Object was created but returns no results | Proceed with sanity checks — likely a data or SQL issue. |

| Tool returns unexpected values | Object works but results look wrong | Report findings and ask user before attempting a fix. |

---

Alerting Guidelines

When results seem wrong, report to the user with:

1. The actual value returned by the query
2. Why it looks suspicious (e.g., negative revenue, 100% NULL, row count doubled)
3. A suggested next step (e.g., "Should I check the SQL expression?" or "Should I verify the join keys?")

Do not silently fix issues — always surface findings and ask before making changes.

---

Quick Reference

| Object | Execute With | Key Checks |

| --------- | --------------------------------------------- | ----------------------------------- |

| Metric | get_data_from_fields with metrics list | Value, magnitude, sign, consistency |

| Attribute | get_data_from_fields with attributes list | Range, distribution, NULLs |

| Entity | list_entities + get_data_from_fields | Exists, has rows, key unique |

| Relation | get_entity + cross-entity field query | Exists, joins work, no fan-out (validated as part of entity) |

| Domain | search_model + get_data_from_fields with domain | Exists, filters apply, fields scoped |