Does a green latest partition prove the dataset is complete?
Today’s rows pass. An older promised partition is still absent.
The question
Lumen promises four daily event partitions to a reporting consumer. A monitor reports green after checking the latest completed partition. An AI-written incident summary says the full reporting interval is healthy.
| Expected source rows | Warehouse rows | Row checks | |
|---|---|---|---|
| Nov 6 | 100 | 100 | Pass |
| Nov 7 | 100 | 0 | No partition |
| Nov 8 | 100 | 100 | Pass |
| Nov 9 | 100 | 100 | Pass |
Authored source manifest: four completed, non-empty source partitions of 100 rows each. The warehouse latest partition is Nov9.
| Data interval | Completion | Validation interval | |
|---|---|---|---|
| Backfill job | Nov7 | Not completed | None |
| Scheduled job | Nov9 | Completed | Nov9 |
The Nov9 test is aligned to its own completed job. It makes no statement about Nov7.
“What is healthy, what is missing, and which interval should the repair validate?”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Separate freshness from coverage
What would you do here? Open the move you’d make.
Report fresh latest data and incomplete interval
The strong move. A passing latest slice does not supply a missing older slice.
Report healthy because the newest test passed
Tempting, but it misses something. That test only examined Nov9.
Report every stored row invalid
Tempting, but it misses something. The three present partitions pass their stated row checks.
See how a strong candidate says it
“Nov9 is present and its rows pass, so the latest slice is fresh under this packet. The promised four-day dataset is incomplete because Nov7 is absent. I would not report the whole interval as healthy.”
Why it works: A passing latest slice does not supply a missing older slice.
Step 2: Reconcile the declared interval
What would you do here? Open the move you’d make.
Count zero failing rows as zero missing rows
Tempting, but it misses something. There are no Nov7 rows to fail the row-level checks.
Compare only Nov9 source and warehouse
Tempting, but it misses something. That leaves the promised older interval untested.
Reconcile four expected partitions and the missing 100 rows
The strong move. Missing partitions do not generate bad rows for a row validator to find.
See how a strong candidate says it
“The source manifest expects 400 rows across four partitions; the warehouse has 300 across three. Nov7 contributes the missing 100. I would check partition coverage and counts against the source rather than infer completeness from no validation failures.”
Why it works: Missing partitions do not generate bad rows for a row validator to find.
Step 3: Repair and validate the actual batch
What would you do here? Open the move you’d make.
Rerun today’s scheduled test
Tempting, but it misses something. A new Nov9 result does not test the repaired Nov7 partition.
Validate Nov7 on completion, then reconcile the full interval
The strong move. The repair evidence must describe the repaired interval.
Add another fixed cron offset
Tempting, but it misses something. A guessed time does not identify which backfill actually completed.
See how a strong candidate says it
“I would complete the Nov7 backfill, then validate its Nov7 output using the actual completed run’s data interval. I would also repeat the four-day completeness reconciliation before clearing the incident.”
Why it works: The repair evidence must describe the repaired interval.
Step 4: State the remaining limit
What would you do here? Open the move you’d make.
Clear only the verified four-day coverage claim
The strong move. Completeness is relative to an explicit source and interval.
Claim all historical data complete
Tempting, but it misses something. Only four declared source partitions were compared.
Keep the incident open even if the stated breach is repaired
Tempting, but it misses something. Closure should follow the actual contract evidence, with unrelated risks tracked separately.
See how a strong candidate says it
“If the completed Nov7 partition contains the expected 100 valid rows, the manifest reconciliation reaches 400. That proves coverage of this declared four-day packet, not that every business event was captured or all future loads are safe.”
Why it works: Completeness is relative to an explicit source and interval.
The answer, in short
- Decision
- Repair Nov7 and verify the full promised interval.
- Because
- The latest partition is healthy while the source-to-warehouse coverage is 300 of 400 rows.
- What would change it
- A revised source manifest, a declared empty Nov7 source partition or a different consumer interval would change the expected coverage.
Common traps
- Treating no failing rows as no missing data.
- Running a new latest-partition test as evidence for an old backfill.
Check your answer
- Did you separate latest freshness from interval completeness?
- Did you calculate the missing 100 of 400 rows?
- Did the repair test follow Nov7 completion?
- Did you limit closure to the declared manifest and interval?
Last reviewed Nov 10, 2026