Skip to content

Generate canary events twice an hour

Aqu requested to merge T356762_generate_canary_events_twice_an_hour into main

The new Refine process senses _IMPORTED flags generated by Gobblin.

But some streams imported with Gobblin got missing _IMPORTED flags for some partitions. When there are no events in the previous hour, the Gobblin code does not add the flag for the last hour. When there is a single canary event per hour, this could happen.

Let's produce canary events twice an hour to prevent this problem.

Bug: T365223

Edited by Aqu

Merge request reports