Merge requests · repos / data-engineering / Airflow DAGs

Add DAG for druid loading of webrequest_sampled_128

!364 · created Apr 21, 2023 by Mforns
- Merged
- 9
- Approved
updated May 03, 2023
Use tmp instead of wmf for creating temporary tables

!373 · created May 03, 2023 by Mforns
- Merged
- Approved
updated May 03, 2023
Hotfix: Fix maven coordinates for wmf-sparksqlclidriver-1.0.0.jar.

!371 · created Apr 27, 2023 by Xcollazo
updated Apr 27, 2023
Make SparkSqlOperator accept HQL files from GitLab raw URLs.

!362 · created Apr 12, 2023 by Xcollazo
- Merged
- 16
updated Apr 27, 2023
Migrate all DAGs for Druid loading of unique devices

!350 · created Apr 03, 2023 by Mforns
- Merged
- 29
- Approved
updated Apr 26, 2023
fixed bug in underlying DP repo

!366 · created Apr 26, 2023 by Htriedman
- Merged
updated Apr 26, 2023
Update URL for DP package and clean up DP DAG

!363 · created Apr 18, 2023 by Htriedman
- Merged
updated Apr 24, 2023
update poke interval for daily DP job

!357 · created Apr 10, 2023 by Htriedman
- Merged
- Approved
updated Apr 18, 2023
Use spark 3 bin & conf provided by deb package

!141 · created Sep 19, 2022 by Aqu
- Merged
- 2
- 1
updated Apr 13, 2023
wmf_airflow_common: Force HDFSArchiveOperator to have retries=0.

!361 · created Apr 11, 2023 by Xcollazo
- Merged
- 1
- Approved
updated Apr 12, 2023
Migrate refine webrequest job to Airflow

!260 · created Mar 06, 2023 by Aqu
- Merged
- 38
- Approved
updated Apr 12, 2023
remove network flows internal sanitization

!359 · created Apr 11, 2023 by Mforns
- Merged
- 1
- Approved
updated Apr 11, 2023
Update spark configuration of pageview hourly dag

!360 · created Apr 11, 2023 by Snwachukwu
updated Apr 11, 2023
Correct num_shards to 1 for network_flows_internal DAGs

!358 · created Apr 11, 2023 by Mforns
- Merged
- 1
- Approved
updated Apr 11, 2023
Suggestions over dag_properties

!353 · created Apr 07, 2023 by Aqu
- Merged
- 6
updated Apr 11, 2023
Increase default timeout of sensor for mediawiki_wikitext_history dag.

!356 · created Apr 10, 2023 by Mforns
- Merged
- 1
- Approved
updated Apr 10, 2023
platform_eng: bump coalesce and executor_memory on image_suggestions.

!355 · created Apr 07, 2023 by Xcollazo
- Merged
- 1
updated Apr 07, 2023
Bump section-topics to 0.6.0-v0.6.0.

!349 · created Apr 03, 2023 by Ghost User
- Merged
- 3
updated Apr 07, 2023
Bump image-suggestions to 0.11.0-v0.11.0.

!354 · created Apr 07, 2023 by Ghost User
- Merged
updated Apr 07, 2023
Fix typos in HTML emails of MW history check denormalize

!346 · created Apr 03, 2023 by Aqu
- Merged
- Approved
updated Apr 07, 2023

Admin message

Admin message

Admin message