Merge requests · repos / data-engineering / Airflow DAGs

search: do not use hive partitions to wait for wmf_raw.mediawiki_page

!465 · created Jul 24, 2023 by DCausse
- Merged
- 5
- Approved
updated Aug 02, 2023
Update airflow to version 2.6.3

!450 · created Jul 04, 2023 by Btullis
- Merged
- 1
- Approved
updated Aug 01, 2023
Add Spark tuning varprops to pageview_hourly DAG.

!471 · created Jul 31, 2023 by Xcollazo
- Merged
- 2
updated Jul 31, 2023
Add Spark tuning varprops to pageview_actor_hourly DAG.

!469 · created Jul 28, 2023 by Xcollazo
- Merged
- 8
- 1
- Approved
updated Jul 31, 2023
Use refinery-hive version 0.2.20 for wikifunctions

!470 · created Jul 28, 2023 by Milimetric
- Merged
updated Jul 28, 2023
Don't pollute skein logs with heartbeats.

!468 · created Jul 28, 2023 by Xcollazo
- Merged
- 3
- Approved
updated Jul 28, 2023
Ingest three additional Hive databases to DataHub

!466 · created Jul 26, 2023 by Btullis
- Merged
- 3
- Approved
updated Jul 27, 2023
Run page_content_change_to_wikitext_raw DAG serially.

!457 · created Jul 14, 2023 by Xcollazo
- Merged
- 4
updated Jul 24, 2023
search: fix table name for wmf_raw.mediawiki_page

!464 · created Jul 24, 2023 by DCausse
- Merged
updated Jul 24, 2023
Fix partition definition for wmf_raw.mediawiki_page_table

!461 · created Jul 18, 2023 by DCausse
- Merged
- 1
- Approved
updated Jul 24, 2023
Fix cassandra loading jobs

!462 · created Jul 19, 2023 by Joal
- Merged
- Approved
updated Jul 19, 2023
Split cassandra loading jobs by datasets

!455 · created Jul 12, 2023 by Joal
- Merged
- 23
- Approved
updated Jul 19, 2023
Make black and isort happy.

!456 · created Jul 14, 2023 by Xcollazo
updated Jul 14, 2023
Implements a job that transforms event data to a dump friendly format.

!430 · created Jun 14, 2023 by Xcollazo
- Merged
- 21
- Approved
updated Jul 14, 2023
search: Change articletopic source to the outlink model

!448 · created Jun 30, 2023 by Ebernhardson
- Merged
- 3
- Approved
updated Jul 10, 2023
Bump imagerec to 0.6.0-v0.6.0.

!452 · created Jul 05, 2023 by Ghost User
- Merged
- Approved
updated Jul 05, 2023
Bump section-topics to 0.11.0-v0.11.0.

!451 · created Jul 05, 2023 by Ghost User
- Merged
- Approved
updated Jul 05, 2023
Update airflow to version 2.6.1

!385 · created May 15, 2023 by Btullis
- Merged
- 7
updated Jun 29, 2023
Remove profiling from DataHub's Druid ingestion

!444 · created Jun 27, 2023 by Mforns
updated Jun 27, 2023
CI pipeline revamp

!417 · created Jun 05, 2023 by Aqu
- Merged
- 34
- Approved
updated Jun 27, 2023

Admin message

Admin message

Admin message