Skip to content

OpenLineage 1.26.0

Latest
Compare
Choose a tag to compare
@mobuchowski mobuchowski released this 20 Dec 16:59
· 5 commits to main since this release

1.26.0 - 2024-12-20

Added

  • dbt: Consume dbt structured logs and report progress in real time. #3314 @MassyB
    If --consume-structured-logs flag is set, dbt integration will consume dbt structured logs and report execution progress in real time.
  • Java: Add transform transport to allow event modification. #3301 @pawel-big-lebowski
    New transport type allows to modify the event based on the specified transformer class.
  • Java: Parallel event emitting for composite transport. #3305[#3305] @pawel-big-lebowski
    Emit events in parallel for composite transport. Running in parallel is a default behaviour continueOnFailure set to true. Default value of continueOnFailure got changed from false to true.
  • Spark: Collect ScanReport and CommitReport in OpenLineage events when dealing with Iceberg tables. #3256 @pawel-big-lebowski
    Collects additional Iceberg metrics for datasets read or written through the library. Visit Dataset Metrics docs for more details.
  • dbt: add support for duckdb adapter #3280 @mobuchowski
    Adds support for duckdb adapter for dbt integration.

Changed

  • Spark: Add DatasetFactory to support Dataset creation. #3207 @pawel-big-lebowski
    Adds DatasetFactory to support Dataset creation. This class is used to create Dataset instances for DatasetFactory.

Fixed