Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark: Exclude META-INF/*TransportBuilder from Spark Extension Interfaces #3244

Merged
merged 2 commits into from
Nov 15, 2024

Conversation

tnazarew
Copy link
Contributor

@tnazarew tnazarew commented Nov 14, 2024

Problem

After building spark extension interfaces there is META-INF/services/io.openlineage.client.transports.TransportBuilder in created jar, which causes conflicts when used by e.g. big-query-connector.

Solution

exclude TransportBuilder in jar creation

One-line summary:

Remove unnecessary item from spark-extension-interfaces to avoid conflicts.

Checklist

  • You've signed-off your work
  • Your pull request title follows our guidelines
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • Your comment includes a one-liner for the changelog about the specific purpose of the change (not required for changes to tests, docs, or CI config)
  • You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)
  • You've added a header to source files (if relevant)

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2024 contributors to the OpenLineage project

@tnazarew tnazarew changed the title exclude TransportBuilder from META-INF Spark: Exclude META-INF/*TransportBuilder from Spark Extension Interfaces Nov 14, 2024
@boring-cyborg boring-cyborg bot added the area:documentation Improvements or additions to documentation label Nov 14, 2024
@tnazarew tnazarew marked this pull request as ready for review November 15, 2024 13:27
@tnazarew tnazarew requested a review from a team as a code owner November 15, 2024 13:27
@tnazarew tnazarew force-pushed the spark/exclude-transport-builder-from-extensions branch from 823c630 to 9285ec7 Compare November 15, 2024 13:28
@mobuchowski mobuchowski merged commit f7942cf into main Nov 15, 2024
53 checks passed
@mobuchowski mobuchowski deleted the spark/exclude-transport-builder-from-extensions branch November 15, 2024 14:20
mayurmadnani pushed a commit to mayurmadnani/OpenLineage that referenced this pull request Nov 19, 2024
…aces (OpenLineage#3244)

* exclude TransportBuilder from META-INF

Signed-off-by: tnazarew <[email protected]>

* add changelog entry

Signed-off-by: tnazarew <[email protected]>

---------

Signed-off-by: tnazarew <[email protected]>
Signed-off-by: Mayur Madnani <10260496 [email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:documentation Improvements or additions to documentation area:integration/spark
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants