Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable building input/output facets from the visitors #3207

Merged
merged 2 commits into from
Nov 12, 2024

Conversation

pawel-big-lebowski
Copy link
Collaborator

@pawel-big-lebowski pawel-big-lebowski commented Oct 31, 2024

Currently, visitors and dataset builders create dataset objects through the DatasetFactory. This concept does not allow providing input nor output facets.

This PR enables it by including DatasetCompositeFacetsBuilder to include already existing facets as well as input/output ones. Extending existing methods to allow it requires changes in several places of the codebase.

@boring-cyborg boring-cyborg bot added area:client/java openlineage-java area:integration/spark area:tests Testing code language:java Uses Java programming language labels Oct 31, 2024
@pawel-big-lebowski pawel-big-lebowski force-pushed the spark/dataset-factory-refactor branch from 4c24424 to 6f49f6a Compare November 11, 2024 12:09
@pawel-big-lebowski pawel-big-lebowski marked this pull request as ready for review November 11, 2024 12:12
@pawel-big-lebowski pawel-big-lebowski requested a review from a team as a code owner November 11, 2024 12:12
@pawel-big-lebowski pawel-big-lebowski force-pushed the spark/dataset-factory-refactor branch from 6f49f6a to 13f2d9a Compare November 12, 2024 12:42
@boring-cyborg boring-cyborg bot added the area:documentation Improvements or additions to documentation label Nov 12, 2024
@pawel-big-lebowski pawel-big-lebowski force-pushed the spark/dataset-factory-refactor branch from 13f2d9a to 83a3f59 Compare November 12, 2024 12:59
@pawel-big-lebowski pawel-big-lebowski force-pushed the spark/dataset-factory-refactor branch from 83a3f59 to e80067b Compare November 12, 2024 13:14
@mobuchowski mobuchowski added the full-tests Run full tests label Nov 12, 2024
@mobuchowski mobuchowski merged commit 1252356 into main Nov 12, 2024
67 checks passed
@mobuchowski mobuchowski deleted the spark/dataset-factory-refactor branch November 12, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:client/java openlineage-java area:documentation Improvements or additions to documentation area:integration/spark area:tests Testing code full-tests Run full tests language:java Uses Java programming language
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants