Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move http transport to httpclient5, move kinesis to separate module, update Flink integration's structure #3205

Merged
merged 1 commit into from
Nov 5, 2024
Merged

Move http transport to httpclient5, move kinesis to separate module, update Flink integration's structure #3205

merged 1 commit into from
Nov 5, 2024

Conversation

mobuchowski
Copy link
Member

@mobuchowski mobuchowski commented Oct 30, 2024 •
edited
Loading

To remove usage of vulnareble library commons-codec coming from outdated HTTP library #3182

  • removed it's usage
  • moved KinesisTransport to separate module, as underneath Kinesis depends on old http client
  • restructured Flink integration - removed cyclic dependency between OL integration and example jobs, now the example jobs are build separately and depend on generated shadow jar, and they themselves produce shadow jar that's used in the integration tests, rather than having direct code dependency.

@boring-cyborg boring-cyborg bot added area:client/java openlineage-java area:documentation Improvements or additions to documentation area:tests Testing code language:java Uses Java programming language labels Oct 30, 2024
@mobuchowski mobuchowski mentioned this pull request Oct 30, 2024
Closed
8 tasks
@mobuchowski mobuchowski force-pushed the bump-java-client branch 5 times, most recently from 7185bee to afd56c0 Compare November 3, 2024 20:20
@boring-cyborg boring-cyborg bot added the area:ci CI label Nov 3, 2024
@mobuchowski mobuchowski force-pushed the bump-java-client branch 5 times, most recently from e66fc59 to ebd787b Compare November 4, 2024 15:53
@mobuchowski mobuchowski marked this pull request as ready for review November 4, 2024 16:03
@mobuchowski mobuchowski requested a review from a team as a code owner November 4, 2024 16:03
@mobuchowski mobuchowski changed the title Move kinesis to separate module, move http transport to httpclient5 Move http transport to httpclient5, move kinesis to separate module, update Flink integration's structure Nov 4, 2024
@mobuchowski mobuchowski requested review from arturowczarek and tnazarew and removed request for a team November 4, 2024 16:09
@mobuchowski mobuchowski force-pushed the bump-java-client branch 3 times, most recently from 4c41cb7 to 081d31e Compare November 4, 2024 21:49
@codecov-commenter
Copy link

codecov-commenter commented Nov 4, 2024 •
edited
Loading

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.73%. Comparing base (7fd8762) to head (be6f6b0).
Report is 5 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #3205    /-   ##
=======================================
  Coverage   85.73%   85.73%           
=======================================
  Files          54       54           
  Lines        3119     3119           
=======================================
  Hits         2674     2674           
  Misses        445      445

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

arturowczarek
arturowczarek approved these changes Nov 5, 2024
View reviewed changes
Copy link
Collaborator

@arturowczarek arturowczarek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flink Gradle part is not completely clear to me, but apart from that everything looks good.

tnazarew
tnazarew approved these changes Nov 5, 2024
View reviewed changes
Copy link
Contributor

@tnazarew tnazarew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good stuff 👍

client/java/src/main/java/io/openlineage/client/transports/FileTransport.java
eventAsJson.replace(System.lineSeparator(), "") System.lineSeparator(),
StandardCharsets.UTF_8,
true);
Files.createDirectories(file.getParentFile().toPath());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious, what does that improve?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed usage of dependency, which was transitive - so no readding it as actual dependency.

@mobuchowski mobuchowski merged commit 46db181 into main Nov 5, 2024
90 checks passed
@mobuchowski mobuchowski deleted the bump-java-client branch November 5, 2024 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tnazarew tnazarew tnazarew approved these changes

@arturowczarek arturowczarek arturowczarek approved these changes

Assignees
No one assigned
Labels
area:ci CI area:client/java openlineage-java area:documentation Improvements or additions to documentation area:integration/flink area:integration/spark area:tests Testing code language:java Uses Java programming language
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mobuchowski @codecov-commenter @arturowczarek @tnazarew