Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(license): improve license normalization #7131

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

feat(license): improve license normalization #7131

wants to merge 15 commits into from

Conversation

pbaumard
Copy link
Contributor

@pbaumard pbaumard commented Jul 9, 2024 •
edited
Loading

Description

  • Normalize CycloneDX POM scanning
  • Normalize " ", "-only" and "-or-later" suffixes
  • Many new mappings from oss-review-toolkit
  • Space normalization, including newlines
  • Normalize US / UK spelling for license / licence.
  • Remove "THE " prefix
  • Remove common suffixes: " LICENSE", " LICENSED", -"LICENSE" and "-LICENSED"
  • Add MIT-0
  • Add tests

Related issues

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@pbaumard
(feat)Improve license normalization
44b0164 
- Remove "THE " prefix
- Remove "LICENSE " suffix
- Fix lowercase key
- Add BSD LICENSE 3
- Add ECLIPSE PUBLIC LICENSE 1.0
- Add MIT-0
- Add tests
@pbaumard pbaumard requested a review from knqyf263 as a code owner July 9, 2024 15:04
@CLAassistant
Copy link

CLAassistant commented Jul 9, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

DmitriyLewen
DmitriyLewen reviewed Jul 10, 2024
View reviewed changes
Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @pbaumard
Thanks for your work!

LGTM.
Can you fix tests?

DmitriyLewen
DmitriyLewen reviewed Jul 12, 2024
View reviewed changes
pkg/licensing/normalize.go
@@ -181,7 169,14 @@ var licenseSplitRegexp = regexp.MustCompile("(,?[_ ] (?:or|and)[_ ] )|(,[ ]*)")

func Normalize(name string) string {
name = strings.TrimSpace(name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add a new variable for the updated license name to avoid renaming the license if we don't find that license in the mapping

Suggested change
name = strings.TrimSpace(name)
license := strings.TrimSpace(name)

See https://github.com/aquasecurity/trivy/actions/runs/9888626513/job/27317191927?pr=7131#step:9:2387

Copy link
Contributor Author

@pbaumard pbaumard Jul 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also performed additional space and spelling normalizations and added many new mappings from oss-review-toolkit

I need to fix integration tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to fix integration tests.

You can use mage test:updateGolden command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@knqyf263 knqyf263 Awaiting requested review from knqyf263 knqyf263 is a code owner

@DmitriyLewen DmitriyLewen Awaiting requested review from DmitriyLewen DmitriyLewen is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@pbaumard @CLAassistant @DmitriyLewen