Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(template) Add misconfigurations to gitlab codequality report #1756

Merged
merged 2 commits into from
Mar 30, 2022

Conversation

natefive
Copy link
Contributor

Gitlab codequality report for misconfigurations was missing

Usage example:
trivy fs --security-checks config,vuln --format template --template "@contrib/gitlab-codequality.tpl" -o report.json {folder}

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@CLAassistant
Copy link

CLAassistant commented Feb 22, 2022

CLA assistant check
All committers have signed the CLA.

@krol3
Copy link
Contributor

krol3 commented Feb 22, 2022

@natefear thanks for your contribution, do you think that you could update the documentation about how to use it, some screenshots about the report will be nice to have in the documentation too.

https://aquasecurity.github.io/trivy/v0.23.0/advanced/integrations/gitlab-ci/

@natefive
Copy link
Contributor Author

natefive commented Feb 23, 2022

Hi @krol3 I'll add some screen shots in a bit I just would like to discuss the template further first :)

At present the gitlab-codequality.tpl template will set the package name and version as the path for vulnerabilities (cve's)

"path": "{{ .PkgName }}-{{ .InstalledVersion }}",
whilst this is more useful when doing an image scan because the target
{{- $target := .Target }}
is just the image name e.g. aquasec/trivy (alpine 3.15.0) when scanning trivy, this is however not as useful as it could be when doing filesystem scan as in this scenario the .Target key is able to point to the exact file the vulnerability (cve) originates from e.g. a pip file and you can then click on it in the Gitlab UI to take you to the file.

To try to get the best of both, should I:

  1. Always set path to $target for vulnerabilities (cve's) but include the package name and version in the description
  2. Try to get the ArtifactType field exposed to the report templating logic then use this to create conditions - at present the untemplated json report includes an ArtifactType field (see code block below) which lets you know if the report is from an image or file system scan however the templating fuction only knows about the contents of the results array. See error: "output template" at <.ArtifactType>: can't evaluate field ArtifactType in type report.Results"
  3. Create a separate template for filesystem scans
  "SchemaVersion": 2,
  "ArtifactName": ".",
  "ArtifactType": "filesystem",
  "Metadata": {
    "ImageConfig": {
      "architecture": "",
      "created": "0001-01-01T00:00:00Z",
      "os": "",
      "rootfs": {
        "type": "",
        "diff_ids": null
      },
      "config": {}
    }
  },
  "Results": [

@natefive
Copy link
Contributor Author

natefive commented Mar 1, 2022

@knqyf263 would I be able to get your thoughts on the above ^? :)

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 2, 2022

To be honest, I've never used GitLab Code Quality, but it seems to be associated with the project's files. I feel like we should use path for file paths. #1 looks good from that perspective.

@natefive
Copy link
Contributor Author

natefive commented Mar 2, 2022

@knqyf263 I've added screen shots to make it clearer :)

At present (master) we get the severity, CVE ID and title in the description and package name and version in the path for CVE's when doing an image scan using the gitlab code quality template:
Screenshot from 2022-03-02 13-42-13

and the same again for CVE's when doing an filesystem scan using the gitlab code quality template:
Screenshot from 2022-03-02 13-43-16

After implementing option 1: Always setting path to $target for vulnerabilities (CVE's) but include the package name and version in the description

The package name and version would be moved the the description (taking suggestions on what separators to use) and the path would output the docker image path (making it abit easier to manually find the source of the CVE)
Screenshot from 2022-03-02 13-51-47
N.B redacted part of the image path hence the blank bits

however this would greatly improve the output of CVE's picked up by filesystem scans as the report can then link you to the file where the CVE originates
Screenshot from 2022-03-02 13-50-03
N.B redacted part of the file path hence the blank bits

Copy link
Contributor

@krol3 krol3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the screenshots in 'gitlab-ci.md' about the expected result in the Gitlab/codeQuality too.

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 3, 2022

@natefear Thanks for the detailed explanation. It looks good.

@natefive natefive force-pushed the feat/template-codequality branch 5 times, most recently from f832ace to 8cab693 Compare March 3, 2022 15:00
@natefive
Copy link
Contributor Author

natefive commented Mar 3, 2022

@krol3 @knqyf263 Added a screen shot of the expected output when using the template to the docs, let me know if you want me to crop the screen shot further :) thanks

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 3, 2022

Thanks, but the link seems to be broken.

image

You can see the doc by running make mkdocs-serve and visiting http://localhost:8000. Also, you can refer to other places.

![standalone](../../imgs/image.png)

@natefive
Copy link
Contributor Author

natefive commented Mar 3, 2022

@knqyf263 It works here and locally via mkdocs, I think it's broken when viewing in rich diff mode in the files changes UI because it's trying to look in the main branch -> https://github.com/aquasecurity/trivy/blob/main/docs/imgs/gitlab-codequality.png which is 404

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 6, 2022

@knqyf263 It works here and locally via mkdocs, I think it's broken when viewing in rich diff mode in the files changes UI because it's trying to look in the main branch -> https://github.com/aquasecurity/trivy/blob/main/docs/imgs/gitlab-codequality.png which is 404

We use MkDocs and it must work here after building the doc. You don't need to care about how it looks like on GitHub.
https://aquasecurity.github.io/trivy/v0.24.2/advanced/integrations/gitlab-ci/

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 6, 2022

Oh, I missed you said it worked locally via mkdocs. Let me check it again.

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 6, 2022

Again, it doesn't work. The path seems to be wrong.
image

@natefive
Copy link
Contributor Author

natefive commented Mar 7, 2022

@knqyf263 ahh you're right apologies I'd not pushed the extra ../ - I've pushed it now

@natefive
Copy link
Contributor Author

natefive commented Mar 9, 2022

@knqyf263 @krol3 can someone kick the tests off, approve and merge this if you think it's ready :) thanks

the tests had already run successfully once, but I've had to rebase because the main branch has been updated

@natefive
Copy link
Contributor Author

@knqyf263 @krol3 - by the time someone responds to run the tests or approves the PR my branch becomes out of sync with main :( try again please?

@natefive
Copy link
Contributor Author

@knqyf263 @krol3 not sure if one of you triggered the tests, but they're done and ready for approval :) thanks

@natefive natefive requested a review from krol3 March 16, 2022 14:06
Copy link
Contributor

@krol3 krol3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement! Please consider this PR #1801
"Updated the GitLab CI examples to use the environment variables TRIVY_CACHE_DIR and TRIVY_NO_PROGRESS instead of providing them for every commands"

@natefive
Copy link
Contributor Author

@krol3 Seems odd that you want me to include the proposed changes from someone else's PR in this one? shouldn't you be letting me merge and then asking other PR author to rebase so they can add it where relevant

@krol3
Copy link
Contributor

krol3 commented Mar 17, 2022

@natefear, I was trying to tell you, is that you'll probably need to do a remerge. I will focus on reviewing your PR

@natefive
Copy link
Contributor Author

@krol3 ahh okay sorry didn't see there MR had already been approved :) I guess I'll wait for them to merge

@knqyf263
Copy link
Collaborator

knqyf263 commented Mar 17, 2022

Merged #1801

@natefive
Copy link
Contributor Author

@krol3 @knqyf263 rebased to include the CI env var changes

@natefive
Copy link
Contributor Author

@krol3 @knqyf263 rebased again :(

Copy link
Contributor

@krol3 krol3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @natefear

@natefive
Copy link
Contributor Author

@krol3 @knqyf263 are either of you authorised to merge :) ty

@knqyf263 knqyf263 merged commit cb171ea into aquasecurity:main Mar 30, 2022
@knqyf263
Copy link
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants