-
Notifications
You must be signed in to change notification settings - Fork 26
Improve: Parse POM from maven repo to extract and save license information in Java DB #21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
namandf
wants to merge
45
commits into
aquasecurity:main
Choose a base branch
from
namandf:java-license
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
45 commits
Select commit
Hold shift + click to select a range
c3c9e2c
improve: [https://github.com/aquasecurity/trivy/discussions/4236] Par…
namandf e5e60fa
Add notes to trim license string beyond a certain limit
namandf eb56cc3
Remove dependency on an external module for POM parsing.
namandf ecab1d4
Update separator for license strings to | instead of ,
namandf 8ff0829
go mod tidy
namandf 6ea37a5
Deduplicate licenses fetched from POM
namandf 239ddc8
Use license classifier for information obtained from POM
namandf 4c23e78
Cache license information and use it on the fly while writing data to db
namandf 302e619
Cache directory creation
namandf 153a133
Cache directory creation
namandf 4dbd769
Cache directory creation
namandf 9bc1dbb
Add test cases for license classifier
namandf 28ef2b3
sort license keys before they are written to indexes cache directory
namandf ff74564
update comment
namandf 9a557a4
update comment
namandf e214d84
precautionary check to limit licnense strings to 30 characters
namandf 21885b1
Update timeout for license classification
namandf 3a4156e
update batch size
namandf afd7dbb
license meta variable name change
namandf c5b40fd
use crawler http client while creating license files
namandf c7fca19
Update license file creation logic
namandf 6608af3
test
namandf 560980f
Add timeout for prepareClassifierData
namandf 0608b3e
improvements
namandf 5ec049b
improvements
namandf 2467ee3
improve processing of license classifier results
namandf 30bc298
Remove files list from crawler struct
namandf d483d27
Get rid of processFilesMap
namandf 7be6cf4
Add constant for normalized license json file name
namandf 2f4c07c
Update test cases
namandf 1640f0a
clean up
namandf 512dba5
clean up
namandf c4108cd
clean up
namandf 7cc48c1
Add a utils file in crawler package
namandf 33f2473
Update comment
namandf b001b9f
Set classification confidence
namandf 029dcfe
Set classification confidence
namandf 176caa8
Add check for license string length
namandf 87c7b41
Speed up license file generation
namandf a0769b5
Add comment
namandf d7a9bdf
reuse http client
namandf 3f3344b
skip file copy error
namandf e991f76
Handle empty normalized license strings
namandf 674534e
Add temporary log to check license strings being trimmed
namandf 8a924f3
Increase license string length threshold to 150 characters
namandf File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.