You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/05/06 10:11:26 UTC

[GitHub] [skywalking-eyes] kezhenxu94 commented on a diff in pull request #103: Provide summary flag to generate the license summary file

kezhenxu94 commented on code in PR #103:
URL: https://github.com/apache/skywalking-eyes/pull/103#discussion_r866652562


##########
README.md:
##########
@@ -157,11 +157,15 @@ INFO Totally checked 20 files, valid: 10, invalid: 10, ignored: 0, fixed: 10
 
 This command serves as assistance for human beings to audit the dependencies license, it's exit code is always 0.
 
-You can also use the `--output` or `-o` to save the dependencies' `LICENSE` files to a specified directory so that
-you can put them in distribution package if needed.
+We also support two flag:

Review Comment:
   ```suggestion
   We also support two flags:
   ```



##########
README.md:
##########
@@ -486,6 +494,10 @@ dependency: # <15>
 14. The `comment_style_id` set the license header comment style, it's the `id` at the `styles.yaml`.
 15. The `dependency` section is configurations for resolving dependencies' licenses.
 16. The `files` are the files that declare the dependencies of a project, typically, `go.mod` in Go project, `pom.xml` in maven project, and `package.json` in NodeJS project. If it's a relative path, it's relative to the `.licenserc.yaml`.
+17. Declare the license which cannot be identified as a dependency.

Review Comment:
   ```suggestion
   17. Declare the licenses which cannot be identified by this tool.
   ```



##########
README.md:
##########
@@ -486,6 +494,10 @@ dependency: # <15>
 14. The `comment_style_id` set the license header comment style, it's the `id` at the `styles.yaml`.
 15. The `dependency` section is configurations for resolving dependencies' licenses.
 16. The `files` are the files that declare the dependencies of a project, typically, `go.mod` in Go project, `pom.xml` in maven project, and `package.json` in NodeJS project. If it's a relative path, it's relative to the `.licenserc.yaml`.
+17. Declare the license which cannot be identified as a dependency.
+18. The `name` of the dependency, The name is different for different projects, `PackagePath` in Go project, `GroupID:ArtifactID` in maven project, `PackageName` in NodeJS project.
+19. The `version` of the dependency, It's locked, prevent version update and change the License.

Review Comment:
   ```suggestion
   19. The `version` of the dependency, it's locked, preventing license changed between different versions.
   ```



##########
commands/deps_resolve.go:
##########
@@ -31,29 +31,40 @@ import (
 )
 
 var outDir string
+var summaryTplPath string
+var summaryTpl *template.Template
 
 func init() {
 	DepsResolveCommand.PersistentFlags().StringVarP(&outDir, "output", "o", "",
 		"the directory to output the resolved dependencies' licenses, if not set the dependencies' licenses won't be saved")
+	DepsResolveCommand.PersistentFlags().StringVarP(&summaryTplPath, "summary", "s", "",
+		"the template file to write the summary the dependencies' licenses, if not set the licenses just print as the table")

Review Comment:
   The result is always printed as a table no matter whether `-s` is specified, no? If so why bother to say `if not set the licenses just print as the table`? This makes me feel if I set `-s` the result won't be printed as a table



##########
commands/deps_resolve.go:
##########
@@ -102,3 +116,21 @@ func writeLicense(result *deps.Result) {
 		return
 	}
 }
+
+func writeSummary(rep *deps.Report) {
+	file, err := os.Create(filepath.Join(outDir, "LICENSE"))
+	if err != nil {
+		logger.Log.Errorf("failed to create summary license file %s: %v", "LICENSE", err)
+		return
+	}
+	defer file.Close()
+	summary, err := deps.GenerateSummary(summaryTpl, &Config.Header, rep)
+	if err != nil {
+		logger.Log.Errorf("failed to generate summary content: %v", err)
+		return
+	}
+	_, err = file.WriteString(summary)
+	if err != nil {
+		logger.Log.Errorf("failed to write summary file, %v", err)

Review Comment:
   Return the error and handle it from the caller method



##########
pkg/deps/golang.go:
##########
@@ -119,7 +125,10 @@ func (resolver *GoModResolver) ResolvePackageLicense(module *packages.Module, re
 			}
 			identifier, err := license.Identify(module.Path, string(content))
 			if err != nil {
-				return err
+				if declareLicense == nil {

Review Comment:
   If the users already declare this license I don't think we need to `license.Identify` it, this gives the users ability to override the license if we make something wrong



##########
pkg/deps/summary.go:
##########
@@ -0,0 +1,114 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package deps
+
+import (
+	"bytes"
+	"os"
+	"path/filepath"
+	"regexp"
+	"text/template"
+
+	"github.com/apache/skywalking-eyes/pkg/header"
+	"github.com/apache/skywalking-eyes/pkg/license"
+)
+
+var fileNamePattern = regexp.MustCompile(`[^a-zA-Z0-9\\.\-]`)
+
+type SummaryRenderContext struct {
+	LicenseContent string                       // Current project license content
+	Groups         []*SummaryRenderLicenseGroup // All dependency license groups
+}
+
+type SummaryRenderLicenseGroup struct {
+	Name string                  // Aggregate all same license ID dependencies
+	Deps []*SummaryRenderLicense // Same license ID dependencies
+}
+
+type SummaryRenderLicense struct {
+	Name      string // Dependency name
+	Version   string // Dependency version
+	Location  string // The filename of generated license file
+	LicenseID string // License ID
+}
+
+func GenerateDependencyLicenseFilename(result *Result) string {

Review Comment:
   This method is not used in `--summary` flag, it's used in `--output` flag, why put it here (`summary.go`)?



##########
README.md:
##########
@@ -157,11 +157,15 @@ INFO Totally checked 20 files, valid: 10, invalid: 10, ignored: 0, fixed: 10
 
 This command serves as assistance for human beings to audit the dependencies license, it's exit code is always 0.
 
-You can also use the `--output` or `-o` to save the dependencies' `LICENSE` files to a specified directory so that
-you can put them in distribution package if needed.
+We also support two flag:
+
+|Flag name|Short name|Description|
+|---------|----------|-----------|
+|--output|-o|Save the dependencies' `LICENSE` files to a specified directory so that you can put them in distribution package if needed.|
+|--summary|-s|Based on the template, aggregate all dependency information and generate a `LICENSE` file.|

Review Comment:
   ```suggestion
   |`--output`|`-o`|Save the dependencies' `LICENSE` files to a specified directory so that you can put them in distribution package if needed.|
   |`--summary`|`-s`|Based on the template, aggregate all dependency information and generate a `LICENSE` file.|
   ```



##########
commands/deps_resolve.go:
##########
@@ -31,29 +31,40 @@ import (
 )
 
 var outDir string
+var summaryTplPath string
+var summaryTpl *template.Template
 
 func init() {
 	DepsResolveCommand.PersistentFlags().StringVarP(&outDir, "output", "o", "",
 		"the directory to output the resolved dependencies' licenses, if not set the dependencies' licenses won't be saved")
+	DepsResolveCommand.PersistentFlags().StringVarP(&summaryTplPath, "summary", "s", "",
+		"the template file to write the summary the dependencies' licenses, if not set the licenses just print as the table")
 }
 
-var fileNamePattern = regexp.MustCompile(`[^a-zA-Z0-9\\.\-]`)
-
 var DepsResolveCommand = &cobra.Command{
 	Use:     "resolve",
 	Aliases: []string{"r"},
 	Long:    "resolves all dependencies of a module and their transitive dependencies",
 	PreRunE: func(cmd *cobra.Command, args []string) error {
-		if outDir == "" {
-			return nil
-		}
-		absPath, err := filepath.Abs(outDir)
-		if err != nil {
-			return err
+		if outDir != "" {
+			absPath, err := filepath.Abs(outDir)
+			if err != nil {
+				return err
+			}
+			outDir = absPath
+			if err := os.MkdirAll(outDir, 0o700); err != nil && !os.IsExist(err) {
+				return err
+			}
 		}
-		outDir = absPath
-		if err := os.MkdirAll(outDir, 0o700); err != nil && !os.IsExist(err) {
-			return err
+		if summaryTplPath != "" {
+			if outDir == "" {
+				return fmt.Errorf("please provide the output directory to write the license summary file")

Review Comment:
   `outDir` is for `-o`, I think it's not related to `-s`, if you do this, users have to use `-o` if they want to use `-s`, i.e. they can't use `-s` without `-o`.
   
   I thought we can simply generate the `LICENSE` file in the same directory as `LICENSE.tpl`



##########
pkg/deps/summary.go:
##########
@@ -0,0 +1,114 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package deps
+
+import (
+	"bytes"
+	"os"
+	"path/filepath"
+	"regexp"
+	"text/template"
+
+	"github.com/apache/skywalking-eyes/pkg/header"
+	"github.com/apache/skywalking-eyes/pkg/license"
+)
+
+var fileNamePattern = regexp.MustCompile(`[^a-zA-Z0-9\\.\-]`)
+
+type SummaryRenderContext struct {
+	LicenseContent string                       // Current project license content
+	Groups         []*SummaryRenderLicenseGroup // All dependency license groups
+}
+
+type SummaryRenderLicenseGroup struct {
+	Name string                  // Aggregate all same license ID dependencies
+	Deps []*SummaryRenderLicense // Same license ID dependencies
+}
+
+type SummaryRenderLicense struct {
+	Name      string // Dependency name
+	Version   string // Dependency version
+	Location  string // The filename of generated license file
+	LicenseID string // License ID
+}
+
+func GenerateDependencyLicenseFilename(result *Result) string {
+	filename := string(fileNamePattern.ReplaceAll([]byte(result.Dependency), []byte("-")))
+	return "license-" + filename + ".txt"
+}
+
+func ParseTemplate(path string) (*template.Template, error) {
+	absPath, err := filepath.Abs(path)
+	if err != nil {
+		return nil, err
+	}
+	tpl, err := os.ReadFile(absPath)
+	if err != nil {
+		return nil, err
+	}
+	return template.New("summary").Parse(string(tpl))
+}
+
+// GenerateSummary generate the summary content by template, license config and dependency report
+func GenerateSummary(tpl *template.Template, head *header.ConfigHeader, rep *Report) (string, error) {
+	var r bytes.Buffer
+	context, err := generateSummaryRenderContext(head, rep)
+	if err != nil {
+		return "", err
+	}
+	if err := tpl.Execute(&r, context); err != nil {
+		return "", err
+	}
+	return r.String(), nil
+}
+
+func generateSummaryRenderContext(head *header.ConfigHeader, rep *Report) (*SummaryRenderContext, error) {
+	// the license id of the project
+	licenseContent, err := license.GetLicenseContent(head.License.SpdxID)
+	if err != nil {
+		return nil, err
+	}

Review Comment:
   `head.License.SpdxID` is not required so it might be empty, users can use other forms to specify their license, not just `SpdxID`. You need to take care the case where `SpdxID` is not set



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org