You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2024/07/16 09:46:45 UTC
[I] CDXGen latest caues [airflow]
potiuk opened a new issue, #40810:
URL: https://github.com/apache/airflow/issues/40810
### Body
When using latest cdxgen for SBOM generation we get this:
```
breeze sbom update-sbom-information --airflow-version ${VERSION} --airflow-site-directory ${AIRFLOW_SITE_DIRECTORY} --force
Using default tag: latest
latest: Pulling from cyclonedx/cdxgen
Digest: sha256:d60b733eedc9170afcbf367582d0609dfe2bd941f241c470f998bddd2f4fe6c9
Status: Image is up to date for ghcr.io/cyclonedx/cdxgen:latest
ghcr.io/cyclonedx/cdxgen:latest
What's next:
View a summary of image vulnerabilities and recommendations → docker scout quickview ghcr.io/cyclonedx/cdxgen
Waiting for cdxgen server to start
Listening on 0.0.0.0 9090
The /Users/utkarsharma/sandbox/released_airflow/airflow-site/docs-archive/apache-airflow/2.9.3/sbom already exists. Forcing update
Attempting to update sbom for 2.9.3.
Updating sbom for Airflow 2.9.3 and python 3.6
Downloading https://raw.githubusercontent.com/apache/airflow/2.9.3/airflow/www/yarn.lock to /Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.6/yarn.lock
Downloaded https://raw.githubusercontent.com/apache/airflow/2.9.3/airflow/www/yarn.lock to /Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.6/yarn.lock
Downloading https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-no-providers-3.6.txt to
/Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.6/requirements.txt
The https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-no-providers-3.6.txt has not been found. Skipping
Failed to download constraints file for 2.9.3 and 3.6. Skipping
Updating sbom for Airflow 2.9.3 and python 3.7
Downloading https://raw.githubusercontent.com/apache/airflow/2.9.3/airflow/www/yarn.lock to /Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.7/yarn.lock
Downloaded https://raw.githubusercontent.com/apache/airflow/2.9.3/airflow/www/yarn.lock to /Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.7/yarn.lock
Downloading https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-no-providers-3.7.txt to
/Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.7/requirements.txt
The https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-no-providers-3.7.txt has not been found. Skipping
Failed to download constraints file for 2.9.3 and 3.7. Skipping
Updating sbom for Airflow 2.9.3 and python 3.8
Downloading https://raw.githubusercontent.com/apache/airflow/2.9.3/airflow/www/yarn.lock to /Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.8/yarn.lock
Downloaded https://raw.githubusercontent.com/apache/airflow/2.9.3/airflow/www/yarn.lock to /Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.8/yarn.lock
Downloading https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-no-providers-3.8.txt to
/Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.8/requirements.txt
Downloaded https://raw.githubusercontent.com/apache/airflow/constraints-2.9.3/constraints-no-providers-3.8.txt to
/Users/utkarsharma/sandbox/released_airflow/airflow/files/sbom/2.9.3/python3.8/requirements.txt
Generating sbom for Airflow 2.9.3 and python 3.8 with cdxgen
Triggering sbom generation in 2.9.3 via http://127.0.0.1:9090/sbom?path=/app/2.9.3/python3.8&project-name=apache-airflow&project-version=2.9.3&multiProject=true
file:///opt/cdxgen/server.js:97
options.projectType = options.type.split(",");
^
TypeError: Cannot read properties of undefined (reading 'split')
at parseQueryString (file:///opt/cdxgen/server.js:97:38)
at file:///opt/cdxgen/server.js:125:24
at call (/opt/cdxgen/node_modules/.pnpm/connect@3.7.0/node_modules/connect/index.js:239:7)
at next (/opt/cdxgen/node_modules/.pnpm/connect@3.7.0/node_modules/connect/index.js:183:5)
at next (/opt/cdxgen/node_modules/.pnpm/connect@3.7.0/node_modules/connect/index.js:161:14)
at compression (/opt/cdxgen/node_modules/.pnpm/compression@1.7.4/node_modules/compression/index.js:220:5)
at call (/opt/cdxgen/node_modules/.pnpm/connect@3.7.0/node_modules/connect/index.js:239:7)
at next (/opt/cdxgen/node_modules/.pnpm/connect@3.7.0/node_modules/connect/index.js:183:5)
at jsonParser (/opt/cdxgen/node_modules/.pnpm/body-parser@1.20.2/node_modules/body-parser/lib/types/json.js:113:7)
at call (/opt/cdxgen/node_modules/.pnpm/connect@3.7.0/node_modules/connect/index.js:239:7)
Node.js v22.4.0
Traceback (most recent call last):
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 537, in _make_request
response = conn.getresponse()
^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connection.py", line 466, in getresponse
httplib_response = super().getresponse()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1395, in getresponse
response.begin()
File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 325, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 294, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/util/retry.py", line 470, in increment
raise reraise(type(error), error, _stacktrace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/util/util.py", line 38, in reraise
raise value.with_traceback(tb)
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 537, in _make_request
response = conn.getresponse()
^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/urllib3/connection.py", line 466, in getresponse
httplib_response = super().getresponse()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1395, in getresponse
response.begin()
File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 325, in begin
version, status, reason = self._read_status()
^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 294, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/utkarsharma/sandbox/released_airflow/env/bin/breeze", line 8, in <module>
sys.exit(main())
^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/rich_click/rich_command.py", line 367, in __call__
return super().__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/rich_click/rich_command.py", line 152, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/airflow/dev/breeze/src/airflow_breeze/commands/sbom_commands.py", line 299, in update_sbom_information
produce_sbom_for_application_via_cdxgen_server(job, output=None)
File "/Users/utkarsharma/sandbox/released_airflow/airflow/dev/breeze/src/airflow_breeze/utils/cdxgen.py", line 467, in produce_sbom_for_application_via_cdxgen_server
return job.produce(output, port)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/airflow/dev/breeze/src/airflow_breeze/utils/cdxgen.py", line 385, in produce
response = requests.get(url)
^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/utkarsharma/sandbox/released_airflow/env/lib/python3.11/site-packages/requests/adapters.py", line 682, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
```
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40810:
URL: https://github.com/apache/airflow/issues/40810#issuecomment-2246520059
> Yes, good idea to use 10.7.x. 10.8.x adds multi type feature, which can have some bugs.
I updated to latest version all looks good and our sboms are nicely generated. BTW. I just opened discussion https://github.com/CycloneDX/cdxgen/discussions/1270 about ability of skipping `pip` installation when we pass a complete (including transitive) list of requirements.txt (pinned) - I would love to have such an option, that could speed up our SBOM generation a bit @prabhu ! Would love your carefull look at the discussion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40810:
URL: https://github.com/apache/airflow/issues/40810#issuecomment-2233596079
(i.e. we workarounded it by using previous version of cdxgen)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "prabhu (via GitHub)" <gi...@apache.org>.
prabhu commented on issue #40810:
URL: https://github.com/apache/airflow/issues/40810#issuecomment-2233536109
Looks like type parameter is treated as mandatory. Could you pass "universal" or "java" as workaround. Will work on a fix which is to add ? before .split
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #40810: CDXGen latest caues error when generating SBOM
URL: https://github.com/apache/airflow/issues/40810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #40810: CDXGen latest caues error when generating SBOM
URL: https://github.com/apache/airflow/issues/40810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40810:
URL: https://github.com/apache/airflow/issues/40810#issuecomment-2233593964
> Looks like type parameter is treated as mandatory. Could you pass "universal" or "java" as workaround. Will work on a fix which is to add ? before .split.
>
> PR is [here](https://github.com/CycloneDX/cdxgen/pull/1252)
Thanks! if it is going to be released soon, possibly we can just wait for it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40810:
URL: https://github.com/apache/airflow/issues/40810#issuecomment-2246476367
> Yes, good idea to use 10.7.x. 10.8.x adds multi type feature, which can have some bugs.
I updated to latest version and it works now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CDXGen latest caues error when generating SBOM [airflow]
Posted by "prabhu (via GitHub)" <gi...@apache.org>.
prabhu commented on issue #40810:
URL: https://github.com/apache/airflow/issues/40810#issuecomment-2233627089
Yes, good idea to use 10.7.x. 10.8.x adds multi type feature, which can have some bugs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org