[I] [Improvement] WARC writer: WARC-Protocol header to follow WARC field proposals [stormcrawler]

["sebastian-nagel (via GitHub)" <gi...@apache.org>]

sebastian-nagel opened a new issue, #1998:
URL: https://github.com/apache/stormcrawler/issues/1998

   ### What would you like to be improved?
   
   The WARC writer in external/warc.
   
   ### How should we improve?
   
   The `WARC-Protocol` headers holding the HTTP protocol and SSL/TLS versions should follow the current WARC field proposals
   - [WARC-Protocol](https://github.com/iipc/warc-specifications/issues/42)
   - [WARC-Cipher-Suite field proposal](https://github.com/iipc/warc-specifications/issues/86)
   
   The current form
   ```
   WARC-Protocol: h2,TLS_1_3,TLS_AES_128_GCM_SHA256
   ```
   should become
   ```
   WARC-Protocol: h2
   WARC-Protocol: TLS_1_3
   WARC-Cipher-Suite: TLS_AES_128_GCM_SHA256
   ```
   That is: split the values and put them separately into the two headers.
   
   See also: 
   - #828/#1010 which added the `WARC-Protocol` header in 2020.
   
   The `WARC-Protocol` field proposal has changed since 2020. The `WARC-Cipher-Suite` field was proposed in 2023. The proposals have been settled and both fields are now used in multiple projects / WARC writers.
   
   Unfortunately, both protocol-related versions and cipher suite are hold below the same key ([PROTOCOL_VERSIONS_KEY](https://javadoc.io/static/org.apache.stormcrawler/stormcrawler-core/3.6.0/org/apache/stormcrawler/protocol/ProtocolResponse.html#PROTOCOL_VERSIONS_KEY)) in metadata. Are there any objections to split protocol version and cipher suite over two containers? This would make the implementation easier, not requiring pattern matching to split the cipher suite value apart.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Improvement] WARC writer: WARC-Protocol header to follow WARC field proposals [stormcrawler]

["rzo1 (via GitHub)" <gi...@apache.org>]

rzo1 closed issue #1998: [Improvement] WARC writer: WARC-Protocol header to follow WARC field proposals
URL: https://github.com/apache/stormcrawler/issues/1998


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org