You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "liferoad (via GitHub)" <gi...@apache.org> on 2025/04/06 14:59:36 UTC

[I] [Task]: Consider reducing dependencies in core [beam]

liferoad opened a new issue, #34554:
URL: https://github.com/apache/beam/issues/34554

   ### What needs to happen?
   
   - Proposal to reduce reducing dependencies in core, e.g., separate/extra packages
   - Implement these changes
   
   ### Issue Priority
   
   Priority: 2 (default / most normal work should be filed as P2)
   
   ### Issue Components
   
   - [x] Component: Python SDK
   - [x] Component: Java SDK
   - [x] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam YAML
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Infrastructure
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Task]: Consider reducing dependencies in core [beam]

Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #34554:
URL: https://github.com/apache/beam/issues/34554#issuecomment-3255519462

   Roughly what I think we should do for Python/Go/TypeScript (haven't gotten to Java yet)
   
   **Python:**
   
   Split the following dependencies out of core [required deps in setup.py](https://github.com/apache/beam/blob/626be06a47b60ae48c5fa20500221f04dc3da287/sdks/python/setup.py#L360) and into extras. Some extras already exist, others need to be added
   
   - TFRecord extra: `crcmod`, ``
   - GCP: `orjson`, `httplib2`, `regex`
   - Hadoop: `hdfs`
   - Interactive: `pydot`
   - Mongo: `pymongo`
   - Redis: `redis` (used broadly by enrichment, but has caused folks dependency version problems)
   - Yaml: `jsonschema`
   
   **Go/Typescript:**
   
   N/A - I don't think we need to do anything here


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Task]: Consider reducing dependencies in core [beam]

Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #34554: [Task]: Consider reducing dependencies in core
URL: https://github.com/apache/beam/issues/34554


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Task]: Consider reducing dependencies in core [beam]

Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #34554:
URL: https://github.com/apache/beam/issues/34554#issuecomment-2818447678

   Possible to move the legacy jar code to a seperate repo?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Task]: Consider reducing dependencies in core [beam]

Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #34554:
URL: https://github.com/apache/beam/issues/34554#issuecomment-2816100378

   some discussions: https://docs.google.com/document/d/1KfTMok67-bZBUcCCYeHkdktyQZ6arWQJEPMc-4hUrX0/edit?disco=AAABhaO_SV4


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Task]: Consider reducing dependencies in core [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #34554:
URL: https://github.com/apache/beam/issues/34554#issuecomment-3249997636

   This Issue is marked as Beam 3 - Milestone 1: No Code and ML focus. Please help us burning down the open blocker tasks.
   
   * If this issue is solved, close it. If it is obsolete, close as not planned. If it is no longer necessary for Beam 3, move it out of the Milestone.
   
   * If the work is pending, please be reminded that Milestone 1 is targeted for completion by Beam 2.72 cut (Feb 18, 2026)
   
   * If this issue involves multiple milestones (e.g. remove a public API should first mark it as deprecated, scoped in Milestone 1; then finally remove it in Beam 3 release candidate, scoped in Milestone 3) and Milestone 1 portion has completed, consider move the Issue to Milestone 3.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org