You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "svetakvsundhar (via GitHub)" <gi...@apache.org> on 2023/05/21 20:53:46 UTC
[GitHub] [beam] svetakvsundhar opened a new issue, #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
svetakvsundhar opened a new issue, #26808:
URL: https://github.com/apache/beam/issues/26808
### What needs to happen?
#26604 added an integration test script to a new folder `sdks/python/apache_beam/testing/notebooks/`, while current notebooks are all under `beam/examples/notebooks/` (https://github.com/apache/beam/tree/master/examples/notebooks).
It would make more sense to move the test script here, to allow for the `.spec` files, the `.ipynb` notebook, and the test scripts to all be in the same folder.
The other (less preferable) alternative, is to add a copy of each notebook under `sdks/python/apache_beam/testing/notebooks/`. This would require everyone checking in notebooks to check them into `beam/examples/notebooks` as well as `sdks/python/apache_beam/testing/notebooks/`, which could eventually lead to inconsistency issues between the two.
### Issue Priority
Priority: 3 (nice-to-have improvement)
### Issue Components
- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [X] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] svetakvsundhar commented on issue #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
Posted by "svetakvsundhar (via GitHub)" <gi...@apache.org>.
svetakvsundhar commented on issue #26808:
URL: https://github.com/apache/beam/issues/26808#issuecomment-1557492532
> I don't understand why this is true; do the notebooks have to be in the same folder in order to run tests on them? That seems like an fixable challenge. Regardless, I'm still +1 on the idea.
No they don't, what I am intending to convey is that it'll be a cleaner experience for the repo, as well as folks checking in notebooks to have it all under one folder.
> Nit: I assume the title should read "Migrate `sdks/python/apache_beam/testing/**notebooks/**` ...", right?
Yes.
Sounds like we are in agreement here -- I'll put up a PR shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm closed issue #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
URL: https://github.com/apache/beam/issues/26808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] liferoad commented on issue #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #26808:
URL: https://github.com/apache/beam/issues/26808#issuecomment-1556301769
I don't think we can cover all notebooks under examples without changing these notebooks. You probably need to look at the most popular workflow and only add these as notebook tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] svetakvsundhar commented on issue #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
Posted by "svetakvsundhar (via GitHub)" <gi...@apache.org>.
svetakvsundhar commented on issue #26808:
URL: https://github.com/apache/beam/issues/26808#issuecomment-1556289615
WDYT @damccorm @liferoad ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #26808: [Task]: Migrate `sdks/python/apache_beam/testing/` to `beam/examples/notebooks`.
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #26808:
URL: https://github.com/apache/beam/issues/26808#issuecomment-1557479127
I think moving the test script to the notebooks folder makes sense. That should make for a more reasonable developer workflow (run `pytest` from the same directory you added the notebook).
> The other (less preferable) alternative, is to add a copy of each notebook under sdks/python/apache_beam/testing/notebooks/. This would require everyone checking in notebooks to check them into beam/examples/notebooks as well as sdks/python/apache_beam/testing/notebooks/, which could eventually lead to inconsistency issues between the two.
I don't understand why this is true; do the notebooks have to be in the same folder in order to run tests on them? That seems like an fixable challenge. Regardless, I'm still +1 on the idea.
Nit: I assume the title should read "Migrate `sdks/python/apache_beam/testing/**notebooks/**` ...", right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org