[GitHub] [incubator-heron] surahman edited a comment on pull request #3768: Native Python Build Rules - Phase 1

[GitBox <gi...@apache.org>]

surahman edited a comment on pull request #3768:
URL: https://github.com/apache/incubator-heron/pull/3768#issuecomment-1037680370


   > I think this is still in draft and the checklist not complete, so not sure if this is approvable yet. Just wanted to make sure it wasn't merged in by accident. But I guess the Draft status should help with that.
   
   Yes, this is still a WIP. There is a long way to go to completion and the protobuf is currently blocking progress.
   
   > I'd like to better understand the refactoring of the protobuf files to this location: https://github.com/surahman/Heron-Protobuf
   >
   > @surahman Will this remove the need for the protobuf files to live in this repo? Or was this just a temporary home to figure out the build tooling?
   
   It is to isolate the development of the protobuf build script and directory structures. It has sped up and greatly declutter the protobuf build script development. The directory structures and build scripts would need to be transplanted into the current Heron build scripts.
   
   The issues to consider are whether the import/includes and directory structure within the protobuf libraries these scripts assemble will match what Heron requires. This could become very complicated if it does not.
   
   _**Edit:**_
   
   For clarity, everything outside of the files in [this directory](https://github.com/surahman/Heron-Protobuf/tree/main/proto) is extraneous and can be classified as build artifacts. The only relevant files are in this tree and will need to be integrated into Heron:
   
   ```bash
   └── proto
       ├── api
       │   ├── BUILD.bazel
       │   └── topology.proto
       ├── ckptmgr
       │   ├── BUILD.bazel
       │   └── ckptmgr.proto
       ├── scheduler
       │   ├── BUILD.bazel
       │   └── scheduler.proto
       ├── stmgr
       │   ├── BUILD.bazel
       │   └── stmgr.proto
       ├── system
       │   ├── BUILD.bazel
       │   ├── common.proto
       │   ├── execution_state.proto
       │   ├── metrics.proto
       │   ├── packing_plan.proto
       │   ├── physical_plan.proto
       │   ├── stats.proto
       │   └── tuple.proto
       ├── testing
       │   ├── BUILD.bazel
       │   └── networktests.proto
       └── manager
           ├── BUILD.bazel
           └── tmanager.proto
   ```
   
   The protobuf is central to all communication between processes in Heron and if it is broken things will fail, so great care and scrutiny are required.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org