You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@eventmesh.apache.org by "qqeasonchen (via GitHub)" <gi...@apache.org> on 2026/05/06 13:42:57 UTC

[I] [Enhancement] Refactor: Unify Runtime, Ingress/Egress Pipelines, and Protocol Processor Migration (eventmesh)

qqeasonchen opened a new issue, #5249:
URL: https://github.com/apache/eventmesh/issues/5249

   ### Search before asking
   
   - [x] I had searched in the [issues](https://github.com/apache/eventmesh/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Enhancement Request
   
   ## Background
   
   EventMesh's current architecture has three separate runtime modules (`eventmesh-runtime`, `eventmesh-runtime-v2`, and various connector/function SDKs) that duplicate and fragment core processing logic. Each protocol processor (HTTP, gRPC, TCP) independently handles filtering, transformation, and routing, resulting in inconsistent behavior across protocols and high maintenance overhead. This PR consolidates these into a single unified runtime with standardized processing pipelines.
   
   
   
   ### Describe the solution you'd like
   
   ## Changes
   
   ### Commit 1: Unify Connector, Function, and Core Runtime into eventmesh-runtime
   
   - **Removed** the standalone `eventmesh-runtime-v2` module entirely (33 files, ~1700 lines deleted), including its separate runtime lifecycle, connector/function managers, health/monitor/status services, meta storage, and configuration
   - **Moved** core abstractions (`Router`, `RouterBuilder`, `ConnectorEventPublisher`, `SourceWorker`) into `eventmesh-runtime` and `eventmesh-function` modules
   - **Added** `RouterEngine`, `FilterEngine`, `TransformerEngine` as core processing engines in `eventmesh-runtime`
   - **Added** `EventMeshConnectorBootstrap` to manage connector lifecycle within the unified runtime
   - **Refactored** `EventMeshServer` to integrate engines and A2A service initialization
   - **Refactored** `ClientGroupWrapper` (TCP) to delegate to the new engines
   - **Added** comprehensive tests: `ClientGroupWrapperTest`
   
   ### Commit 2: Unified Ingress/Egress Pipelines
   
   - **Introduced** `IngressProcessor` — a centralized pipeline implementing `Filter -> Transformer -> Router` for incoming messages
   - **Introduced** `EgressProcessor` — a centralized pipeline implementing `Filter -> Transformer` for outgoing messages
   - **Refactored** `ClientGroupWrapper` (TCP SDK) to use `IngressProcessor` and `EgressProcessor`
   - **Refactored** `EventMeshConnectorBootstrap` (Connectors) to use the new processors
   - **Updated** `SinkWorker` to support embedded mode for unified runtime execution
   - **Updated** documentation (`unified-runtime-design.md`)
   
   ### Commit 3: Complete HTTP and gRPC Processor Migration
   
   - **HTTP Processors (5)**: Migrated `SendAsyncMessageProcessor`, `SendSyncMessageProcessor`, `BatchSendMessageProcessor`, `BatchSendMessageV2Processor`, and `SendAsyncEventProcessor` to use `IngressProcessor`
   - **gRPC Processors (3)**: Migrated `PublishCloudEventsProcessor`, `BatchPublishCloudEventProcessor`, and `RequestCloudEventProcessor` with bidirectional pipeline support (`Ingress` for requests, `Egress` for responses)
   - **Added** `BatchProcessResult` — utility class for tracking success/filtered/failed counts with detailed per-message statistics
   - **Added** cross-module accessors (`IngressProcessor`, `EgressProcessor` getters in `EventMeshServer` and `EventMeshGrpcServer`)
   - **Unified** pipeline key format: `{producerGroup}-{topic}`
   - **Standardized** filter behavior: filtered messages return `SUCCESS` status (except request-reply which returns error)
   - **Added** comprehensive tests: `IngressProcessorTest`, `EgressProcessorTest`, `BatchProcessResultTest`, enhanced `SendAsyncEventProcessorTest`
   
   ## Key Files
   
   | File | Action |
   |------|--------|
   | `eventmesh-runtime-v2/` (entire module) | **Deleted** |
   | `eventmesh-runtime/src/main/java/.../core/protocol/IngressProcessor.java` | **Added** |
   | `eventmesh-runtime/src/main/java/.../core/protocol/EgressProcessor.java` | **Added** |
   | `eventmesh-runtime/src/main/java/.../core/protocol/BatchProcessResult.java` | **Added** |
   | `eventmesh-runtime/src/main/java/.../boot/RouterEngine.java` | **Added** |
   | `eventmesh-runtime/src/main/java/.../boot/EventMeshConnectorBootstrap.java` | **Added** |
   | `eventmesh-runtime/src/main/java/.../boot/EventMeshServer.java` | **Modified** |
   | `eventmesh-runtime/src/main/java/.../boot/FilterEngine.java` | **Modified** |
   | `eventmesh-runtime/src/main/java/.../boot/TransformerEngine.java` | **Modified** |
   | All HTTP gRPC processors in `.../protocol/http/processor/` and `.../protocol/grpc/processor/` | **Modified** |
   
   ## Impact
   
   - **No functional regression** — all existing tests pass
   - Consistent `Filter -> Transformer -> Router` pipeline across all three protocols (TCP, HTTP, gRPC)
   - Simplified build and module structure with single `eventmesh-runtime` module
   - Centralized batch statistics and error handling
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) *


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org


Re: [I] [Enhancement] Refactor: Unify Runtime, Ingress/Egress Pipelines, and Protocol Processor Migration (eventmesh)

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5249:
URL: https://github.com/apache/eventmesh/issues/5249#issuecomment-5183699007

   It has been 90 days since the last activity on this issue. Apache EventMesh values the voices of the community. Please don't hesitate to share your latest insights on this matter at any time, as the community is more than willing to engage in discussions regarding the development and optimization directions of this feature.
   
   If you feel that your issue has been resolved, please feel free to close it. Should you have any additional information to share, you are welcome to reopen this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org


Re: [I] [Enhancement] Refactor: Unify Runtime, Ingress/Egress Pipelines, and Protocol Processor Migration (eventmesh)

Posted by "qqeasonchen (via GitHub)" <gi...@apache.org>.
qqeasonchen closed issue #5249: [Enhancement] Refactor: Unify Runtime, Ingress/Egress Pipelines, and Protocol Processor Migration
URL: https://github.com/apache/eventmesh/issues/5249


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org