You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "lizhimins (via GitHub)" <gi...@apache.org> on 2026/07/22 03:52:41 UTC

[PR] [ISSUE #133] refactor: restructure to multi-module layout and harden the remoting connector [rocketmq-flink]

lizhimins opened a new pull request, #134:
URL: https://github.com/apache/rocketmq-flink/pull/134

   ## What is the purpose of the change
   
   Closes #133.
   
   Restructure the project into a multi-module layout and fix the critical issues found in review and verified against a real Aliyun commercial 5.x instance. A follow-up PR will add the gRPC (LiteSimpleConsumer) connector track on top of this layout.
   
   ## Brief changelog
   
   - Split into `flink-connector-rocketmq` + `flink-sql-connector-rocketmq`; legacy classes moved under `org.apache.flink.streaming.connectors.rocketmq`
   - Implement the no-op SQL row deserialization (SQL source previously emitted empty rows)
   - `RocketMQWriter`: close the producer, rework AT_LEAST_ONCE to async send with a pending counter and flush barrier, call `serializationSchema.open()`
   - Propagate `fetch()`/`fetchOffsets` failures instead of swallowing them
   - Consumer-side fallback for min/max/committed/timestamp offset lookup when admin broker-name resolution fails on cloud instances
   - Implement `sanityCheck()` in source/sink builders; apply ghost parameters (send timeout/retry, pull batch size, namespace, filters, scan.mode auto-infer, transaction timeout unit); wire `USE_TLS`, `MQ_CLIENT_API_TIMEOUT`, `CLIENT_CALLBACK_EXECUTOR_THREADS` and pull flow-control options; drop dead options and unused code
   - Add GitHub Actions CI (spotless:check + clean package on JDK 11/17), remove the stale `.travis.yml`; rewrite the README following the official Flink connector layout and split option references into `docs/`; add url/licenses/mailing lists to the pom
   
   ## Verifying this change
   
   - 101 unit tests pass (`mvn clean package`)
   - Manual integration tests against a real Aliyun commercial instance:
     - FLIP-27 DataStream source -> SinkV2 sink: 20/20 delivered
     - EXACTLY_ONCE two-phase commit sink: 10/10 visible after commit
     - SQL DDL end-to-end: 10/10
   - CI commands verified locally: `mvn spotless:check` and `mvn clean package` both succeed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [ISSUE #133] refactor: restructure to multi-module layout and harden the remoting connector [rocketmq-flink]

Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins merged PR #134:
URL: https://github.com/apache/rocketmq-flink/pull/134


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org