[GitHub] [shardingsphere] azexcy commented on a diff in pull request #25787: Refactor pipeline channel, read records by transaction at increment task

["azexcy (via GitHub)" <gi...@apache.org>]

azexcy commented on code in PR #25787:
URL: https://github.com/apache/shardingsphere/pull/25787#discussion_r1198666166


##########
kernel/data-pipeline/core/src/main/java/org/apache/shardingsphere/data/pipeline/core/ingest/channel/memory/SimpleMemoryPipelineChannel.java:
##########
@@ -54,13 +54,22 @@ public void pushRecord(final Record dataRecord) {
     public List<Record> fetchRecords(final int batchSize, final int timeout, final TimeUnit timeUnit) {
         List<Record> result = new ArrayList<>(batchSize);
         long start = System.currentTimeMillis();
-        while (batchSize > queue.size()) {
+        int recordsCount = 0;
+        while (batchSize > recordsCount) {
+            List<Record> records = queue.poll();
+            if (null == records || records.isEmpty()) {
+                TimeUnit.MILLISECONDS.sleep(100L);

Review Comment:
   Yes, queue `poll` may return null 
   ```
       /**
        * Retrieves and removes the head of this queue,
        * or returns {@code null} if this queue is empty.
        *
        * @return the head of this queue, or {@code null} if this queue is empty
        */
       E poll();
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org