You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "Wan95u (via GitHub)" <gi...@apache.org> on 2026/01/19 07:31:46 UTC

[I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Wan95u opened a new issue, #10365:
URL: https://github.com/apache/seatunnel/issues/10365

   ### Search before asking
   
   - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
   
   
   ### What happened
   
   Reading data from a MySQL table and writing it into an Excel file on an SFTP server results in incorrect column order in the Excel file: the sequence does not match the order defined in sink_columns. 
   <img width="2919" height="278" alt="Image" src="https://github.com/user-attachments/assets/9713db7b-882f-4c47-80ce-2ea9170934d9" />
   
   <img width="3819" height="1995" alt="Image" src="https://github.com/user-attachments/assets/ca549c1f-634c-42f2-8904-7de3853c909a" />
   
   ### SeaTunnel Version
   
   2.3.2、2.3.8
   
   ### SeaTunnel Config
   
   ```conf
   {
     "env": {
       "execution.parallelism": 2,
       "job.mode": "BATCH"
     },
     "source": [
       {
         "password": "***",
         "database": "dataos_dev",
         "driver": "com.mysql.jdbc.Driver",
         "query": "select company_name,social_code,fd_1,fd_2,fd_3,fd_4,fd_5,fd_6,fd_7,fd_8,fd_9,fd_10,fd_138 from test_table_tt ",
         "connection_check_timeout_sec": 100,
         "result_table_name": "jdbc1",
         "plugin_name": "Jdbc",
         "user": "dataos",
         "url": "jdbc:mysql://***/dataos_dev?serverTimezone=GMT%2b8"
       }
     ],
     "sink": [
       {
         "custom_filename": true,
         "file_format_type": "excel",
         "file_name_expression": "ftp40_pu.csv_${transactionId}",
         "host": "*****",
         "is_enable_transaction": true,
         "password": "***",
         "path": "/data/dataos/test/out",
         "plugin_name": "SftpFile",
         "port": "22022",
         "sink_columns": [
           "company_name",
           "social_code",
           "fd_1",
           "fd_2",
           "fd_3",
           "fd_4",
           "fd_5",
           "fd_6",
           "fd_7",
           "fd_8",
           "fd_9",
           "fd_10",
           "fd_138"
         ],
         "source_table_name": [
           "FtpFile_source_4"
         ],
         "user": "puaiuc"
       }
     ]
   }
   ```
   
   ### Running Command
   
   ```shell
   ./bin/seatunnel.sh -c ./job/mysql2sftp.json
   ```
   
   ### Error Exception
   
   ```log
   Program completed successfully
   ```
   
   ### Zeta or Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   _No response_
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo commented on issue #10365:
URL: https://github.com/apache/seatunnel/issues/10365#issuecomment-6376353972

   Closing this issue as completed after rechecking the original report and its resolution.
   
   Excel output now creates header and data cells at sequential output-column positions while reading values through the configured `sink_columns` source indexes. This fixes the incorrect Excel column order in this report.
   
   Resolution evidence:
   
   - [PR #10366](https://github.com/apache/seatunnel/pull/10366), merged into `dev` on 2026-04-21 ([merge commit `0bc6b3406ae2`](https://github.com/apache/seatunnel/commit/0bc6b3406ae2a42b5b511cf18da1d581332ac047)).
   
   The corresponding implementation remains present in [`dev` at `14a27ac26a0a`](https://github.com/apache/seatunnel/commit/14a27ac26a0a71b56722f86cdf8ec2779094ceaf).
   
   <!-- seatunnel-closure-audit:20261023:10365 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "Wan95u (via GitHub)" <gi...@apache.org>.
Wan95u closed issue #10365: [Bug] [Connector-file] File connectors writing to Excel files are out of order
URL: https://github.com/apache/seatunnel/issues/10365


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "Wan95u (via GitHub)" <gi...@apache.org>.
Wan95u commented on issue #10365:
URL: https://github.com/apache/seatunnel/issues/10365#issuecomment-3767448384

   > 感谢你举报。我已经确认这个bug存在于Excel连接器里。
   > 
   > **根源**:错误地使用Excel的原始字段索引作为列号,而非顺序列位置(0,1,2,3......)。例如,如果映射到索引,Excel会将列写在0,12,2,3,4的位置,而不是0,1,2,3,4。`ExcelGenerator.java``sinkColumnsIndexInRow``sink_columns``[0,12,2,3,4...]`
   > 
   > **受影响代码**:
   > 
   > * `seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/util/ExcelGenerator.java`(第76-80行,97-100行,120-121行)
   > 
   > 当 时,这会影响所有基于文件的接收器(SFTP/HDFS/本地/OSS)。其他格式(CSV/JSON/ORC)通过 正确重排列。`file_format_type=excel``seaTunnelRow.copy()`
   > 
   > **建议的解决**办法:在头部和数据写入循环中都替换为 。在 中加入带有非顺序索引的测试用例(例如 )。`createCell(i)``createCell(colIndex++)``[3,0,2]``ExcelGeneratorTest.java`
   > 
   > **变通**方法:使用CSV格式作为输出,这样可以正确处理列的排序。
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "DanielCarter-stack (via GitHub)" <gi...@apache.org>.
DanielCarter-stack commented on issue #10365:
URL: https://github.com/apache/seatunnel/issues/10365#issuecomment-3767152035

   <!-- code-pr-reviewer -->
   Thank you for reporting this. I've confirmed the bug exists in the Excel connector.
   
   **Root Cause**: `ExcelGenerator.java` incorrectly uses original field indices from `sinkColumnsIndexInRow` as Excel column numbers instead of sequential column positions (0,1,2,3...). For example, if `sink_columns` maps to indices `[0,12,2,3,4...]`, Excel writes columns at positions 0,12,2,3,4 instead of 0,1,2,3,4.
   
   **Affected Code**:
   - `seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/util/ExcelGenerator.java` (lines 76-80, 97-100, 120-121)
   
   This affects all file-based sinks (SFTP/HDFS/Local/OSS) when `file_format_type=excel`. Other formats (CSV/JSON/ORC) correctly reorder columns via `seaTunnelRow.copy()`.
   
   **Suggested Fix**: Replace `createCell(i)` with `createCell(colIndex++)` in both header and data writing loops. Add test case with non-sequential indices (e.g., `[3,0,2]`) in `ExcelGeneratorTest.java`.
   
   **Workaround**: Use CSV format as output instead, which handles column ordering correctly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "zhangshenghang (via GitHub)" <gi...@apache.org>.
zhangshenghang commented on issue #10365:
URL: https://github.com/apache/seatunnel/issues/10365#issuecomment-3768312985

   thanks @Wan95u 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "Wan95u (via GitHub)" <gi...@apache.org>.
Wan95u commented on issue #10365:
URL: https://github.com/apache/seatunnel/issues/10365#issuecomment-3767457942

   ty,I'll fix try to this bug.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] [Connector-file] File connectors writing to Excel files are out of order [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo closed issue #10365: [Bug] [Connector-file] File connectors writing to Excel files are out of order
URL: https://github.com/apache/seatunnel/issues/10365


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org