[GitHub] [iceberg] geonyeongkim commented on issue #7568: Flink DataStream Small file Issue And RewriteDataFiles Action

["geonyeongkim (via GitHub)" <gi...@apache.org>]

geonyeongkim commented on issue #7568:
URL: https://github.com/apache/iceberg/issues/7568#issuecomment-1543799584

   @Fokko 
   I pre-generated a table with Hadoop catalog using spark sql.
   
   Below is my spark sql code.
   
   ```python
   spark.sql("""
   CREATE TABLE `iceberg`.`cpdep_tacct_l`.`cow` (
   `ACNO` STRING,
   `TX_DT` STRING,
   `TX_SEQNO` STRING,
   `DTL_TX_SEQNO` STRING,
   `FRMW_CHNG_TMST` STRING,
   `TX_GUID` STRING,
   `SLIP_NO` STRING,
   `DEP_TX_STCD` STRING,
   `DEP_CANC_DVCD` STRING,
   `TX_TIME` STRING,
   `RECKN_DT` STRING,
   `SYS_DT` STRING,
   `SYS_TIME` STRING,
   `DEP_TRRC_KNCD` STRING,
   `TX_CHNL_DVCD` STRING,
   `TX_FRMN_CD` STRING,
   `TX_CD` STRING,
   `TX_BRCD` STRING,
   `TX_TLRNO` STRING,
   `ACCO_MGMT_BRCD` STRING,
   `XN_PB_DVCD` STRING,
   `CURN_CD` STRING,
   `ACCO_SBCD` STRING,
   `GDS_CD` STRING,
   `AFTR_BAL_CHNG_TX_YN` STRING,
   `TX_AMT` STRING,
   `AFTR_BAL` STRING,
   `CASH_AMT` STRING,
   `SMPL_TRFAMT` STRING,
   `TOT_CBPC_AMT` STRING,
   `ITLK_ALTR_AMT` STRING,
   `CSCHQ_ISSU_AMT` STRING,
   `CSCHQ_PRVS_AMT` STRING,
   `OJBB_MNRC_AMT` STRING,
   `OBNK_MNRC_AMT` STRING,
   `DEP_TRRC_CBPC_CD` STRING,
   `DRAF_CHQ_NO` STRING,
   `CBPC_AMT` STRING,
   `FEE_PRFR_RSCD` STRING,
   `DEP_FEE_RCV_DVCD` STRING,
   `FEE_TX_SEQNO` STRING,
   `INSTL_MM` STRING,
   `INSTL_NTH` STRING,
   `OVRD_PPAY_DAYS` STRING,
   `CNGT_MSG_CD` STRING,
   `BBPR_CTNT` STRING,
   `PB_ARR_TRGT_YN` STRING,
   `PSWD_NSTUP_PRVS_YN` STRING,
   `NPSWD_RSCD` STRING,
   `ACCO_UZ_DVCD` STRING,
   `INPUT_ACNO` STRING,
   `BTCH_NO` STRING,
   `AFCL_TX_YN` STRING,
   `PRCG_APRV_TLRNO` STRING,
   `ITLK_TX_DVCD` STRING,
   `DEP_TRRC_TYCD` STRING,
   `RBF_SRVC_ID` STRING,
   `REAL_ACCP_AMT` STRING,
   `INT_CCAM` STRING,
   `UNPAID_BAL` STRING,
   `FRUP_BAL_TX_DT` STRING,
   `BNCF_TX_ANNT` STRING,
   `BNCF_AFTR_ANNT` STRING,
   `SLE_PAY_AMT` STRING,
   `MDCL_FEE` STRING,
   `BNLN_INT` STRING,
   `FX_ITLK_TX_NO` STRING,
   `FX_PRFR_XRT_APPC_NO` STRING,
   `FX_APLY_XRT` STRING,
   `FX_XRT_ANOUN_NTH` STRING,
   `BOK_RPT_RSCD` STRING,
   `EXPRT_CPT_YN` STRING,
   `WCUC_AMT` STRING,
   `WCUC_SMPL_TRFAMT` STRING,
   `WCUC_ITLK_TRFAMT` STRING,
   `WCUC_CASH_AMT` STRING,
   `CNTR_MOVE_FRST_PYMN_YN` STRING,
   `SCFC_TCFND_BZWK_DVCD` STRING,
   `SCFC_TCFND_MESG_ID` STRING,
   `SCFC_TCFND_PL_INT` STRING,
   `MNRC_RQER_RLNM_DVCD` STRING,
   `MNRC_RQER_RNNO` STRING,
   `MNRC_RQER_NM` STRING,
   `CMS_CD` STRING,
   `CMS_RCPR_NM` STRING,
   `ORTR_DT` STRING,
   `ORTR_SEQNO` STRING,
   `INT_CLCL_DTL_HIST_CRTN_YN` STRING,
   `SEIZ_DMAN_MGMT_NO` STRING,
   `AIQRY_CSCHQ_AMT` STRING,
   `OSDCH_TX_UNQ_NO` STRING,
   `BANK_GIRCD` STRING,
   `GDS_DTLS_CD` STRING,
   `WCUC_DEAL_PFLS_AMT` STRING,
   `WCUC_ANTT_AMT` STRING,
   `DMAN_NO` STRING,
   `TX_RSN` STRING,
   `FUND_ITMS_CD` STRING,
   `MDCL_CTRINT` STRING,
   `ITLK_TX_LDNG_YN` STRING,
   `FRST_TRNM_IPAD` STRING,
   `GUID` STRING,
   `SYS_FRST_REG_DTTM` STRING,
   `SYS_FRST_REG_EMPNO` STRING,
   `SYS_LAST_CHNG_DTTM` STRING,
   `SYS_LAST_CHNG_EMPNO` STRING,
   `TRSF_BANK_CD` STRING,
   `TRSF_ACNO` STRING,
   `TRSF_ACCO_DEPR_NM` STRING
   )
   USING iceberg
   TBLPROPERTIES (
       'format-version'='2',
       'write.delete.mode'='copy-on-write',
       'write.update.mode'='copy-on-write',
       'write.merge.mode'='copy-on-write',
       'write.metadata.delete-after-commit.enabled'='true',
       'write.metadata.previous-versions-max'='10'
   )
   """)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org