You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/25 10:30:24 UTC
[GitHub] [hudi] adward32135 opened a new issue, #7058: [SUPPORT]问题发生在T+1表插入到huidi表中,字段不一致导致数据插不进去
adward32135 opened a new issue, #7058:
URL: https://github.com/apache/hudi/issues/7058
请求修复 ，支持一下。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] adward32135 closed issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
adward32135 closed issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
URL: https://github.com/apache/hudi/issues/7058
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] adward32135 commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
adward32135 commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291644209
> @adward32135 Hello, can you provide the table structure in Hive metadata? I suspect 46 fields are stored in the metadata. In addition, what is your Spark version
The hive metadata as follows and Spark version is spark3.1
CREATE TABLE T2(
stat_date STRING,
id BIGINT COMMENT '-',
cid BIGINT COMMENT '-',
shard STRING COMMENT '-',
productcode STRING COMMENT '-',
subproductcode STRING COMMENT '-',
serialid STRING COMMENT '',
uin BIGINT COMMENT '-',
appid BIGINT COMMENT '-',
payeruin BIGINT COMMENT '',
payerappid BIGINT COMMENT '-',
platform BIGINT COMMENT '',
projectid BIGINT COMMENT '',
regionid BIGINT COMMENT '',
zoneid BIGINT COMMENT '',
paymode BIGINT COMMENT '',
pid BIGINT COMMENT '-',
actid BIGINT COMMENT '',
calcdate STRING COMMENT '',
starttime STRING COMMENT '',
endtime STRING COMMENT '',
calcnum STRING COMMENT '',
timespan STRING COMMENT '',
calcstatus BIGINT COMMENT '',
calcchannel BIGINT COMMENT '',
datastatus BIGINT COMMENT '',
settletype BIGINT COMMENT '',
params STRING COMMENT '',
parts STRING COMMENT '',
billingparams STRING COMMENT '',
reduceinfo STRING COMMENT '',
currency STRING COMMENT '',
price DOUBLE COMMENT '',
money DOUBLE COMMENT '',
totalcost DOUBLE COMMENT '',
taxrate DOUBLE COMMENT '-',
taxamount DOUBLE COMMENT '-',
orderid STRING COMMENT '-',
feebillid STRING COMMENT '',
addtime STRING COMMENT '',
modifytime STRING COMMENT '',
errormsg STRING COMMENT '',
riorderid STRING COMMENT ''
)
PARTITION BY LIST( stat_date )
(
PARTITION p_20221022 VALUES IN ( '20221022' )
)
STORED AS ORCFILE COMPRESS;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] dongkelun commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
dongkelun commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291649927
> CREATE TABLE T2(
> stat_date STRING,
> id BIGINT COMMENT '-',
> cid BIGINT COMMENT '-',
> shard STRING COMMENT '-',
> productcode STRING COMMENT '-',
> subproductcode STRING COMMENT '-',
> serialid STRING COMMENT '',
> uin BIGINT COMMENT '-',
> appid BIGINT COMMENT '-',
> payeruin BIGINT COMMENT '',
> payerappid BIGINT COMMENT '-',
> platform BIGINT COMMENT '',
> projectid BIGINT COMMENT '',
> regionid BIGINT COMMENT '',
> zoneid BIGINT COMMENT '',
> paymode BIGINT COMMENT '',
> pid BIGINT COMMENT '-',
> actid BIGINT COMMENT '',
> calcdate STRING COMMENT '',
> starttime STRING COMMENT '',
> endtime STRING COMMENT '',
> calcnum STRING COMMENT '',
> timespan STRING COMMENT '',
> calcstatus BIGINT COMMENT '',
> calcchannel BIGINT COMMENT '',
> datastatus BIGINT COMMENT '',
> settletype BIGINT COMMENT '',
> params STRING COMMENT '',
> parts STRING COMMENT '',
> billingparams STRING COMMENT '',
> reduceinfo STRING COMMENT '',
> currency STRING COMMENT '',
> price DOUBLE COMMENT '',
> money DOUBLE COMMENT '',
> totalcost DOUBLE COMMENT '',
> taxrate DOUBLE COMMENT '-',
> taxamount DOUBLE COMMENT '-',
> orderid STRING COMMENT '-',
> feebillid STRING COMMENT '',
> addtime STRING COMMENT '',
> modifytime STRING COMMENT '',
> errormsg STRING COMMENT '',
> riorderid STRING COMMENT ''
> )
> PARTITION BY LIST( stat_date )
> (
> PARTITION p_20221022 VALUES IN ( '20221022' )
> )
> STORED AS ORCFILE COMPRESS;
@dongkelun Sorry, I didn't make it clear. I want to see the table structure of Table T1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] adward32135 commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
adward32135 commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291751216
CREATE TABLE T1
(
ftime STRING,
`id` BIGINT,
`cid` INT,
`shard` STRING,
`productCode` STRING,
`subProductCode` STRING,
`serialId` STRING,
`uin` BIGINT ,
`appId` INT,
`payerUin` BIGINT ,
`payerAppId` INT,
`platform` INT,
`projectId` INT,
`regionId` INT,
`zoneId` INT,
`payMode` INT ,
`pid` INT,
`actId` INT,
`calcDate` STRING ,
`startTime` STRING,
`endTime` STRING,
`calcNum` STRING,
`timeSpan` STRING,
`calcStatus` INT,
`calcChannel` INT,
`dataStatus` INT,
`settleType` INT,
`params` STRING,
`parts` STRING,
`billingParams` STRING,
`reduceInfo` STRING,
`currency` STRING,
`price` DOUBLE,
`money` DOUBLE,
`totalCost` DOUBLE,
`taxRate` DOUBLE,
`taxAmount` DOUBLE,
`orderId` STRING,
`feeBillId` STRING,
`addTime` STRING,
`modifyTime` STRING,
`errorMsg` STRING,
`riOrderId` STRING,
`timeUnit` STRING,
record_time TIMESTAMP(3),
PRIMARY KEY (`id`, `shard`) NOT ENFORCED
)
PARTITIONED BY (`ftime`)
WITH (
'connector' = 'hudi',
'table.type' = 'MERGE_ON_READ' ,
'write.operation' = 'upsert',
'write.bucket_assign.tasks' = '20',
'write.tasks' = '20',
'hoodie.logfile.to.parquet.compression.ratio'='0.5',
'changelog.enabled' = 'true',
'read.streaming.enabled' = 'true',
'read.streaming.check-interval' = '10',
'compaction.schedule.enabled' = 'true',
'compaction.async.enabled' = 'true',
'compaction.delta_seconds' = '600',
'compaction.trigger.strategy' = 'num_or_time',
'compaction.delta_commits' = '30',
'compaction.tasks'='8',
'compaction.max_memory' = '4096',
'clean.retain_commits' = '30',
'hive_sync.enable' = 'true',
'hive_sync.mode' = 'hms',
'index.type' = 'BUCKET',
'hoodie.bucket.index.num.buckets' = '100',
'hive_sync.metastore.uris' = 'thrift://ss-qe-nginx-tauth.tencent-distribute.com:8106',
'path' = 'hdfs://qy-pcg-11-v3/user/tdw/warehouse/xxx.db/T1',
'hive_sync.db' = 'xxx',
'hive_sync.table' = 'T1' ,
'hoodie.datasource.write.recordkey.field' = 'id,shard',
'write.precombine.field' = 'addTime'
)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1306527029
Hello, can we close this issue though @adward32135 , did you try the option then ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] dongkelun commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
dongkelun commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1290445528
> @adward32135 Hello, can you provide more detailed exception information, and what is your hudi version?
@adward32135 In addition, there are only four fields in your example. Can you provide the exception information corresponding to the example
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] adward32135 commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
adward32135 commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291413206
> @adward32135 In addition, there are only four fields in your example. Can you provide the exception information corresponding to the example
CREATE TABLE T1
(
ftime STRING,
`id` BIGINT,
`cid` INT,
`shard` STRING,
`productCode` STRING,
`subProductCode` STRING,
`serialId` STRING,
`uin` BIGINT ,
`appId` INT,
`payerUin` BIGINT ,
`payerAppId` INT,
`platform` INT,
`projectId` INT,
`regionId` INT,
`zoneId` INT,
`payMode` INT ,
`pid` INT,
`actId` INT,
`calcDate` STRING ,
`startTime` STRING,
`endTime` STRING,
`calcNum` STRING,
`timeSpan` STRING,
`calcStatus` INT,
`calcChannel` INT,
`dataStatus` INT,
`settleType` INT,
`params` STRING,
`parts` STRING,
`billingParams` STRING,
`reduceInfo` STRING,
`currency` STRING,
`price` DOUBLE,
`money` DOUBLE,
`totalCost` DOUBLE,
`taxRate` DOUBLE,
`taxAmount` DOUBLE,
`orderId` STRING,
`feeBillId` STRING,
`addTime` STRING,
`modifyTime` STRING,
`errorMsg` STRING,
`riOrderId` STRING,
`timeUnit` STRING,
record_time TIMESTAMP(3),
PRIMARY KEY (`id`, `shard`) NOT ENFORCED
)
PARTITIONED BY (`ftime`)
WITH (
'connector' = 'hudi',
'table.type' = 'MERGE_ON_READ' ,
'write.operation' = 'upsert',
'write.bucket_assign.tasks' = '20',
'write.tasks' = '20',
'hoodie.logfile.to.parquet.compression.ratio'='0.5',
'changelog.enabled' = 'true',
'read.streaming.enabled' = 'true',
'read.streaming.check-interval' = '10',
'compaction.schedule.enabled' = 'true',
'compaction.async.enabled' = 'true',
'compaction.delta_seconds' = '600',
'compaction.trigger.strategy' = 'num_or_time',
'compaction.delta_commits' = '30',
'compaction.tasks'='8',
'compaction.max_memory' = '4096',
'clean.retain_commits' = '30',
'hive_sync.enable' = 'true',
'hive_sync.mode' = 'hms',
'index.type' = 'BUCKET',
'hoodie.bucket.index.num.buckets' = '50',
'hive_sync.metastore.uris' = 'xxx',
'path' = 'hdfs://xxx/xx.db/T1',
'hive_sync.db' = 'xxx',
'hive_sync.table' = 'T1' ,
'hoodie.datasource.write.recordkey.field' = 'id,shard',
'write.precombine.field' = 'addTime'
)
hive T+1 table T2
Use sparksql to insert data of a certain partition into the hudi table The statement is as follows:
INSERT INTO T1 PARTITION (ftime = '20221005')
select
id,
cid,
shard,
productcode,
subproductcode,
serialid,
uin,
appid,
payeruin,
payerappid,
platform,
projectid,
regionid,
zoneid,
paymode,
pid,
actid,
calcdate,
starttime,
endtime,
calcnum,
timespan,
calcstatus,
calcchannel,
datastatus,
settletype,
params,
parts,
billingparams,
reduceinfo,
currency,
price,
money,
totalcost,
taxrate,
taxamount,
orderid,
feebillid,
addtime,
modifytime,
errormsg,
riorderid,
'',
0
from
T2 where stat_date = '20221005'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1299495331
I have encountered similar error logs when I query hive, you can set ```hoodie.datasource.write.drop.partition.columns=true```, and try again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] dongkelun commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
dongkelun commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1290430336
@adward32135 Hello, can you provide more detailed exception information, and what is your hudi version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] dongkelun commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
dongkelun commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1314868671
We need to persist the ALLOW_OPERATION_METADATA_FIELD configuration item to the file in Flink and Spark to make Spark and Spark unified. I have submitted the pr on the Spark side
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] dongkelun commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
dongkelun commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291649844
> CREATE TABLE T2(
> stat_date STRING,
> id BIGINT COMMENT '-',
> cid BIGINT COMMENT '-',
> shard STRING COMMENT '-',
> productcode STRING COMMENT '-',
> subproductcode STRING COMMENT '-',
> serialid STRING COMMENT '',
> uin BIGINT COMMENT '-',
> appid BIGINT COMMENT '-',
> payeruin BIGINT COMMENT '',
> payerappid BIGINT COMMENT '-',
> platform BIGINT COMMENT '',
> projectid BIGINT COMMENT '',
> regionid BIGINT COMMENT '',
> zoneid BIGINT COMMENT '',
> paymode BIGINT COMMENT '',
> pid BIGINT COMMENT '-',
> actid BIGINT COMMENT '',
> calcdate STRING COMMENT '',
> starttime STRING COMMENT '',
> endtime STRING COMMENT '',
> calcnum STRING COMMENT '',
> timespan STRING COMMENT '',
> calcstatus BIGINT COMMENT '',
> calcchannel BIGINT COMMENT '',
> datastatus BIGINT COMMENT '',
> settletype BIGINT COMMENT '',
> params STRING COMMENT '',
> parts STRING COMMENT '',
> billingparams STRING COMMENT '',
> reduceinfo STRING COMMENT '',
> currency STRING COMMENT '',
> price DOUBLE COMMENT '',
> money DOUBLE COMMENT '',
> totalcost DOUBLE COMMENT '',
> taxrate DOUBLE COMMENT '-',
> taxamount DOUBLE COMMENT '-',
> orderid STRING COMMENT '-',
> feebillid STRING COMMENT '',
> addtime STRING COMMENT '',
> modifytime STRING COMMENT '',
> errormsg STRING COMMENT '',
> riorderid STRING COMMENT ''
> )
> PARTITION BY LIST( stat_date )
> (
> PARTITION p_20221022 VALUES IN ( '20221022' )
> )
> STORED AS ORCFILE COMPRESS;
@dongkelun Sorry, I didn't make it clear. I want to see the table structure of Table T1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] adward32135 commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
adward32135 commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291401038
> @adward32135 Hello, can you provide more detailed exception information, and what is your hudi version?
hudi version：0.11.x和0.12.0
detail exception:
connectionId = 73bb34bf-8f57-4bc0-a1c6-69788b23da1a, statementId = 0 -> SQLException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns: (id,cid,shard,productcode,subproductcode,serialid,uin,appid,payeruin,payerappid,platform,projectid,regionid,zoneid,paymode,pid,actid,calcdate,starttime,endtime,calcnum,timespan,calcstatus,calcchannel,datastatus,settletype,params,parts,billingparams,reduceinfo,currency,price,money,totalcost,taxrate,taxamount,orderid,feebillid,addtime,modifytime,errormsg,riorderid,,0,ftime)
at scala.Predef$.assert(Predef.scala:223)
at org.apache.spark.sql.hudi.command.InsertIntoHoodieTableCommand$.alignOutputFields(InsertIntoHoodieTableCommand.scala:133)
at org.apache.spark.sql.hudi.command.InsertIntoHoodieTableCommand$.run(InsertIntoHoodieTableCommand.scala:87)
at org.apache.spark.sql.hudi.command.InsertIntoHoodieTableCommand.run(InsertIntoHoodieTableCommand.scala:48)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:231)
at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3699)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:105)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:172)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:92)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:801)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:66)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3697)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:231)
at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:102)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:801)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:96)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:623)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:801)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:616)
at org.apache.livy.thriftserver.session.SqlJob.executeSql(SqlJob.java:93)
at org.apache.livy.thriftserver.session.SqlJob.call(SqlJob.java:73)
at org.apache.livy.thriftserver.session.SqlJob.call(SqlJob.java:40)
at org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:84)
at org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:34)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.apache.calcite.avatica.Helper.createException(Helper.java:68)
at org.apache.calcite.avatica.Helper.createSQLException(Helper.java:114)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:234)
at org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:287)
at jdk.internal.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.base/java.lang.Thread.run(Thread.java:829)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] dongkelun commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
dongkelun commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1291543658
@adward32135 Hello, can you provide the table structure in Hive metadata? I suspect 46 fields are stored in the metadata. In addition, what is your Spark version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] adward32135 commented on issue #7058: [SUPPORT]java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.AssertionError: assertion failed: Required select columns count: 46, Current select columns(including static partition column) count: 45，columns:
Posted by GitBox <gi...@apache.org>.
adward32135 commented on issue #7058:
URL: https://github.com/apache/hudi/issues/7058#issuecomment-1314878225
ok, the problem has been solved, thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org