[GitHub] [incubator-paimon] gitfortian opened a new issue, #694: [Bug] Recreate the same paimon hive catalog table after drop it failed because of schema file already exists

["gitfortian (via GitHub)" <gi...@apache.org>]

gitfortian opened a new issue, #694:
URL: https://github.com/apache/incubator-paimon/issues/694

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
   
   
   ### Paimon version
   
   0.4-snapshot
   
   ### Compute Engine
   
   flink-1.16
   
   ### Minimal reproduce step
   
   CREATE CATALOG paimon WITH (
       'type' = 'paimon',
       'metastore' = 'hive', 
       'uri' = 'thrift://xxx:9083',
       'warehouse' ='hdfs://hadoop3/table_store/warehouse',
       'table.type'='EXTERNAL'
   );
   use catalog paimon;
   CREATE TABLE if not exists demo_log_01 (
       user_id BIGINT,
       item_id BIGINT,
       behavior STRING,
       dt STRING,
       hh STRING,
       PRIMARY KEY (dt, hh, user_id) NOT ENFORCED
   ) ;
   drop table demo_log_01;
   then create table again:
   CREATE TABLE if not exists demo_log_01 (
       user_id BIGINT,
       item_id BIGINT,
       behavior STRING,
       dt STRING,
       hh STRING,
       PRIMARY KEY (dt, hh, user_id) NOT ENFORCED
   ) ;
   
   Error message:
    org.apache.flink.table.api.TableException: Could not execute CreateTable in path `fts`.`default`.`demo_log_01`
   	at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:891)
   	at org.apache.flink.table.catalog.CatalogManager.createTable(CatalogManager.java:652)
   	at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:929)
   	at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:730)
   	at com.dlink.executor.Executor.executeSql(Executor.java:249)
   	at com.dlink.job.JobManager.executeSql(JobManager.java:425)
   	at com.dlink.service.impl.StudioServiceImpl.executeFlinkSql(StudioServiceImpl.java:202)
   	at com.dlink.service.impl.StudioServiceImpl.executeSql(StudioServiceImpl.java:189)
   	at com.dlink.service.impl.StudioServiceImpl$$FastClassBySpringCGLIB$$e3eb787.invoke(<generated>)
   	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
   	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
   	at com.dlink.aop.UdfClassLoaderAspect.round(UdfClassLoaderAspect.java:65)
   	at sun.reflect.GeneratedMethodAccessor125.invoke(Unknown Source)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
   	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
   	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
   	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
   	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
   	at com.dlink.service.impl.StudioServiceImpl$$EnhancerBySpringCGLIB$$c912ffb1.executeSql(<generated>)
   	at com.dlink.controller.StudioController.executeSql(StudioController.java:78)
   	at com.dlink.controller.StudioController$$FastClassBySpringCGLIB$$e6483d87.invoke(<generated>)
   	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
   	at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:57)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
   	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
   	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
   	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
   	at com.dlink.controller.StudioController$$EnhancerBySpringCGLIB$$d39000f.executeSql(<generated>)
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
   	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
   	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
   	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
   	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
   	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
   	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071)
   	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964)
   	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
   	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
   	at javax.servlet.http.HttpServlet.service(HttpServlet.java:696)
   	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
   	at javax.servlet.http.HttpServlet.service(HttpServlet.java:779)
   	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
   	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
   	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
   	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
   	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
   	at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:124)
   	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
   	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
   	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
   	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
   	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
   	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
   	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
   	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
   	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
   	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
   	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
   	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
   	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
   	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
   	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
   	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
   	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
   	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
   	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
   	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
   	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
   	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
   	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
   	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
   	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)
   	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
   	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
   	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
   	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   	at java.lang.Thread.run(Thread.java:750)
   Caused by: java.lang.RuntimeException: Failed to commit changes of table default.demo_log_01 to underlying files
   	at org.apache.paimon.hive.HiveCatalog.createTable(HiveCatalog.java:237)
   	at org.apache.paimon.flink.FlinkCatalog.createTable(FlinkCatalog.java:219)
   	at org.apache.flink.table.catalog.CatalogManager.lambda$createTable$11(CatalogManager.java:663)
   	at org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:885)
   	... 94 more
   Caused by: java.lang.IllegalStateException: Schema in filesystem exists, please use updating, latest schema is: Optional[{
     "id" : 0,
     "fields" : [ {
       "id" : 0,
       "name" : "user_id",
       "type" : "BIGINT NOT NULL"
     }, {
       "id" : 1,
       "name" : "item_id",
       "type" : "BIGINT"
     }, {
       "id" : 2,
       "name" : "behavior",
       "type" : "STRING"
     }, {
       "id" : 3,
       "name" : "dt",
       "type" : "STRING NOT NULL"
     }, {
       "id" : 4,
       "name" : "hh",
       "type" : "STRING NOT NULL"
     } ],
     "highestFieldId" : 4,
     "partitionKeys" : [ "dt", "hh" ],
     "primaryKeys" : [ "dt", "hh", "user_id" ],
     "options" : {
       "bucket" : "4"
     }
   }]
   	at org.apache.paimon.schema.SchemaManager.lambda$createTable$0(SchemaManager.java:118)
   	at java.util.Optional.ifPresent(Optional.java:159)
   	at org.apache.paimon.schema.SchemaManager.createTable(SchemaManager.java:113)
   	at org.apache.paimon.hive.HiveCatalog.createTable(HiveCatalog.java:233)
   	... 97 more
   
   ### What doesn't meet your expectations?
   
   recreate a table should be supportted 
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-paimon] JingsongLi closed issue #694: [Bug] Recreate the same paimon hive catalog table after drop it failed because of schema file already exists

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi closed issue #694: [Bug] Recreate the same paimon hive catalog table after drop it failed because of schema file already exists 
URL: https://github.com/apache/incubator-paimon/issues/694


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org