You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "jiangshouzhuang (JIRA)" <ji...@apache.org> on 2016/08/03 09:55:20 UTC
[jira] [Comment Edited] (KYLIN-1760) Save query hits
org.apache.hadoop.hbase.TableNotFoundException: kylin_metadata_user
[ https://issues.apache.org/jira/browse/KYLIN-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405617#comment-15405617 ]
jiangshouzhuang edited comment on KYLIN-1760 at 8/3/16 9:54 AM:
----------------------------------------------------------------
I'm very happy that you resolve the bug which cannot save queries by Web GUI.
But when I query the htable 'kylin_metadata_user' in release 1.5.3, I find that 'kylin_metadata_user' column family is 'a', but it need 'q'.
hbase(main):004:0> desc 'kylin_metadata_user'
Table kylin_metadata_user is ENABLED
kylin_metadata_user, {TABLE_ATTRIBUTES => {METADATA => {'UUID' => '41e483de-7e3e-4374-9b0f-2f361841aba2'}}
COLUMN FAMILIES DESCRIPTION
{NAME => 'a', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREV
ER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}
1 row(s) in 0.0520 seconds
hbase(main):005:0> alter 'kylin_metadata_user' ,{ NAME => 'q', VERSIONS => '1' }
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 3.0320 seconds
When I add then column family 'q', I can use "http://ipaddress:7070/kylin/query" to save queries.
hbase(main):006:0> scan 'kylin_metadata_user'
ROW COLUMN+CELL
ADMIN column=q:c, timestamp=1470211633112, value=[{"name":"s_1","project":"myproject_pvuv","sql":"select * from STREAMING_SALES_TA
BLE ","description":null,"id":"-932856725"}]
1 row(s) in 0.0480 seconds
The source code is location in 'apache-kylin-1.5.3\server-base\src\main\java\org\apache\kylin\rest\service\QueryService.java',as following:
public static final String USER_QUERY_FAMILY = "q";
public void saveQuery(final String creator, final Query query) throws IOException {
List<Query> queries = getQueries(creator);
queries.add(query);
Query[] queryArray = new Query[queries.size()];
byte[] bytes = querySerializer.serialize(queries.toArray(queryArray));
HTableInterface htable = null;
try {
htable = HBaseConnection.get(hbaseUrl).getTable(userTableName);
Put put = new Put(Bytes.toBytes(creator));
put.add(Bytes.toBytes(USER_QUERY_FAMILY), Bytes.toBytes(USER_QUERY_COLUMN), bytes);
htable.put(put);
htable.flushCommits();
} finally {
IOUtils.closeQuietly(htable);
}
}
I hope that you can take a little time to solve the bug. Thank you very much.
Error log as following :
2016-08-03 17:50:59,656 DEBUG [http-bio-7070-exec-1] hbase.HBaseConnection:238 : HTable 'kylin_metadata_user' already exists
2016-08-03 17:50:59,661 ERROR [http-bio-7070-exec-1] controller.BasicController:44 :
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family q does not exist in region kylin_metadata_user,,1470217040618.dfee2dff003b16dfd569b9318e09ddeb. in table 'kylin_metadata_user', {TABLE_ATTRIBUTES => {METADATA => {'UUID' => 'a05e588b-8c62-4d81-8b24-60fa66e2fc43'}}, {NAME => 'a', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}
at org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7371)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6531)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1936)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32381)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2127)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:326)
at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:861)
at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:848)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:865)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:831)
at org.apache.kylin.rest.service.QueryService.getQueries(QueryService.java:192)
at org.apache.kylin.rest.service.QueryService.saveQuery(QueryService.java:129)
at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$f2ce01fc.saveQuery(<generated>)
at org.apache.kylin.rest.controller.QueryController.saveQuery(QueryController.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:195)
at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:266)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
was (Author: jiangshouzhuang):
I'm very happy that you resolve the bug which cannot save queries by Web GUI.
But when I query the htable 'kylin_metadata_user' in release 1.5.3, I find that 'kylin_metadata_user' column family is 'a', but it need 'q'.
hbase(main):004:0> desc 'kylin_metadata_user'
Table kylin_metadata_user is ENABLED
kylin_metadata_user, {TABLE_ATTRIBUTES => {METADATA => {'UUID' => '41e483de-7e3e-4374-9b0f-2f361841aba2'}}
COLUMN FAMILIES DESCRIPTION
{NAME => 'a', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => 'FOREV
ER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}
1 row(s) in 0.0520 seconds
hbase(main):005:0> alter 'kylin_metadata_user' ,{ NAME => 'q', VERSIONS => '1' }
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 3.0320 seconds
When I add then column family 'q', I can use "http://ipaddress:7070/kylin/query" to save queries.
hbase(main):006:0> scan 'kylin_metadata_user'
ROW COLUMN+CELL
ADMIN column=q:c, timestamp=1470211633112, value=[{"name":"s_1","project":"myproject_pvuv","sql":"select * from STREAMING_SALES_TA
BLE ","description":null,"id":"-932856725"}]
1 row(s) in 0.0480 seconds
The source code is location in 'apache-kylin-1.5.3\server-base\src\main\java\org\apache\kylin\rest\service\QueryService.java',as following:
public static final String USER_QUERY_FAMILY = "q";
public void saveQuery(final String creator, final Query query) throws IOException {
List<Query> queries = getQueries(creator);
queries.add(query);
Query[] queryArray = new Query[queries.size()];
byte[] bytes = querySerializer.serialize(queries.toArray(queryArray));
HTableInterface htable = null;
try {
htable = HBaseConnection.get(hbaseUrl).getTable(userTableName);
Put put = new Put(Bytes.toBytes(creator));
put.add(Bytes.toBytes(USER_QUERY_FAMILY), Bytes.toBytes(USER_QUERY_COLUMN), bytes);
htable.put(put);
htable.flushCommits();
} finally {
IOUtils.closeQuietly(htable);
}
}
I hope that you can take a little time to solve the bug. Thank you very much.
> Save query hits org.apache.hadoop.hbase.TableNotFoundException: kylin_metadata_user
> -----------------------------------------------------------------------------------
>
> Key: KYLIN-1760
> URL: https://issues.apache.org/jira/browse/KYLIN-1760
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v1.5.1
> Reporter: jiangshouzhuang
> Assignee: liyang
> Fix For: v1.5.3
>
>
> When I use "http://ipaddress:7070/kylin/query" to save queries, I notice the log file kylin.log which report some errors :
> org.apache.hadoop.hbase.TableNotFoundException: Table 'kylin_metadata_user' was not found, got: kylin_metadata.
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1275)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1156)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1140)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1097)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:932)
> at org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
> at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:889)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:855)
> at org.apache.kylin.rest.service.QueryService.getQueries(QueryService.java:189)
> at org.apache.kylin.rest.service.QueryService.saveQuery(QueryService.java:129)
> at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$67382972.saveQuery(<generated>)
> at org.apache.kylin.rest.controller.QueryController.saveQuery(QueryController.java:110)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> By anaylzing the source codes of Kylin, I see the file apache-kylin-1.5.2\server\src\main\java\org\apache\kylin\rest\service\QueryService.java, so I create the table "kylin_metadata_user" in the HBase.
> hbase(main):001:0> create 'kylin_metadata_user','q'
> 0 row(s) in 1.5880 seconds
> => Hbase::Table - kylin_metadata_user
> But I think that it maybe a bug. Thank you.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)