You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Rupeng Wang <wa...@live.cn> on 2020/05/19 08:15:45 UTC
Re: 答复: 问题咨询
Hi,
You can override scoop property in kylin.properties like this“kylin.source.jdbc.sqoop-config-override.${scoop-property}=${value}”
发件人: 周欢磊总部IT <hu...@bkchina.cn>
答复: <us...@kylin.apache.org>
日期: 2020年5月18日 星期一 16:35
收件人: "user@kylin.apache.org" <us...@kylin.apache.org>
主题: 答复: 问题咨询
Hello：
How do I add sqoop parameters to kylin？
for example ：Sqoop parameter -relaxed-isolation
thanks
发件人: Yaqian Zhang <Ya...@126.com>
发送时间: 2020年5月18日 16:29:28
收件人: user@kylin.apache.org <us...@kylin.apache.org>
主题: Re: 问题咨询
Hi harry:
Maybe you can turn to the Sqoop community for help.
在 2020年5月15日，15:31，harry <hu...@bkchina.cn> 写道：
你好，我在用kylin连接sql server数据库时遇到这样一错误：
20/05/15 07:20:55 ERROR manager.SqlManager: Error executing statement: com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.
com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:256)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:108)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:28)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:2519)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:2524)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.setTransactionIsolation(SQLServerConnection.java:2858)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:910)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:253)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:336)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1872)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1671)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
20/05/15 07:20:55 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1677)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
查询了一下，感觉像是sqoop那边的错误：
查询的连接
https://issues.apache.org/jira/browse/SQOOP-2349
但是加了这个连接给出的参数--relaxed-isolation还是报错
能帮忙看一下吗？谢谢！
回复: 答复: 问题咨询
Posted by harry <hu...@bkchina.cn>.
Hi
Thank you. I already know this problem, but I have another problem。
This is an error：
NoSuchObjectException(message:default.kylin_intermediate_pmixdetails_cube_e4de7758_67c7_95db_a81e_137b3da5e515 table not found)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_core(HiveMetaStore.java:1808)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1778)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy78.get_table(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1208)
at org.apache.kylin.source.hive.CLIHiveClient.getHiveTableRows(CLIHiveClient.java:129)
at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.computeRowCount(RedistributeFlatHiveTableStep.java:38)
at org.apache.kylin.source.hive.RedistributeFlatHiveTableStep.doWork(RedistributeFlatHiveTableStep.java:81)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
But I do have this default.kylin_intermediate_pmixdetails_cube_e4de7758_67c7_95db_a81e_137b3da5e515 in my hive
why is this? Ths
发送自 Windows 10 版邮件应用
发件人: Rupeng Wang
发送时间: 2020年5月19日 16:33
收件人: user@kylin.apache.org
主题: Re: 答复: 问题咨询
Oh, sorry, I misunderstood what you meant. If you want to add --relaxed-isolation for sqoop, you need to set the transaction isolation level of HIVE to TRANSACTION_READ_COMMITTED or
TRANSACTION_READ_UNCOMMITTED. Kylin will set --relaxed-isolation for sqoop only when HIVE transaction isolation level is set. You can see more from JdbcHiveInputBase.java
-------------------
Best wishes,
Rupeng Wang
发件人: Rupeng Wang <wa...@live.cn>
答复: <us...@kylin.apache.org>
日期: 2020年5月19日 星期二 16:16
收件人: <us...@kylin.apache.org>
主题: Re: 答复: 问题咨询
Hi,
You can override scoop property in kylin.properties like this“kylin.source.jdbc.sqoop-config-override.${scoop-property}=${value}”
发件人: 周欢磊总部IT <hu...@bkchina.cn>
答复: <us...@kylin.apache.org>
日期: 2020年5月18日 星期一 16:35
收件人: "user@kylin.apache.org" <us...@kylin.apache.org>
主题: 答复: 问题咨询
Hello：
How do I add sqoop parameters to kylin？
for example ：Sqoop parameter -relaxed-isolation
thanks
发件人: Yaqian Zhang <Ya...@126.com>
发送时间: 2020年5月18日 16:29:28
收件人: user@kylin.apache.org <us...@kylin.apache.org>
主题: Re: 问题咨询
Hi harry:
Maybe you can turn to the Sqoop community for help.
在 2020年5月15日，15:31，harry <hu...@bkchina.cn> 写道：
你好，我在用kylin连接sql server数据库时遇到这样一错误：
20/05/15 07:20:55 ERROR manager.SqlManager: Error executing statement: com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.
com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:256)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:108)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:28)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:2519)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:2524)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.setTransactionIsolation(SQLServerConnection.java:2858)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:910)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:253)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:336)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1872)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1671)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
20/05/15 07:20:55 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1677)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
查询了一下，感觉像是sqoop那边的错误：
查询的连接
https://issues.apache.org/jira/browse/SQOOP-2349
但是加了这个连接给出的参数--relaxed-isolation还是报错
能帮忙看一下吗？谢谢！
Re: 答复: 问题咨询
Posted by Rupeng Wang <wa...@live.cn>.
Oh, sorry, I misunderstood what you meant. If you want to add --relaxed-isolation for sqoop, you need to set the transaction isolation level of HIVE to TRANSACTION_READ_COMMITTED or
TRANSACTION_READ_UNCOMMITTED. Kylin will set --relaxed-isolation for sqoop only when HIVE transaction isolation level is set. You can see more from JdbcHiveInputBase.java
-------------------
Best wishes,
Rupeng Wang
发件人: Rupeng Wang <wa...@live.cn>
答复: <us...@kylin.apache.org>
日期: 2020年5月19日 星期二 16:16
收件人: <us...@kylin.apache.org>
主题: Re: 答复: 问题咨询
Hi,
You can override scoop property in kylin.properties like this“kylin.source.jdbc.sqoop-config-override.${scoop-property}=${value}”
发件人: 周欢磊总部IT <hu...@bkchina.cn>
答复: <us...@kylin.apache.org>
日期: 2020年5月18日 星期一 16:35
收件人: "user@kylin.apache.org" <us...@kylin.apache.org>
主题: 答复: 问题咨询
Hello：
How do I add sqoop parameters to kylin？
for example ：Sqoop parameter -relaxed-isolation
thanks
发件人: Yaqian Zhang <Ya...@126.com>
发送时间: 2020年5月18日 16:29:28
收件人: user@kylin.apache.org <us...@kylin.apache.org>
主题: Re: 问题咨询
Hi harry:
Maybe you can turn to the Sqoop community for help.
在 2020年5月15日，15:31，harry <hu...@bkchina.cn> 写道：
你好，我在用kylin连接sql server数据库时遇到这样一错误：
20/05/15 07:20:55 ERROR manager.SqlManager: Error executing statement: com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.
com.microsoft.sqlserver.jdbc.SQLServerException: Setting IsolationLevel to ReadCommitted is not supported.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:256)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:108)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:28)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:2519)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:2524)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.setTransactionIsolation(SQLServerConnection.java:2858)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:910)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:253)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:336)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1872)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1671)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
20/05/15 07:20:55 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1677)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
查询了一下，感觉像是sqoop那边的错误：
查询的连接
https://issues.apache.org/jira/browse/SQOOP-2349
但是加了这个连接给出的参数--relaxed-isolation还是报错
能帮忙看一下吗？谢谢！