You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liugs0213 (via GitHub)" <gi...@apache.org> on 2023/03/13 09:39:19 UTC

[GitHub] [doris] liugs0213 opened a new issue, #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

liugs0213 opened a new issue, #17746:
URL: https://github.com/apache/doris/issues/17746

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Version
   
   doris-1.2.2-rc01-a3521b3
   
   ### What's Wrong?
   
   1.Iceberg底层存储使用S3
   
   通过Iceberg Catalog连接HMS, Doris读取的数据未NULL,hive读取数据正常
   ![image](https://user-images.githubusercontent.com/55676558/224663006-0f23ab38-c3c7-41d9-9489-779922f33440.png)
   
   
   ### What You Expected?
   
   希望能通过Doris读取和HMS相同的数据
   
   ### How to Reproduce?
   
   通过spark连接HMS 创建表并且插入n条数据，然后使用hive读正常，使用doris读却未null
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] liugs0213 commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "liugs0213 (via GitHub)" <gi...@apache.org>.
liugs0213 commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1467425418

   > 
   
   
   
   > This may because the file column name not matched with name in table. You can try following step to check:
   > 
   > 1. use table value function `hdfs()`: https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-functions/table-functions/hdfs
   >    To read the parquet file directly.
   > 
   > You can also use `describe function hdfs(xx)` to check the column name in parquet file, to see if they are same.
   
   1.function hdfs() s3 not supported:
   ![image](https://user-images.githubusercontent.com/55676558/224911250-e7d8af43-7a93-4922-81d6-e27b69651656.png)
   2.Using spark to read and view metadata columns is the same:
   ![image](https://user-images.githubusercontent.com/55676558/224911463-9c35fe27-e2ba-41dd-869b-4a75dfe5769f.png)
   ![image](https://user-images.githubusercontent.com/55676558/224911574-2b1ff163-6bf6-49cc-bbae-d8d49621834e.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] liugs0213 commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "liugs0213 (via GitHub)" <gi...@apache.org>.
liugs0213 commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1476202971

   
   > It is strange. This is my wechat: morningman-cmy, you can contact me in private.
   
   在大佬的建议下，升级到最新版本解决


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] liugs0213 commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "liugs0213 (via GitHub)" <gi...@apache.org>.
liugs0213 commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1467431060

   > This may because the file column name not matched with name in table. You can try following step to check:
   > 
   > 1. use table value function `hdfs()`: https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-functions/table-functions/hdfs
   >    To read the parquet file directly.
   > 
   > You can also use `describe function hdfs(xx)` to check the column name in parquet file, to see if they are same.
   
   function s3() is ok:
   ![image](https://user-images.githubusercontent.com/55676558/224912662-aa24b3cb-867f-472d-bf4c-b910568fbfe0.png)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] morningman commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1467273465

   This may because the file column name not matched with name in table.
   You can try following step to check:
   
   1. use table value function `hdfs()`: https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-functions/table-functions/hdfs
   To read the parquet file directly.
   
   You can also use `describe function hdfs(xx)` to check the column name in parquet file, to see if they are same.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] liugs0213 closed issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "liugs0213 (via GitHub)" <gi...@apache.org>.
liugs0213 closed issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug] 
URL: https://github.com/apache/doris/issues/17746


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] morningman commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1466121583

   What is your hive version?
   And what is the file format?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] morningman commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1469247190

   It is strange.
   This is my wechat: morningman-cmy, you can contact me in private.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] liugs0213 commented on issue #17746: Doris1.2.2 通过HMS 访问 Iceberg异常[Bug]

Posted by "liugs0213 (via GitHub)" <gi...@apache.org>.
liugs0213 commented on issue #17746:
URL: https://github.com/apache/doris/issues/17746#issuecomment-1466149026

   > What is your hive version? And what is the file format?
   
   hive version: 3.1.2
   ![image](https://user-images.githubusercontent.com/55676558/224715967-88a88e1e-2b4c-4d9c-a4d6-a0ccbacfafba.png)
   fileformat: parquet
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org