You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/06/18 20:55:49 UTC

[GitHub] [iceberg] uncleGen opened a new pull request #2711: Fix wrong position to get field from record

uncleGen opened a new pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711


   This bug is met when read iceberg format V2 table with hive on spark or hive on tez. As shown bellow, the position of filed `env_key` in table schema is 22. But in internal processing, the record has three fields: <'id', 'env_key', 'xxx'>. `id` and `env_key` is the `deleteSchema`, and `xxx` is the `requestSchema`. So we should use pos 2 to get field data from record, but should get with `getFieldName`.
   
   ![image](https://user-images.githubusercontent.com/7402327/122536915-9c6c2f80-d057-11eb-959d-3ff319c0d933.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: Fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864758608


   ut failure pending on #2717


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen commented on pull request #2711: Fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen commented on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864758608


   pending on ut fix


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug on Iceberg master branch and Tez-0.9.2, so it should contain #2171.  I will check `TEZ-4248` whether can fix this problem or not. By the way, should we concern fixing this bug when using tez-0.9.2 or lower?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] rdblue commented on pull request #2711: Fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864165444


   @pvary and @marton-bod, can you take a look at this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug on Iceberg master branch and Tez-0.9.2, so it should contain #2171.  I will check `TEZ-4248` whether will fix this problem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug upon Iceberg master branch and Tez-0.9.2, so it should contain #2171.  I will check `TEZ-4248` whether can fix this problem or not. By the way, should we concern fixing this bug when using tez-0.9.2 or lower?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug upon Iceberg master branch and Tez-0.9.2, so it should contain #2171.  I will check `TEZ-4248` whether can fix this problem or not. By the way, should we concern fixing this bug when using tez-0.9.2 or lower if this commit make sense to you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen commented on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen commented on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug on Iceberg master branch and Tez-0.9.2. I will check `TEZ-4248` whether will fix this problem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] pvary commented on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864968453


   @uncleGen: Which version of Hive, Iceberg are you using?
   We have not tested on Hive on Spark, but have seen similar problem in Hive on Tez, where the schema projection properties were not propagated. 
   
   You can check #2171, #2601 for more details. Hive on Tez issue should be fixed by TEZ-4248.
   
   CC: @marton-bod
   
   Thanks,
   Peter


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen commented on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen commented on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug on Iceberg master branch and Tez-0.9.2. I will check `TEZ-4248` whether will fix this problem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] pvary commented on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
pvary commented on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864968453


   @uncleGen: Which version of Hive, Iceberg are you using?
   We have not tested on Hive on Spark, but have seen similar problem in Hive on Tez, where the schema projection properties were not propagated. 
   
   You can check #2171, #2601 for more details. Hive on Tez issue should be fixed by TEZ-4248.
   
   CC: @marton-bod
   
   Thanks,
   Peter


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: Fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864758608






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug upon Iceberg master branch and Tez-0.9.2, so it should contain #2171.  I will check `TEZ-4248` whether can fix this problem or not. Besides, this bug is found both on Hive on Tez and Hive on Spark. By the way, should we concern fixing this bug when using tez-0.9.2 or lower if this commit make sense to you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864995427


   @pvary I find and test this bug on Iceberg master branch and Tez-0.9.2, so it should contain #2171.  I will check `TEZ-4248` whether can fix this problem or not.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] uncleGen edited a comment on pull request #2711: Fix wrong position to get field from record

Posted by GitBox <gi...@apache.org>.
uncleGen edited a comment on pull request #2711:
URL: https://github.com/apache/iceberg/pull/2711#issuecomment-864758608


   pending on #2717


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org