You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "leaves12138 (via GitHub)" <gi...@apache.org> on 2023/10/30 08:16:46 UTC
[PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine shema evolution and selected fields [incubator-paimon]
leaves12138 opened a new pull request, #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202
### Purpose
* Refactory `OrcReaderFactory` to combine orc schema evolution and orc selected fields push down.
* Orc version of 1.5.6 has problem in supporting filter push down while have a schema evolution(read schema is not compitable with orc file schema) This is a bug of orc version 1.5.6. related issue: https://github.com/apache/orc/pull/452
### Documentation
No influence to user in theory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 closed pull request #2202: [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down"
URL: https://github.com/apache/incubator-paimon/pull/2202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 closed pull request #2202: [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down"
URL: https://github.com/apache/incubator-paimon/pull/2202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on code in PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202#discussion_r1377032551
##########
paimon-format/pom.xml:
##########
@@ -279,7 +282,9 @@ under the License.
<include>org.apache.hive:hive-storage-api</include>
<include>io.airlift:aircompressor</include>
<include>commons-lang:commons-lang</include>
+ <include>org.apache.commons:commons-lang3</include>
Review Comment:
orc.apache.commons.lang3 has already be shaded by
<!-- Relocate Common. -->
<relocation>
<pattern>org.apache.commons</pattern>
<shadedPattern>org.apache.paimon.shade.org.apache.commons</shadedPattern>
</relocation>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202#issuecomment-1786373451
Done simple compatible test. It could read the data before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on code in PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202#discussion_r1377034282
##########
paimon-format/pom.xml:
##########
@@ -279,7 +282,9 @@ under the License.
<include>org.apache.hive:hive-storage-api</include>
<include>io.airlift:aircompressor</include>
<include>commons-lang:commons-lang</include>
+ <include>org.apache.commons:commons-lang3</include>
Review Comment:
final jar:

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 closed pull request #2202: [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down"
URL: https://github.com/apache/incubator-paimon/pull/2202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202#discussion_r1377030673
##########
paimon-format/pom.xml:
##########
@@ -279,7 +282,9 @@ under the License.
<include>org.apache.hive:hive-storage-api</include>
<include>io.airlift:aircompressor</include>
<include>commons-lang:commons-lang</include>
+ <include>org.apache.commons:commons-lang3</include>
Review Comment:
relocate for org.apache.commons.lang3?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on code in PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202#discussion_r1377032839
##########
paimon-format/pom.xml:
##########
@@ -279,7 +282,9 @@ under the License.
<include>org.apache.hive:hive-storage-api</include>
<include>io.airlift:aircompressor</include>
<include>commons-lang:commons-lang</include>
+ <include>org.apache.commons:commons-lang3</include>
Review Comment:

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine "shema evolution" and "selected fields push down" [incubator-paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #2202:
URL: https://github.com/apache/incubator-paimon/pull/2202#discussion_r1377031107
##########
paimon-format/pom.xml:
##########
@@ -279,7 +282,9 @@ under the License.
<include>org.apache.hive:hive-storage-api</include>
<include>io.airlift:aircompressor</include>
<include>commons-lang:commons-lang</include>
+ <include>org.apache.commons:commons-lang3</include>
Review Comment:
Please check final jar classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine shema evolution and selected fields [incubator-paimon]
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 closed pull request #2202: [format] update orc version to support schema evolution filter push down, refactory OrcReaderFactory to combine shema evolution and selected fields
URL: https://github.com/apache/incubator-paimon/pull/2202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org