You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/11/03 02:54:02 UTC
[GitHub] [hudi] xiarixiaoyao opened a new pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
xiarixiaoyao opened a new pull request #3910:
URL: https://github.com/apache/hudi/pull/3910
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
now when we use hive to query hudi table and set hive.input.format=org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat;
all read values will be print. This can lead to performance problems and data security problems,
## Verify this pull request
Existing UT
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4a2a953dda077e92264164a3540de5741767a6e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a2a953dda077e92264164a3540de5741767a6e1",
"triggerType" : "PUSH"
}, {
"hash" : "de652ade904eb0b2f2118961d4bef5d20bbe6b56",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3087",
"triggerID" : "de652ade904eb0b2f2118961d4bef5d20bbe6b56",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a2a953dda077e92264164a3540de5741767a6e1 UNKNOWN
* de652ade904eb0b2f2118961d4bef5d20bbe6b56 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3087)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4a2a953dda077e92264164a3540de5741767a6e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a2a953dda077e92264164a3540de5741767a6e1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a2a953dda077e92264164a3540de5741767a6e1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#discussion_r741592164
##########
File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieCombineRealtimeRecordReader.java
##########
@@ -66,8 +65,6 @@ public HoodieCombineRealtimeRecordReader(JobConf jobConf, CombineFileSplit split
@Override
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
if (this.currentRecordReader.next(key, value)) {
- LOG.info("Reading from record reader");
Review comment:
we shouldn't have let this in. really surprised we let this slide for so long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #3910:
URL: https://github.com/apache/hudi/pull/3910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4a2a953dda077e92264164a3540de5741767a6e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a2a953dda077e92264164a3540de5741767a6e1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a2a953dda077e92264164a3540de5741767a6e1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613216
@leesf @nsivabalan could you pls help me review this code. thanks
this is a performance problem and data security problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958618774
thanks for catching this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #3910:
URL: https://github.com/apache/hudi/pull/3910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4a2a953dda077e92264164a3540de5741767a6e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a2a953dda077e92264164a3540de5741767a6e1",
"triggerType" : "PUSH"
}, {
"hash" : "de652ade904eb0b2f2118961d4bef5d20bbe6b56",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de652ade904eb0b2f2118961d4bef5d20bbe6b56",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a2a953dda077e92264164a3540de5741767a6e1 UNKNOWN
* de652ade904eb0b2f2118961d4bef5d20bbe6b56 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#discussion_r741592164
##########
File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieCombineRealtimeRecordReader.java
##########
@@ -66,8 +65,6 @@ public HoodieCombineRealtimeRecordReader(JobConf jobConf, CombineFileSplit split
@Override
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
if (this.currentRecordReader.next(key, value)) {
- LOG.info("Reading from record reader");
Review comment:
we shouldn't have let this in. really surprised we let this slide for so long.
##########
File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieCombineRealtimeRecordReader.java
##########
@@ -66,8 +65,6 @@ public HoodieCombineRealtimeRecordReader(JobConf jobConf, CombineFileSplit split
@Override
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
if (this.currentRecordReader.next(key, value)) {
- LOG.info("Reading from record reader");
Review comment:
we shouldn't have let this in. really surprised we let this slide for so long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4a2a953dda077e92264164a3540de5741767a6e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a2a953dda077e92264164a3540de5741767a6e1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a2a953dda077e92264164a3540de5741767a6e1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958618774
thanks for catching this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #3910:
URL: https://github.com/apache/hudi/pull/3910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958618774
thanks for catching this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot edited a comment on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot edited a comment on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958618774
thanks for catching this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613692
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4a2a953dda077e92264164a3540de5741767a6e1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4a2a953dda077e92264164a3540de5741767a6e1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4a2a953dda077e92264164a3540de5741767a6e1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run travis` re-run the last Travis build
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #3910:
URL: https://github.com/apache/hudi/pull/3910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613216
@leesf @nsivabalan could you pls help me review this code. thanks
this is a performance problem and data security problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613216
@leesf @nsivabalan could you pls help me review this code. thanks
this is a performance problem and data security problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#issuecomment-958613216
@leesf @nsivabalan could you pls help me review this code. thanks
this is a performance problem and data security problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #3910: [HUDI-2674] hudi hive reader should not print read values.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #3910:
URL: https://github.com/apache/hudi/pull/3910#discussion_r741592164
##########
File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/HoodieCombineRealtimeRecordReader.java
##########
@@ -66,8 +65,6 @@ public HoodieCombineRealtimeRecordReader(JobConf jobConf, CombineFileSplit split
@Override
public boolean next(NullWritable key, ArrayWritable value) throws IOException {
if (this.currentRecordReader.next(key, value)) {
- LOG.info("Reading from record reader");
Review comment:
we shouldn't have let this in. really surprised we let this slide for so long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org