You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/10/13 07:58:35 UTC

[GitHub] [incubator-seatunnel] Super-Sky opened a new pull request, #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Super-Sky opened a new pull request, #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087

   ### What happened？
   There are 1 security vulnerabilities found in org.apache.hive:hive-exec 2.3.9
   - [CVE-2021-34538](https://www.oscs1024.com/hd/CVE-2021-34538)
   
   
   ### What did I do？
   Upgrade org.apache.hive:hive-exec from 2.3.9 to 3.1.3 for vulnerability fix
   
   ### What did you expect to happen？
   Ideally, no insecure libs should be used.
   
   ### The specification of the pull request
   [PR Specification](https://www.oscs1024.com/docs/pr-specification/) from OSCS


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] TyrantLucifer commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on code in PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#discussion_r994640473


##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -30,7 +27,7 @@
     <artifactId>connector-hive</artifactId>
     
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   hive2 and hive3 are in conflict. I had tested it. For better compatibility, I do not recommend modifying this version.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] CalvinKirs commented on pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#issuecomment-1277220668

   Thank you for your contribution, this is what we are planning later, but I think you may have missed the test, the version upgrade currently does not have a very good IT to test, so we have more difficulties to upgrade, can you improve the corresponding IT by the way? https://github.com/apache/incubator-seatunnel/tree/dev/seatunnel-e2e


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#issuecomment-1277569423

   Thank you for your contribution, overall LGTM except nits. I think rename your pull request name to `[Hotfix][Connector-V2][Hive]` is better. You can refer to https://seatunnel.apache.org/docs/contribution/coding-guide to get more details about how to submit a high quility pull request.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [seatunnel] SinyoWong commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by "SinyoWong (via GitHub)" <gi...@apache.org>.
SinyoWong commented on code in PR #3087:
URL: https://github.com/apache/seatunnel/pull/3087#discussion_r1214118583


##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -30,7 +27,7 @@
     <artifactId>connector-hive</artifactId>
     
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   The typical error is : Caused by: org.apache.thrift.transport.TTransportException.
   ![企业微信截图_16856966145741](https://github.com/apache/seatunnel/assets/38577585/cbad9ff9-a2b7-41f0-aff8-88c7223eb448)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on code in PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#discussion_r994612076


##########
seatunnel-connectors-v2/connector-hudi/pom.xml:
##########
@@ -29,7 +26,7 @@
 
     <artifactId>connector-hudi</artifactId>
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   same as above.



##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -30,7 +27,7 @@
     <artifactId>connector-hive</artifactId>
     
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   You need add test case to ensure whether hive `3.1.3` can used in all of `hive 2.x` and `hive 3.x` environment . This means you should test:
   1. run hive connector in a spark and flink docker container which integrate hive2.x.
   2. run hive connector in a spark and flink docker container which integrate hive3.x.
   
   If `hive 3.1.3` can run fine in all of `hive 2.x` and `hive 3.x` environment, you need update the document of the hive connector tell user it support hive2.x and hive3.x.
   
   If `hive 3.1.3` can only run fine in `hive 3.x` environment and can not run in `hive2.x` environment, you can create a connector named `connector-hive3`.



##########
seatunnel-connectors-v2/connector-iceberg/pom.xml:
##########
@@ -33,7 +30,7 @@
         <iceberg.version>0.14.0</iceberg.version>
         <parquet-avro.version>1.12.3</parquet-avro.version>
         <avro.version>1.10.2</avro.version>
-        <hive.version>2.3.9</hive.version>
+        <hive.version>3.1.3</hive.version>

Review Comment:
   same as above.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] TyrantLucifer commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on code in PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#discussion_r995583333


##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -30,7 +27,7 @@
     <artifactId>connector-hive</artifactId>
     
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   hive2 can run in hive3 environment.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] EricJoy2048 closed pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
EricJoy2048 closed pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3
URL: https://github.com/apache/incubator-seatunnel/pull/3087


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on code in PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#discussion_r995570366


##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -30,7 +27,7 @@
     <artifactId>connector-hive</artifactId>
     
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   > 
   
   If hive2 and hive3 are in conflict. we need update connector-hive to connector-hive2 and add connector-hive3 connector.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by GitBox <gi...@apache.org>.
hailin0 commented on code in PR #3087:
URL: https://github.com/apache/incubator-seatunnel/pull/3087#discussion_r994294604


##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -16,10 +16,7 @@
     See the License for the specific language governing permissions and
     limitations under the License.
 
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0"
-         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+--><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

Review Comment:
   revert？



##########
seatunnel-connectors-v2/connector-hudi/pom.xml:
##########
@@ -16,10 +16,7 @@
     See the License for the specific language governing permissions and
     limitations under the License.
 
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0"
-         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+--><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

Review Comment:
   revert？



##########
seatunnel-connectors-v2/connector-iceberg/pom.xml:
##########
@@ -16,10 +16,7 @@
     See the License for the specific language governing permissions and
     limitations under the License.
 
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0"
-         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+--><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

Review Comment:
   revert



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [seatunnel] SinyoWong commented on a diff in pull request #3087: fix(sec): upgrade org.apache.hive:hive-exec to 3.1.3

Posted by "SinyoWong (via GitHub)" <gi...@apache.org>.
SinyoWong commented on code in PR #3087:
URL: https://github.com/apache/seatunnel/pull/3087#discussion_r1214103990


##########
seatunnel-connectors-v2/connector-hive/pom.xml:
##########
@@ -30,7 +27,7 @@
     <artifactId>connector-hive</artifactId>
     
     <properties>
-        <hive.exec.version>2.3.9</hive.exec.version>
+        <hive.exec.version>3.1.3</hive.exec.version>

Review Comment:
   > hive2 can run in hive3 environment.
   
   But when the hive2 job runs on yarn which based on hive3 environment, dependency will be wrong.
   Could you plz help me to solve this dependency problem? Thks a lot!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org