You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/09/24 12:14:00 UTC
[jira] [Commented] (PARQUET-127) Use published Thrift
plugin/artifact
[ https://issues.apache.org/jira/browse/PARQUET-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625728#comment-16625728 ]
ASF GitHub Bot commented on PARQUET-127:
----------------------------------------
gszadovszky closed pull request #81: PARQUET-127: Point to Maven Central for Thrift artifacts.
URL: https://github.com/apache/parquet-mr/pull/81
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/parquet-cascading/pom.xml b/parquet-cascading/pom.xml
index 25e6e641d..d1d1ddd90 100644
--- a/parquet-cascading/pom.xml
+++ b/parquet-cascading/pom.xml
@@ -82,7 +82,7 @@
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
- <version>0.1.10</version>
+ <version>0.1.11</version>
<configuration>
<thriftExecutable>${thrift.executable}</thriftExecutable>
</configuration>
diff --git a/parquet-scrooge/pom.xml b/parquet-scrooge/pom.xml
index 035a0acc3..84520ebe6 100644
--- a/parquet-scrooge/pom.xml
+++ b/parquet-scrooge/pom.xml
@@ -110,7 +110,7 @@
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
- <version>0.1.10</version>
+ <version>0.1.11</version>
<configuration>
<thriftExecutable>${thrift.executable}</thriftExecutable>
</configuration>
diff --git a/parquet-thrift/pom.xml b/parquet-thrift/pom.xml
index 08ba97549..efedf8e68 100644
--- a/parquet-thrift/pom.xml
+++ b/parquet-thrift/pom.xml
@@ -97,7 +97,11 @@
<version>3.4</version>
<scope>test</scope>
</dependency>
-
+ <dependency>
+ <groupId>org.apache.thrift</groupId>
+ <artifactId>libthrift</artifactId>
+ <version>0.9.1</version>
+ </dependency>
</dependencies>
<build>
@@ -113,7 +117,7 @@
<plugin>
<groupId>org.apache.thrift.tools</groupId>
<artifactId>maven-thrift-plugin</artifactId>
- <version>0.1.10</version>
+ <version>0.1.11</version>
<configuration>
<thriftExecutable>${thrift.executable}</thriftExecutable>
</configuration>
diff --git a/parquet-thrift/src/main/java/parquet/hadoop/thrift/ThriftBytesWriteSupport.java b/parquet-thrift/src/main/java/parquet/hadoop/thrift/ThriftBytesWriteSupport.java
index 5eb4a30b3..2f84c37ea 100644
--- a/parquet-thrift/src/main/java/parquet/hadoop/thrift/ThriftBytesWriteSupport.java
+++ b/parquet-thrift/src/main/java/parquet/hadoop/thrift/ThriftBytesWriteSupport.java
@@ -118,14 +118,6 @@ public WriteContext init(Configuration configuration) {
private TProtocol protocol(BytesWritable record) {
TProtocol protocol = protocolFactory.getProtocol(new TIOStreamTransport(new ByteArrayInputStream(record.getBytes())));
-
- /* Reduce the chance of OOM when data is corrupted. When readBinary is called on TBinaryProtocol, it reads the length of the binary first,
- so if the data is corrupted, it could read a big integer as the length of the binary and therefore causes OOM to happen.
- Currently this fix only applies to TBinaryProtocol which has the setReadLength defined.
- */
- if (protocol instanceof TBinaryProtocol) {
- ((TBinaryProtocol)protocol).setReadLength(record.getLength());
- }
return protocol;
}
diff --git a/pom.xml b/pom.xml
index 95bacea61..031e023cc 100644
--- a/pom.xml
+++ b/pom.xml
@@ -49,7 +49,6 @@
<name>Nexus Release Repository</name>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
-
</distributionManagement>
<repositories>
@@ -65,15 +64,6 @@
</repository>
</repositories>
- <!-- this is needed for maven-thrift-plugin, would like to remove this.
- see: https://issues.apache.org/jira/browse/THRIFT-1536 -->
- <pluginRepositories>
- <pluginRepository>
- <id>Twitter public Maven repo</id>
- <url>http://maven.twttr.com</url>
- </pluginRepository>
- </pluginRepositories>
-
<properties>
<targetJavaVersion>1.6</targetJavaVersion>
<maven.compiler.source>1.6</maven.compiler.source>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Use published Thrift plugin/artifact
> ------------------------------------
>
> Key: PARQUET-127
> URL: https://issues.apache.org/jira/browse/PARQUET-127
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Reporter: Frank Austin Nothaft
> Assignee: Jake Farrell
> Priority: Minor
> Labels: pull-request-available
>
> Thrift is now publishing the [Thrift tools|https://issues.apache.org/jira/browse/THRIFT-1536] to Maven Central. We should move Parquet to depend on the Maven Central artifact instead of the artifact published in the Twitter repo. Also, we should depend on the Maven libthrift artifact instead of requiring users to have libthrift installed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)