You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "VenkatSNarayanan (via GitHub)" <gi...@apache.org> on 2024/07/17 21:21:43 UTC
[PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
VenkatSNarayanan opened a new pull request, #5355:
URL: https://github.com/apache/hive/pull/5355
### What changes were proposed in this pull request?
HCatStorer now respects hive.file.output.extension for the output files it writes.
### Why are the changes needed?
Brings HCatStorer's feature set more in line with Hive's.
### Does this PR introduce _any_ user-facing change?
Adds support for a property to HCatStorer that would previously have been ignored.
### Is the change a dependency upgrade?
No
### How was this patch tested?
TestHCatExtension was added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #5355:
URL: https://github.com/apache/hive/pull/5355#issuecomment-2234520545
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5355) **Quality Gate passed**
Issues
 [2 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5355&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5355&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5355&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5355&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5355&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5355)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "difin (via GitHub)" <gi...@apache.org>.
difin commented on code in PR #5355:
URL: https://github.com/apache/hive/pull/5355#discussion_r1762084959
##########
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/HCatMapReduceTest.java:
##########
@@ -282,7 +282,7 @@ public void map(WritableComparable key, HCatRecord value, Context context)
Job runMRCreate(Map<String, String> partitionValues, List<HCatFieldSchema> partitionColumns,
List<HCatRecord> records, int writeCount, boolean assertWrite) throws Exception {
return runMRCreate(partitionValues, partitionColumns, records, writeCount, assertWrite,
- true, null);
+ true, new HashMap<String, String>());
Review Comment:
Can `Collections.newMap()` be used instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "VenkatSNarayanan (via GitHub)" <gi...@apache.org>.
VenkatSNarayanan commented on PR #5355:
URL: https://github.com/apache/hive/pull/5355#issuecomment-2357044803
@yigress So for my implementation I just tried to match what Hive does in its existing implementation. If a sequential job wants to have different extensions for different tables, it can simply adjust the setting between HCat queries. Does Hive do something other than that to support that case that I missed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "yigress (via GitHub)" <gi...@apache.org>.
yigress commented on PR #5355:
URL: https://github.com/apache/hive/pull/5355#issuecomment-2274021851
can the file extension be detected instead of user setting it explicitly? this may not be a good idea if a single job writes to different file formats. for example user set hive.output.file.extension to be '.parquet' but then same data is loaded into both parquet and orc tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "difin (via GitHub)" <gi...@apache.org>.
difin commented on code in PR #5355:
URL: https://github.com/apache/hive/pull/5355#discussion_r1762084191
##########
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatExtension.java:
##########
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hive.hcatalog.mapreduce;
+
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.Warehouse;
+import org.apache.hadoop.hive.metastore.api.FieldSchema;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hive.hcatalog.common.HCatException;
+import org.apache.hive.hcatalog.data.DefaultHCatRecord;
+import org.apache.hive.hcatalog.data.HCatRecord;
+import org.apache.hive.hcatalog.data.schema.HCatFieldSchema;
+import org.apache.hive.hcatalog.data.schema.HCatSchemaUtils;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+
+public class TestHCatExtension extends HCatMapReduceTest {
+ private static List<HCatRecord> writeRecords;
+ private static List<HCatFieldSchema> dataColumns;
+ protected static final int NUM_RECORDS = 20;
+ protected static final int NUM_TOP_PARTITIONS = 5;
+
+ public TestHCatExtension(String formatName, String serdeClass, String inputFormatClass, String outputFormatClass) throws Exception {
+ super(formatName, serdeClass, inputFormatClass, outputFormatClass);
+ tableName = "testHCatExtension_" + formatName;
+ generateWriteRecords(NUM_RECORDS, NUM_TOP_PARTITIONS, 0);
+ generateDataColumns();
+ }
+
+ protected static void generateDataColumns() throws HCatException {
+ dataColumns = new ArrayList<HCatFieldSchema>();
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("c1", serdeConstants.INT_TYPE_NAME, "")));
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("c2", serdeConstants.STRING_TYPE_NAME, "")));
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("p1", serdeConstants.STRING_TYPE_NAME, "")));
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("p2", serdeConstants.STRING_TYPE_NAME, "")));
+
+ }
+
+ protected static void generateWriteRecords(int max, int mod, int offset) {
+ writeRecords = new ArrayList<HCatRecord>();
+
+ for (int i = 0; i < max; i++) {
+ List<Object> objList = new ArrayList<Object>();
+
+ objList.add(i);
+ objList.add("strvalue" + i);
+ objList.add(String.valueOf((i % mod) + offset));
+ objList.add(String.valueOf((i / (max / 2)) + offset));
+ writeRecords.add(new DefaultHCatRecord(objList));
+ }
+ }
+
+ @Override
+ protected List<FieldSchema> getPartitionKeys() {
+ List<FieldSchema> fields = new ArrayList<FieldSchema>();
+ fields.add(new FieldSchema("p1", serdeConstants.STRING_TYPE_NAME, ""));
+ fields.add(new FieldSchema("p2", serdeConstants.STRING_TYPE_NAME, ""));
+ return fields;
+ }
+
+ @Override
+ protected List<FieldSchema> getTableColumns() {
+ List<FieldSchema> fields = new ArrayList<FieldSchema>();
+ fields.add(new FieldSchema("c1", serdeConstants.INT_TYPE_NAME, ""));
+ fields.add(new FieldSchema("c2", serdeConstants.STRING_TYPE_NAME, ""));
+ return fields;
+ }
+
+ @Test
+ public void testHCatExtension() throws Exception {
+ HashMap<String, String> properties = new HashMap<String, String>();
Review Comment:
Can be simplified as `Map<String, String> properties = new HashMap<>();`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "VenkatSNarayanan (via GitHub)" <gi...@apache.org>.
VenkatSNarayanan commented on code in PR #5355:
URL: https://github.com/apache/hive/pull/5355#discussion_r1764002019
##########
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/HCatMapReduceTest.java:
##########
@@ -282,7 +282,7 @@ public void map(WritableComparable key, HCatRecord value, Context context)
Job runMRCreate(Map<String, String> partitionValues, List<HCatFieldSchema> partitionColumns,
List<HCatRecord> records, int writeCount, boolean assertWrite) throws Exception {
return runMRCreate(partitionValues, partitionColumns, records, writeCount, assertWrite,
- true, null);
+ true, new HashMap<String, String>());
Review Comment:
Changed.
##########
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatExtension.java:
##########
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hive.hcatalog.mapreduce;
+
+import org.apache.hadoop.fs.LocatedFileStatus;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RemoteIterator;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.Warehouse;
+import org.apache.hadoop.hive.metastore.api.FieldSchema;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hive.hcatalog.common.HCatException;
+import org.apache.hive.hcatalog.data.DefaultHCatRecord;
+import org.apache.hive.hcatalog.data.HCatRecord;
+import org.apache.hive.hcatalog.data.schema.HCatFieldSchema;
+import org.apache.hive.hcatalog.data.schema.HCatSchemaUtils;
+import org.junit.Assert;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+
+public class TestHCatExtension extends HCatMapReduceTest {
+ private static List<HCatRecord> writeRecords;
+ private static List<HCatFieldSchema> dataColumns;
+ protected static final int NUM_RECORDS = 20;
+ protected static final int NUM_TOP_PARTITIONS = 5;
+
+ public TestHCatExtension(String formatName, String serdeClass, String inputFormatClass, String outputFormatClass) throws Exception {
+ super(formatName, serdeClass, inputFormatClass, outputFormatClass);
+ tableName = "testHCatExtension_" + formatName;
+ generateWriteRecords(NUM_RECORDS, NUM_TOP_PARTITIONS, 0);
+ generateDataColumns();
+ }
+
+ protected static void generateDataColumns() throws HCatException {
+ dataColumns = new ArrayList<HCatFieldSchema>();
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("c1", serdeConstants.INT_TYPE_NAME, "")));
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("c2", serdeConstants.STRING_TYPE_NAME, "")));
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("p1", serdeConstants.STRING_TYPE_NAME, "")));
+ dataColumns.add(HCatSchemaUtils.getHCatFieldSchema(new FieldSchema("p2", serdeConstants.STRING_TYPE_NAME, "")));
+
+ }
+
+ protected static void generateWriteRecords(int max, int mod, int offset) {
+ writeRecords = new ArrayList<HCatRecord>();
+
+ for (int i = 0; i < max; i++) {
+ List<Object> objList = new ArrayList<Object>();
+
+ objList.add(i);
+ objList.add("strvalue" + i);
+ objList.add(String.valueOf((i % mod) + offset));
+ objList.add(String.valueOf((i / (max / 2)) + offset));
+ writeRecords.add(new DefaultHCatRecord(objList));
+ }
+ }
+
+ @Override
+ protected List<FieldSchema> getPartitionKeys() {
+ List<FieldSchema> fields = new ArrayList<FieldSchema>();
+ fields.add(new FieldSchema("p1", serdeConstants.STRING_TYPE_NAME, ""));
+ fields.add(new FieldSchema("p2", serdeConstants.STRING_TYPE_NAME, ""));
+ return fields;
+ }
+
+ @Override
+ protected List<FieldSchema> getTableColumns() {
+ List<FieldSchema> fields = new ArrayList<FieldSchema>();
+ fields.add(new FieldSchema("c1", serdeConstants.INT_TYPE_NAME, ""));
+ fields.add(new FieldSchema("c2", serdeConstants.STRING_TYPE_NAME, ""));
+ return fields;
+ }
+
+ @Test
+ public void testHCatExtension() throws Exception {
+ HashMap<String, String> properties = new HashMap<String, String>();
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #5355:
URL: https://github.com/apache/hive/pull/5355#issuecomment-2251374525
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5355) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5355&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5355&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5355&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5355&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5355&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5355)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #5355:
URL: https://github.com/apache/hive/pull/5355#issuecomment-2357042417
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5355) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5355&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5355&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5355&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5355&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5355&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5355)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "difin (via GitHub)" <gi...@apache.org>.
difin commented on code in PR #5355:
URL: https://github.com/apache/hive/pull/5355#discussion_r1762084959
##########
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/HCatMapReduceTest.java:
##########
@@ -282,7 +282,7 @@ public void map(WritableComparable key, HCatRecord value, Context context)
Job runMRCreate(Map<String, String> partitionValues, List<HCatFieldSchema> partitionColumns,
List<HCatRecord> records, int writeCount, boolean assertWrite) throws Exception {
return runMRCreate(partitionValues, partitionColumns, records, writeCount, assertWrite,
- true, null);
+ true, new HashMap<String, String>());
Review Comment:
Can `Collections.emptyMap()` be used instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "yigress (via GitHub)" <gi...@apache.org>.
yigress commented on PR #5355:
URL: https://github.com/apache/hive/pull/5355#issuecomment-2357140138
> @yigress So for my implementation I just tried to match what Hive does in its existing implementation. If a sequential job wants to have different extensions for different tables, it can simply adjust the setting between HCat queries. Does Hive do something other than that to support that case that I missed?
in hive the property is only used for text file format https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L916
+1 for this useful feature, if a single job only loads into one table and the table format is known user can switch the values for the property between jobs. I am not expert in pig, somehow I have the impression that one pig job can load into multiple tables simultaneously, then user needs to be careful about setting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on code in PR #5355:
URL: https://github.com/apache/hive/pull/5355#discussion_r1826558278
##########
hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/FileOutputFormatContainer.java:
##########
@@ -97,8 +97,9 @@ public RecordWriter<WritableComparable<?>, HCatRecord> getRecordWriter(TaskAttem
(org.apache.hadoop.mapred.RecordWriter)null, context);
} else {
Path parentDir = new Path(context.getConfiguration().get("mapred.work.output.dir"));
+ String extension = HiveConf.getVar(context.getConfiguration(), HiveConf.ConfVars.OUTPUT_FILE_EXTENSION,"");
Review Comment:
nit, space
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] HIVE-28377: Add support for hive.output.file.extension to HCatStorer [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ merged PR #5355:
URL: https://github.com/apache/hive/pull/5355
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org