You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Jonathan Hsieh (Updated) (JIRA)" <ji...@apache.org> on 2011/11/29 22:41:40 UTC
[jira] [Updated] (HADOOP-7870) fix SequenceFile#createWriter with
boolean createParent arg to respect createParent.
[ https://issues.apache.org/jira/browse/HADOOP-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HADOOP-7870:
-----------------------------------
Description:
After HBASE-6840, one set of calls to createNonRecursive(...) seems fishy - the new boolean createParent variable from the signature isn't used at all.
{code}
+ public static Writer
+ createWriter(FileSystem fs, Configuration conf, Path name,
+ Class keyClass, Class valClass, int bufferSize,
+ short replication, long blockSize, boolean createParent,
+ CompressionType compressionType, CompressionCodec codec,
+ Metadata metadata) throws IOException {
+ if ((codec instanceof GzipCodec) &&
+ !NativeCodeLoader.isNativeCodeLoaded() &&
+ !ZlibFactory.isNativeZlibLoaded(conf)) {
+ throw new IllegalArgumentException("SequenceFile doesn't work with " +
+ "GzipCodec without native-hadoop code!");
+ }
+
+ switch (compressionType) {
+ case NONE:
+ return new Writer(conf,
+ fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+ keyClass, valClass, metadata).ownStream();
+ case RECORD:
+ return new RecordCompressWriter(conf,
+ fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+ keyClass, valClass, codec, metadata).ownStream();
+ case BLOCK:
+ return new BlockCompressWriter(conf,
+ fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+ keyClass, valClass, codec, metadata).ownStream();
+ default:
+ return null;
+ }
+ }
+
{code}
Nicolas Spiegelberg suggests changing it to
{code}
if (createParent) { use fs.createNonRecursive(); }
else { use fs.create() }
{code}
was:
After HBASE-6840, one set of calls to createNonRecursive(...) seems fishy - the new boolean createParent variable from the signature isn't used at all.
{code}
+ public static Writer
+ createWriter(FileSystem fs, Configuration conf, Path name,
+ Class keyClass, Class valClass, int bufferSize,
+ short replication, long blockSize, boolean createParent,
+ CompressionType compressionType, CompressionCodec codec,
+ Metadata metadata) throws IOException {
+ if ((codec instanceof GzipCodec) &&
+ !NativeCodeLoader.isNativeCodeLoaded() &&
+ !ZlibFactory.isNativeZlibLoaded(conf)) {
+ throw new IllegalArgumentException("SequenceFile doesn't work with " +
+ "GzipCodec without native-hadoop code!");
+ }
+
+ switch (compressionType) {
+ case NONE:
+ return new Writer(conf,
+ fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+ keyClass, valClass, metadata).ownStream();
+ case RECORD:
+ return new RecordCompressWriter(conf,
+ fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+ keyClass, valClass, codec, metadata).ownStream();
+ case BLOCK:
+ return new BlockCompressWriter(conf,
+ fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
+ keyClass, valClass, codec, metadata).ownStream();
+ default:
+ return null;
+ }
+ }
+
{code}
Nicolas Spiegelberg suggest changing it to
{code}
if (createParent) { use fs.createNonRecursive(); }
else { use fs.create() }
{code}
> fix SequenceFile#createWriter with boolean createParent arg to respect createParent.
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-7870
> URL: https://issues.apache.org/jira/browse/HADOOP-7870
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Jonathan Hsieh
>
> After HBASE-6840, one set of calls to createNonRecursive(...) seems fishy - the new boolean createParent variable from the signature isn't used at all.
> {code}
> + public static Writer
> + createWriter(FileSystem fs, Configuration conf, Path name,
> + Class keyClass, Class valClass, int bufferSize,
> + short replication, long blockSize, boolean createParent,
> + CompressionType compressionType, CompressionCodec codec,
> + Metadata metadata) throws IOException {
> + if ((codec instanceof GzipCodec) &&
> + !NativeCodeLoader.isNativeCodeLoaded() &&
> + !ZlibFactory.isNativeZlibLoaded(conf)) {
> + throw new IllegalArgumentException("SequenceFile doesn't work with " +
> + "GzipCodec without native-hadoop code!");
> + }
> +
> + switch (compressionType) {
> + case NONE:
> + return new Writer(conf,
> + fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
> + keyClass, valClass, metadata).ownStream();
> + case RECORD:
> + return new RecordCompressWriter(conf,
> + fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
> + keyClass, valClass, codec, metadata).ownStream();
> + case BLOCK:
> + return new BlockCompressWriter(conf,
> + fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
> + keyClass, valClass, codec, metadata).ownStream();
> + default:
> + return null;
> + }
> + }
> +
> {code}
> Nicolas Spiegelberg suggests changing it to
> {code}
> if (createParent) { use fs.createNonRecursive(); }
> else { use fs.create() }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira