You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Ran Mingxuan (JIRA)" <ji...@apache.org> on 2017/04/26 08:31:04 UTC
[jira] [Resolved] (SQOOP-3176) decompressing error while load 'gz'
and 'bz2' data into table
[ https://issues.apache.org/jira/browse/SQOOP-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ran Mingxuan resolved SQOOP-3176.
---------------------------------
Resolution: Won't Fix
I opened a wrong ticket.
> decompressing error while load 'gz' and 'bz2' data into table
> -------------------------------------------------------------
>
> Key: SQOOP-3176
> URL: https://issues.apache.org/jira/browse/SQOOP-3176
> Project: Sqoop
> Issue Type: Bug
> Environment: spark 2.1.0
> Reporter: Ran Mingxuan
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> import org.apache.spark.sql.SparkSession
> import org.apache.spark.sql.CarbonSession._
> val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://nsha/user/ranmx/test/carbon")
> carbon.sql("CREATE TABLE IF NOT EXISTS test_table(id string, name string, city string, age Int) STORED BY 'carbondata'")
> carbon.sql("LOAD DATA inpath '/ranmx/test/sh.csv.bz2' INTO TABLE test_table1")
> 会报错：
> 17/04/26 11:11:26 ERROR LoadTable: main
> java.lang.NullPointerException
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.isNativeBzip2Loaded(Bzip2Factory.java:54)
> at org.apache.hadoop.io.compress.bzip2.Bzip2Factory.getBzip2DecompressorType(Bzip2Factory.java:120)
> at org.apache.hadoop.io.compress.BZip2Codec.getDecompressorType(BZip2Codec.java:242)
> at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:176)
> at org.apache.hadoop.io.compress.CompressionCodec$Util.createInputStreamWithCodecPool(CompressionCodec.java:157)
> at org.apache.hadoop.io.compress.BZip2Codec.createInputStream(BZip2Codec.java:157)
> at org.apache.carbondata.core.datastore.impl.FileFactory.getDataInputStream(FileFactory.java:139)
> at org.apache.carbondata.core.datastore.impl.FileFactory.getDataInputStream(FileFactory.java:104)
> at org.apache.carbondata.core.util.CarbonUtil.readHeader(CarbonUtil.java:1273)
> at org.apache.carbondata.spark.util.CommonUtil$.getCsvHeaderColumns(CommonUtil.scala:319)
> at org.apache.spark.sql.execution.command.LoadTable.run(carbonTableSchema.scala:474)
> ...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)