[jira] [Created] (HADOOP-13970) garbage data read from the beginning of a tar file

["Steve Yang (JIRA)" <ji...@apache.org>]

Steve Yang created HADOOP-13970:
-----------------------------------

             Summary: garbage data read from the beginning of a tar file
                 Key: HADOOP-13970
                 URL: https://issues.apache.org/jira/browse/HADOOP-13970
             Project: Hadoop Common
          Issue Type: Bug
          Components: common
    Affects Versions: 2.6.0
         Environment: Linux EL6
            Reporter: Steve Yang


on Spark using databricks ('com.databricks:spark-csv_2.10:1.5.0') to read in a tar file which consists of 3 .csv files. 
sqlCtx.read().format("com.databricks.spark.csv").option(...)
.load(objectName);
The tar file contains 3 files:
taxi_simplified1.csv
taxi2.csv
simplified3.csv
where the first line (header) is:
trip_distance,dropoff_datetime,dropoff_geocode,passenger_count,medallion,rate_code,tip_amount,total_amount,store_and_fwd_flag,mta_tax,pickup_geocode,trip_time_in_secs,surcharge,vendor_id,tolls_amount,fare_amount,pickup_datetime,hack_license,payment_type,ordertime

Note the first column header is "trip_distance". But the read data shows:
taxi_simplified1.csv^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@0000644^@0010013^@3001121^@00000046004^@13002371150^@013521^@ 0^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ustar  ^@optitest^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@trip_distance




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org