You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Bolke de Bruin (JIRA)" <ji...@apache.org> on 2015/08/12 21:23:45 UTC

[jira] [Created] (SPARK-9887) After recent hive patches PySpark fails with IllegalArgumentException: Wrong FS: hdfs:

Bolke de Bruin created SPARK-9887:
-------------------------------------

             Summary: After recent hive patches PySpark fails with IllegalArgumentException: Wrong FS: hdfs:
                 Key: SPARK-9887
                 URL: https://issues.apache.org/jira/browse/SPARK-9887
             Project: Spark
          Issue Type: Bug
          Components: SQL, PySpark
    Affects Versions: 1.5.0
         Environment: Yarn Kerberos hive 1.2.1
            Reporter: Bolke de Bruin


After the recent hive patches (spark build 2 days ago) PySpark fails with below error. We confirmed it is working with spark before these patches with equal configurations. 

hc.sql("CREATE TABLE %s AS SELECT * FROM name_pair_distances" % table_name)
  File "/usr/hdp/current/spark/python/lib/pyspark.zip/pyspark/sql/context.py", line 552, in sql
  File "/usr/hdp/current/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/java_gateway.py", line 538, in __call__
  File "/usr/hdp/current/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 36, in deco
  File "/usr/hdp/current/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/protocol.py", line 300, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o43.sql.
: java.lang.IllegalArgumentException: Wrong FS: hdfs://hdpnlcb/apps/hive/warehouse/wcs.db/target_table/.hive-staging_hive_2015-08-12_20-27-48_357_3158824749894776248-1/-ext-10000/part-00000, expected: file:///




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org