You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Davies Liu (JIRA)" <ji...@apache.org> on 2015/08/13 19:59:46 UTC
[jira] [Created] (SPARK-9942) Broken pandas could crash PySpark SQL
Davies Liu created SPARK-9942:
---------------------------------
Summary: Broken pandas could crash PySpark SQL
Key: SPARK-9942
URL: https://issues.apache.org/jira/browse/SPARK-9942
Project: Spark
Issue Type: Bug
Components: PySpark, SQL
Affects Versions: 1.4.1, 1.3.1, 1.5.0
Reporter: Davies Liu
Assignee: Davies Liu
Priority: Blocker
{code}
Error from python worker:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 151, in _run_module_as_main
mod_name, loader, code, fname = _get_module_details(mod_name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 101, in _get_module_details
loader = get_loader(mod_name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pkgutil.py", line 430, in iter_importers
__import__(pkg)
File "/Users/davies/work/spark/python/pyspark/__init__.py", line 52, in <module>
from pyspark.sql import SQLContext, HiveContext, SchemaRDD, Row
File "/Users/davies/work/spark/python/pyspark/sql/__init__.py", line 63, in <module>
from pyspark.sql.context import SQLContext, HiveContext
File "/Users/davies/work/spark/python/pyspark/sql/context.py", line 40, in <module>
import pandas
File "/Library/Python/2.7/site-packages/pandas/__init__.py", line 7, in <module>
from pandas import hashtable, tslib, lib
File "pandas/src/numpy.pxd", line 157, in init pandas.hashtable (pandas/hashtable.c:23654)
ValueError: numpy.dtype has the wrong size, try recompiling
PYTHONPATH was:
/Users/davies/work/spark/python/lib/pyspark.zip:/Users/davies/work/spark/python/lib/py4j-0.8.2.1-src.zip:/Users/davies/work/spark/sbin/../python/lib/py4j-0.8.2.1-src.zip:/Users/davies/work/spark/sbin/../python:/Users/davies/work/spark/sbin/../python/lib/py4j-0.8.2.1-src.zip:/Users/davies/work/spark/sbin/../python:
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org