You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by jo...@apache.org on 2015/10/18 20:39:28 UTC
spark git commit: [SPARK-11158][SQL] Modified _verify_type() to be
more informative on Errors by presenting the Object
Repository: spark
Updated Branches:
refs/heads/master 8d4449c7f -> a337c235a
[SPARK-11158][SQL] Modified _verify_type() to be more informative on Errors by presenting the Object
The _verify_type() function had Errors that were raised when there were Type conversion issues but left out the Object in question. The Object is now added in the Error to reduce the strain on the user to debug through to figure out the Object that failed the Type conversion.
The use case for me was a Pandas DataFrame that contained 'nan' as values for columns of Strings.
Author: Mahmoud Lababidi <ma...@thehumangeo.com>
Author: Mahmoud Lababidi <la...@gmail.com>
Closes #9149 from lababidi/master.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a337c235
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a337c235
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a337c235
Branch: refs/heads/master
Commit: a337c235a12d4ea6a7d6db457acc6b32f1915241
Parents: 8d4449c
Author: Mahmoud Lababidi <ma...@thehumangeo.com>
Authored: Sun Oct 18 11:39:19 2015 -0700
Committer: Josh Rosen <jo...@databricks.com>
Committed: Sun Oct 18 11:39:19 2015 -0700
----------------------------------------------------------------------
python/pyspark/sql/types.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/a337c235/python/pyspark/sql/types.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py
index 1f86894..5bc0773 100644
--- a/python/pyspark/sql/types.py
+++ b/python/pyspark/sql/types.py
@@ -1127,15 +1127,15 @@ def _verify_type(obj, dataType):
return
_type = type(dataType)
- assert _type in _acceptable_types, "unknown datatype: %s" % dataType
+ assert _type in _acceptable_types, "unknown datatype: %s for object %r" % (dataType, obj)
if _type is StructType:
if not isinstance(obj, (tuple, list)):
- raise TypeError("StructType can not accept object in type %s" % type(obj))
+ raise TypeError("StructType can not accept object %r in type %s" % (obj, type(obj)))
else:
# subclass of them can not be fromInternald in JVM
if type(obj) not in _acceptable_types[_type]:
- raise TypeError("%s can not accept object in type %s" % (dataType, type(obj)))
+ raise TypeError("%s can not accept object %r in type %s" % (dataType, obj, type(obj)))
if isinstance(dataType, ArrayType):
for i in obj:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org