You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Smruthi Rajmohan (JIRA)" <ji...@apache.org> on 2017/07/19 05:26:00 UTC
[jira] [Created] (SPARK-21466) com.cloudant.spark throws an error
in python notebook
Smruthi Rajmohan created SPARK-21466:
----------------------------------------
Summary: com.cloudant.spark throws an error in python notebook
Key: SPARK-21466
URL: https://issues.apache.org/jira/browse/SPARK-21466
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.1.1, 2.0.0
Environment: Data Science Experience- Python Notebook version-2
Reporter: Smruthi Rajmohan
When I try to establish a connection with cloudant. I get an error-
Py4JJavaErrorTraceback (most recent call last)
<ipython-input-1-503b9b8606de> in <module>()
1 import pyspark
2 sqlContext = SQLContext(sc)
----> 3 cloudantdata = sqlContext.read.format("com.cloudant.spark").option("cloudant.host","3d1a8ae1-9d67-4859-a3c4-8fed8d7548db-bluemix.cloudant.com").option("cloudant.username", "3d1a8ae1-9d67-4859-a3c4-8fed8d7548db-bluemix").option("cloudant.password","ccf3f8349ffb5964513973f90223f6178b41c2a50345b1120f0322466eb14ba9").load("iotp_4wkoz2_default_2017-07-17")
/usr/local/src/spark20master/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
145 self.options(**options)
146 if isinstance(path, basestring):
--> 147 return self._df(self._jreader.load(path))
148 elif path is not None:
149 if type(path) != list:
/usr/local/src/spark20master/spark/python/lib/py4j-0.10.3-src.zip/py4j/java_gateway.py in __call__(self, *args)
1131 answer = self.gateway_client.send_command(command)
1132 return_value = get_return_value(
-> 1133 answer, self.gateway_client, self.target_id, self.name)
1134
1135 for temp_arg in temp_args:
/usr/local/src/spark20master/spark/python/pyspark/sql/utils.py in deco(*a, **kw)
61 def deco(*a, **kw):
62 try:
---> 63 return f(*a, **kw)
64 except py4j.protocol.Py4JJavaError as e:
65 s = e.java_exception.toString()
/usr/local/src/spark20master/spark/python/lib/py4j-0.10.3-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
317 raise Py4JJavaError(
318 "An error occurred while calling {0}{1}{2}.\n".
--> 319 format(target_id, ".", name), value)
320 else:
321 raise Py4JError(
Py4JJavaError: An error occurred while calling o86.load.
: java.lang.ClassNotFoundException: Failed to find data source: com.cloudant.spark. Please find packages at https://cwiki.apache.org/confluence/display/SPARK/Third+Party+Projects
at org.apache.spark.sql.execution.datasources.DataSource.lookupDataSource(DataSource.scala:148)
at org.apache.spark.sql.execution.datasources.DataSource.providingClass$lzycompute(DataSource.scala:79)
at org.apache.spark.sql.execution.datasources.DataSource.providingClass(DataSource.scala:79)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:340)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:149)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:507)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:785)
Caused by: java.lang.ClassNotFoundException: com.cloudant.spark.DefaultSource
at java.net.URLClassLoader.findClass(URLClassLoader.java:607)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:844)
at java.lang.ClassLoader.loadClass(ClassLoader.java:823)
at java.lang.ClassLoader.loadClass(ClassLoader.java:803)
at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5$$anonfun$apply$1.apply(DataSource.scala:132)
at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5$$anonfun$apply$1.apply(DataSource.scala:132)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5.apply(DataSource.scala:132)
at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5.apply(DataSource.scala:132)
at scala.util.Try.orElse(Try.scala:84)
at org.apache.spark.sql.execution.datasources.DataSource.lookupDataSource(DataSource.scala:132)
... 16 more
This is the code for the same
sqlContext = SQLContext(sc)
cloudantdata = sqlContext.read.format("com.cloudant.spark").\
option("cloudant.host","some_hostname").\
option("cloudant.username", "some_username").\
option("cloudant.password","password").\
load("somedb")
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org