You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by dvlpr <na...@gmail.com> on 2016/06/14 07:29:09 UTC
Custom receiver to connect MySQL database
Hi folks,
I have written some codes for custom receiver to get data from MySQL db.
Belowed code:
class CustomReceiver(url: String, username: String, password: String)
extends Receiver[String](StorageLevel.MEMORY_AND_DISK_2) with Logging {
case class customer(c_sk: Int, c_add_sk: Int, c_first: String)
def onStart() {
// Start the thread that receives data over a connection
new Thread("MySQL Receiver") {
override def run() { receive() }
}.start()
}
def onStop() {
// There is nothing much to do as the thread calling receive()
// is designed to stop by itself isStopped() returns false
}
private def receive() {
Class.forName("com.mysql.jdbc.Driver").newInstance()
val con = DriverManager.getConnection(url, username, password)
}
while executing this code i am getting an error: Exception in thread "main"
java.lang.IllegalArgumentException: requirement failed: No output operations
registered, so nothing to execute
Please help me to solve my problem ?
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Custom receiver to connect MySQL database
Posted by dvlpr <na...@gmail.com>.
I have tried but it gives me an error. because something is missing in my
code
*Prasiddh Nandola*
MSc Computer Science
Technical University Kaiserslautern
+49 176 67817239
On Wed, Jun 15, 2016 at 8:05 AM, Prasiddh Nandola <nandolaprasiddh@gmail.com
> wrote:
> hii
>
> *Prasiddh Nandola*
> MSc Computer Science
> Technical University Kaiserslautern
> +49 176 67817239
>
> On Tue, Jun 14, 2016 at 9:24 PM, dvlpr [via Apache Spark Developers List]
> <ml...@n3.nabble.com> wrote:
>
>> I have tried but it gives me an error. because something is missing in my
>> code
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895p17912.html
>> To unsubscribe from Custom receiver to connect MySQL database, click here
>> <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=17895&code=bmFuZG9sYXByYXNpZGRoQGdtYWlsLmNvbXwxNzg5NXwtOTMwMTg3NTg3>
>> .
>> NAML
>> <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895p17918.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
Re: Custom receiver to connect MySQL database
Posted by dvlpr <na...@gmail.com>.
hii
*Prasiddh Nandola*
MSc Computer Science
Technical University Kaiserslautern
+49 176 67817239
On Tue, Jun 14, 2016 at 9:24 PM, dvlpr [via Apache Spark Developers List] <
ml-node+s1001551n17912h36@n3.nabble.com> wrote:
> I have tried but it gives me an error. because something is missing in my
> code
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895p17912.html
> To unsubscribe from Custom receiver to connect MySQL database, click here
> <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=17895&code=bmFuZG9sYXByYXNpZGRoQGdtYWlsLmNvbXwxNzg5NXwtOTMwMTg3NTg3>
> .
> NAML
> <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895p17917.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
Re: Custom receiver to connect MySQL database
Posted by dvlpr <na...@gmail.com>.
I have tried but it gives me an error. because something is missing in my
code
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895p17912.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: Custom receiver to connect MySQL database
Posted by Matthias Niehoff <ma...@codecentric.de>.
You must add an output operation your normal stream application that uses
the receiver. Calling print() on the DStream will do the job
2016-06-14 9:29 GMT+02:00 dvlpr <na...@gmail.com>:
> Hi folks,
> I have written some codes for custom receiver to get data from MySQL db.
> Belowed code:
>
> class CustomReceiver(url: String, username: String, password: String)
> extends Receiver[String](StorageLevel.MEMORY_AND_DISK_2) with Logging {
>
> case class customer(c_sk: Int, c_add_sk: Int, c_first: String)
>
> def onStart() {
> // Start the thread that receives data over a connection
> new Thread("MySQL Receiver") {
> override def run() { receive() }
> }.start()
> }
>
> def onStop() {
> // There is nothing much to do as the thread calling receive()
> // is designed to stop by itself isStopped() returns false
> }
>
> private def receive() {
> Class.forName("com.mysql.jdbc.Driver").newInstance()
> val con = DriverManager.getConnection(url, username, password)
> }
>
> while executing this code i am getting an error: Exception in thread "main"
> java.lang.IllegalArgumentException: requirement failed: No output
> operations
> registered, so nothing to execute
>
> Please help me to solve my problem ?
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Custom-receiver-to-connect-MySQL-database-tp17895.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
--
Matthias Niehoff | IT-Consultant | Agile Software Factory | Consulting
codecentric AG | Zeppelinstr 2 | 76185 Karlsruhe | Deutschland
tel: +49 (0) 721.9595-681 | fax: +49 (0) 721.9595-666 | mobil: +49 (0)
172.1702676
www.codecentric.de | blog.codecentric.de | www.meettheexperts.de |
www.more4fi.de
Sitz der Gesellschaft: Solingen | HRB 25917| Amtsgericht Wuppertal
Vorstand: Michael Hochgürtel . Mirko Novakovic . Rainer Vehns
Aufsichtsrat: Patric Fedlmeier (Vorsitzender) . Klaus Jäger . Jürgen Schütz
Diese E-Mail einschließlich evtl. beigefügter Dateien enthält vertrauliche
und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige
Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie
bitte sofort den Absender und löschen Sie diese E-Mail und evtl.
beigefügter Dateien umgehend. Das unerlaubte Kopieren, Nutzen oder Öffnen
evtl. beigefügter Dateien sowie die unbefugte Weitergabe dieser E-Mail ist
nicht gestattet