You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Something Something <ma...@gmail.com> on 2015/10/12 23:03:47 UTC

Storing object in spark streaming

In my custom receiver for Spark Streaming I've code such as this:

                messages.toArray().foreach(msg => {
                  val m = msg.asInstanceOf[Message]
*                  store(m.getBody)*
                })

Instead of 'body' which is of type 'String', I would rather pass the entire
Message object, but when I say store(m), I get a compiler error saying:
"Cannot resolve reference store with such signature"

But I see this method in 'Receiver.scala':

  def store(dataItem: T) {
    executor.pushSingle(dataItem)
  }


How do I store the entire object? Please help. Thanks.

Re: Storing object in spark streaming

Posted by Jeff Nadler <jn...@srcginc.com>.
Your receiver must extend Receiver[String].    Try changing it to extend
Receiver[Message]?

On Mon, Oct 12, 2015 at 2:03 PM, Something Something <
mailinglists19@gmail.com> wrote:

> In my custom receiver for Spark Streaming I've code such as this:
>
>                 messages.toArray().foreach(msg => {
>                   val m = msg.asInstanceOf[Message]
> *                  store(m.getBody)*
>                 })
>
> Instead of 'body' which is of type 'String', I would rather pass the
> entire Message object, but when I say store(m), I get a compiler error
> saying: "Cannot resolve reference store with such signature"
>
> But I see this method in 'Receiver.scala':
>
>   def store(dataItem: T) {
>     executor.pushSingle(dataItem)
>   }
>
>
> How do I store the entire object? Please help. Thanks.
>
>
>
>