Spark Avro Generation

[Devl Devel <de...@gmail.com>]

Hi

So far I've been managing to build Spark from source but since a change in
spark-streaming-flume I have no idea how to generate classes (e.g.
SparkFlumeProtocol) from the avro schema.

I have used sbt to run avro:generate (from the top level spark dir) but it
produces nothing - it just says:

> avro:generate
[success] Total time: 0 s, completed Aug 11, 2014 12:26:49 PM.

Please can someone send me their build.sbt or just tell me how to build
spark so that all avro files get generated as well?

Sorry for the noob question but I really have tried by best on this one!
Cheers

Re: Spark Avro Generation

[Devl Devel <de...@gmail.com>]

Thanks very much that helps, not having to generate the entire build.


On Mon, Aug 11, 2014 at 6:09 PM, Ron Gonzalez <zl...@yahoo.com> wrote:

> If you don't want to build the entire thing, you can also do
>
> mvn generate-sources in externals/flume-sink
>
> Thanks,
> Ron
>
> Sent from my iPhone
>
> > On Aug 11, 2014, at 8:32 AM, Hari Shreedharan <hs...@cloudera.com>
> wrote:
> >
> > Jay running sbt compile or assembly should generate the sources.
> >
> >> On Monday, August 11, 2014, Devl Devel <de...@gmail.com>
> wrote:
> >>
> >> Hi
> >>
> >> So far I've been managing to build Spark from source but since a change
> in
> >> spark-streaming-flume I have no idea how to generate classes (e.g.
> >> SparkFlumeProtocol) from the avro schema.
> >>
> >> I have used sbt to run avro:generate (from the top level spark dir) but
> it
> >> produces nothing - it just says:
> >>
> >>> avro:generate
> >> [success] Total time: 0 s, completed Aug 11, 2014 12:26:49 PM.
> >>
> >> Please can someone send me their build.sbt or just tell me how to build
> >> spark so that all avro files get generated as well?
> >>
> >> Sorry for the noob question but I really have tried by best on this one!
> >> Cheers
> >>
>

Re: Spark Avro Generation

[Ron Gonzalez <zl...@yahoo.com.INVALID>]

If you don't want to build the entire thing, you can also do

mvn generate-sources in externals/flume-sink

Thanks,
Ron

Sent from my iPhone

> On Aug 11, 2014, at 8:32 AM, Hari Shreedharan <hs...@cloudera.com> wrote:
> 
> Jay running sbt compile or assembly should generate the sources.
> 
>> On Monday, August 11, 2014, Devl Devel <de...@gmail.com> wrote:
>> 
>> Hi
>> 
>> So far I've been managing to build Spark from source but since a change in
>> spark-streaming-flume I have no idea how to generate classes (e.g.
>> SparkFlumeProtocol) from the avro schema.
>> 
>> I have used sbt to run avro:generate (from the top level spark dir) but it
>> produces nothing - it just says:
>> 
>>> avro:generate
>> [success] Total time: 0 s, completed Aug 11, 2014 12:26:49 PM.
>> 
>> Please can someone send me their build.sbt or just tell me how to build
>> spark so that all avro files get generated as well?
>> 
>> Sorry for the noob question but I really have tried by best on this one!
>> Cheers
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org

Re: Spark Avro Generation

[Hari Shreedharan <hs...@cloudera.com>]

Jay running sbt compile or assembly should generate the sources.

On Monday, August 11, 2014, Devl Devel <de...@gmail.com> wrote:

> Hi
>
> So far I've been managing to build Spark from source but since a change in
> spark-streaming-flume I have no idea how to generate classes (e.g.
> SparkFlumeProtocol) from the avro schema.
>
> I have used sbt to run avro:generate (from the top level spark dir) but it
> produces nothing - it just says:
>
> > avro:generate
> [success] Total time: 0 s, completed Aug 11, 2014 12:26:49 PM.
>
> Please can someone send me their build.sbt or just tell me how to build
> spark so that all avro files get generated as well?
>
> Sorry for the noob question but I really have tried by best on this one!
> Cheers
>