You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Stuart Davidson <re...@spedge.com> on 2015/07/24 10:31:52 UTC
Review Request 36768: SAMZA-740: Add ElasticsearchProducer example to
samza-hello-samza
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/
-----------------------------------------------------------
Review request for samza.
Repository: samza-hello-samza
Description
-------
I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
Diffs
-----
bin/grid-elastic PRE-CREATION
pom.xml f9c4fa9
src/main/assembly/src.xml f57fee2
src/main/config/wikipedia-elastic.properties PRE-CREATION
src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
Diff: https://reviews.apache.org/r/36768/diff/
Testing
-------
Thanks,
Stuart Davidson
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Yan Fang <ya...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review93137
-----------------------------------------------------------
I think it's ok to keep the elastic grid seperate, though there is a little redundant code. Because conceptally they are different - this one is totally an add-on, while the "main" grid is for the Samza-must-have compoenents.
bin/grid-elastic (line 53)
<https://reviews.apache.org/r/36768/#comment147398>
remove space
bin/grid-elastic (line 121)
<https://reviews.apache.org/r/36768/#comment147399>
remove space
bin/grid-elastic (line 127)
<https://reviews.apache.org/r/36768/#comment147400>
remove the space
- Yan Fang
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Stuart Davidson <re...@spedge.com>.
> On July 24, 2015, 6:18 p.m., Navina Ramesh wrote:
> > bin/grid-elastic, line 1
> > <https://reviews.apache.org/r/36768/diff/1/?file=1020797#file1020797line1>
> >
> > Can you make changes to bin/grid by adding options to install elastic_search and/or kibana? Code seems redundant.
I deliberately split it up because I didn't think that all people would want to know how to use Elasticsearch - thus, not everyone would want to install it. I really don't think we should combine the two - most of hello-samza talks about the core components of using Samza, whereas this example doesn't.
Are you sure you want me to combine the two?
> On July 24, 2015, 6:18 p.m., Navina Ramesh wrote:
> > pom.xml, line 116
> > <https://reviews.apache.org/r/36768/diff/1/?file=1020798#file1020798line116>
> >
> > You won't need this if you apply your patch to the latest branch in samza-hello-samza.
So do you want me to get rid of this change? Without 0.10.0 (and above), this patch won't work - thus the deliberate change to force the upgrade.
- Stuart
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92951
-----------------------------------------------------------
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Navina Ramesh <nr...@linkedin.com>.
> On July 24, 2015, 6:18 p.m., Navina Ramesh wrote:
> > bin/grid-elastic, line 1
> > <https://reviews.apache.org/r/36768/diff/1/?file=1020797#file1020797line1>
> >
> > Can you make changes to bin/grid by adding options to install elastic_search and/or kibana? Code seems redundant.
>
> Stuart Davidson wrote:
> I deliberately split it up because I didn't think that all people would want to know how to use Elasticsearch - thus, not everyone would want to install it. I really don't think we should combine the two - most of hello-samza talks about the core components of using Samza, whereas this example doesn't.
>
> Are you sure you want me to combine the two?
Ok. That makes sense :) Let's keep it separate.
- Navina
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92951
-----------------------------------------------------------
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92951
-----------------------------------------------------------
bin/grid-elastic (line 1)
<https://reviews.apache.org/r/36768/#comment147209>
Can you make changes to bin/grid by adding options to install elastic_search and/or kibana? Code seems redundant.
pom.xml (line 116)
<https://reviews.apache.org/r/36768/#comment147210>
You won't need this if you apply your patch to the latest branch in samza-hello-samza.
I believe ElasticsearchProducer is going to be part of the 0.10 release in Samza. So, this patch should be against the "latest" branch in hello-samza. This diff looks like it is against the master branch. Can you please make the changes against latest branch ?
- Navina Ramesh
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Navina Ramesh <nr...@linkedin.com>.
> On July 24, 2015, 6:19 p.m., Navina Ramesh wrote:
> > It will also be nice to have a tutorial added to the website on how to setup and run the example Samza job that uses ElasticSearchProducer.
>
> Stuart Davidson wrote:
> Happy to write something up. Do you want it on the hello-samza index.md, or a seperate page?
You can create a web page and add it to the tutorial list here - http://samza.apache.org/learn/tutorials/latest/ Thanks!
- Navina
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92953
-----------------------------------------------------------
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Stuart Davidson <re...@spedge.com>.
> On July 24, 2015, 6:19 p.m., Navina Ramesh wrote:
> > It will also be nice to have a tutorial added to the website on how to setup and run the example Samza job that uses ElasticSearchProducer.
Happy to write something up. Do you want it on the hello-samza index.md, or a seperate page?
- Stuart
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92953
-----------------------------------------------------------
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Stuart Davidson <re...@spedge.com>.
> On July 24, 2015, 6:19 p.m., Navina Ramesh wrote:
> > It will also be nice to have a tutorial added to the website on how to setup and run the example Samza job that uses ElasticSearchProducer.
>
> Stuart Davidson wrote:
> Happy to write something up. Do you want it on the hello-samza index.md, or a seperate page?
>
> Navina Ramesh wrote:
> You can create a web page and add it to the tutorial list here - http://samza.apache.org/learn/tutorials/latest/ Thanks!
Sorry about the delay - will try and write something up soon.
- Stuart
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92953
-----------------------------------------------------------
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>
Re: Review Request 36768: SAMZA-740: Add ElasticsearchProducer
example to samza-hello-samza
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36768/#review92953
-----------------------------------------------------------
It will also be nice to have a tutorial added to the website on how to setup and run the example Samza job that uses ElasticSearchProducer.
- Navina Ramesh
On July 24, 2015, 8:31 a.m., Stuart Davidson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36768/
> -----------------------------------------------------------
>
> (Updated July 24, 2015, 8:31 a.m.)
>
>
> Review request for samza.
>
>
> Repository: samza-hello-samza
>
>
> Description
> -------
>
> I'd put together an example of how to use the new ElasticsearchProvider for my work colleagues and I thought it'd be worth submitting it back to the community. This also includes a script to start elasticsearch and kibana on the host - be aware, there's a 64bit version depending on what machine you want to run against.
>
> Also note, this is against 0.10.0 of Samza which is not released yet. I am making the assumption that the Elasticsearch jars are bundled as part of that - if not, we'll need to add them to the POM here.
>
>
> Diffs
> -----
>
> bin/grid-elastic PRE-CREATION
> pom.xml f9c4fa9
> src/main/assembly/src.xml f57fee2
> src/main/config/wikipedia-elastic.properties PRE-CREATION
> src/main/java/samza/examples/wikipedia/task/WikipediaElasticStreamTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/36768/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Stuart Davidson
>
>