You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yingchun Lai (Code Review)" <ge...@cloudera.org> on 2019/07/26 04:02:34 UTC
[kudu-CR] [docs] fix spark integration examples
Yingchun Lai has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13927
Change subject: [docs] fix spark integration examples
......................................................................
[docs] fix spark integration examples
Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
---
M docs/developing.adoc
1 file changed, 14 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/13927/1
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
[kudu-CR] [docs] fix spark integration examples
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins, Priyanka Chheda, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13927
to look at the new patch set (#3).
Change subject: [docs] fix spark integration examples
......................................................................
[docs] fix spark integration examples
Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
---
M docs/developing.adoc
1 file changed, 20 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/13927/3
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Priyanka Chheda <pc...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [docs] fix spark integration examples
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc
File docs/developing.adoc:
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc@112
PS1, Line 112: Below is a minimal Spark SQL "select" example. We first import the kudu spark package,
> You don't need to do this, but this could be a good opportunity to link to
Done
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc@116
PS1, Line 116: NOTE: You can find link:https://github.com/apache/kudu/tree/master/examples/quickstart/spark[quickstart]
> Unfortunately this table will haven a random name instead of `default.my_ta
The second sample shows how Spark code creates a table, for the first minimal simple, it's not needed to do that in my opinion.
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc@124
PS1, Line 124: import org.apache.kudu.spark.kudu._
> Why reduce this to a single master? Given multi-maser is a more common prod
I just want to keep the same style with the following two samples.
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Fri, 16 Aug 2019 07:10:04 +0000
Gerrit-HasComments: Yes
[kudu-CR] [docs] fix spark integration examples
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc
File docs/developing.adoc:
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@116
PS2, Line 116: NOTE: There is also a Spark link:https://github.com/apache/kudu/tree/master/examples/quickstart/spark[quickstart]
> Maybe adjust a little to the below (with the links).
Done
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@117
PS2, Line 117: ple]
> nit: another example ?
Done
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@119
PS2, Line 119:
> use the Kudu
Done
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@189
PS2, Line 189: .options(Map("kudu.master"-> "kudu.master:7051", "kudu.table"-> "test_table"))
: .mode("append")
> nit: maybe, move this just after 'kuduContext.createTable()'
Done
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Priyanka Chheda <pc...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Mon, 02 Sep 2019 15:44:57 +0000
Gerrit-HasComments: Yes
[kudu-CR] [docs] fix spark integration examples
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Priyanka Chheda <pc...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Wed, 04 Sep 2019 01:31:39 +0000
Gerrit-HasComments: No
[kudu-CR] [docs] fix spark integration examples
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 1:
preview; https://github.com/acelyc111/kudu/blob/fdf6acd7ee682bf74bf72c5eac2de04c4e6c962b/docs/developing.adoc
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Fri, 26 Jul 2019 04:03:37 +0000
Gerrit-HasComments: No
[kudu-CR] [docs] fix spark integration examples
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13927
to look at the new patch set (#2).
Change subject: [docs] fix spark integration examples
......................................................................
[docs] fix spark integration examples
Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
---
M docs/developing.adoc
1 file changed, 17 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/13927/2
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [docs] fix spark integration examples
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc
File docs/developing.adoc:
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@117
PS2, Line 117: example
nit: another example ?
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@189
PS2, Line 189: // Check for the existence of a Kudu table
: kuduContext.tableExists("test_table")
nit: maybe, move this just after 'kuduContext.createTable()'
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Priyanka Chheda <pc...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Thu, 29 Aug 2019 17:28:20 +0000
Gerrit-HasComments: Yes
[kudu-CR] [docs] fix spark integration examples
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
[docs] fix spark integration examples
Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Reviewed-on: http://gerrit.cloudera.org:8080/13927
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Reviewed-by: Grant Henke <gr...@apache.org>
---
M docs/developing.adoc
1 file changed, 20 insertions(+), 13 deletions(-)
Approvals:
Kudu Jenkins: Verified
Alexey Serbin: Looks good to me, approved
Grant Henke: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 4
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Priyanka Chheda <pc...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
[kudu-CR] [docs] fix spark integration examples
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Priyanka Chheda <pc...@cloudera.com>
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Tue, 03 Sep 2019 21:25:46 +0000
Gerrit-HasComments: No
[kudu-CR] [docs] fix spark integration examples
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc
File docs/developing.adoc:
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc@112
PS1, Line 112: Below is a minimal Spark SQL "select" example. We first import the kudu spark package,
You don't need to do this, but this could be a good opportunity to link to the Spark quickstart:
https://github.com/apache/kudu/tree/master/examples/quickstart/spark
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc@116
PS1, Line 116: NOTE: You can use Kudu CLI tool to create table and generate data by
Unfortunately this table will haven a random name instead of `default.my_table`. Can we instead show Spark code that creates the table?
http://gerrit.cloudera.org:8080/#/c/13927/1/docs/developing.adoc@124
PS1, Line 124: val df = spark.read.options(Map("kudu.master" -> "kudu.master:7051",
Why reduce this to a single master? Given multi-maser is a more common production scenario, I think it's good to show 3.
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Mon, 12 Aug 2019 17:58:55 +0000
Gerrit-HasComments: Yes
[kudu-CR] [docs] fix spark integration examples
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 2:
(2 comments)
Just a couple small adjustments and this should be good.
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc
File docs/developing.adoc:
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@116
PS2, Line 116: NOTE: You can find link:https://github.com/apache/kudu/tree/master/examples/quickstart/spark[quickstart]
Maybe adjust a little to the below (with the links).
NOTE: There is also a Spark quickstart guide and a full Spark example available.
http://gerrit.cloudera.org:8080/#/c/13927/2/docs/developing.adoc@119
PS2, Line 119: use Kudu
use the Kudu
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 2
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Wed, 28 Aug 2019 15:19:45 +0000
Gerrit-HasComments: Yes
[kudu-CR] [docs] fix spark integration examples
Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/13927 )
Change subject: [docs] fix spark integration examples
......................................................................
Patch Set 1:
I'm not familiar with spark, however, I have run to check these examples in my environment, it works OK.
--
To view, visit http://gerrit.cloudera.org:8080/13927
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a48cc34aea3cc42afd48f43e142a669081f14e2
Gerrit-Change-Number: 13927
Gerrit-PatchSet: 1
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>
Gerrit-Comment-Date: Fri, 26 Jul 2019 04:06:13 +0000
Gerrit-HasComments: No