You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Woojin Joe <jw...@gmail.com> on 2019/11/01 08:58:16 UTC
[Suggestion] Add a configuration property for LivySessionController
Hi, All
I'm Woojin Joe, a data analytics engineer in Lineplus.
Our team is trying to test Apache Nifi for the data ingestion pipeline
integrated with the Spark cluster.
I found that LivySessionController would be proper to integrate between
Nifi and Spark through Livy.
But there is none property to include `conf` for creating a session through
it.
Thus if possible, I'd like to include `conf` property as a JSON format in
LivySessionController.
IMO, it would take very trivial changes.
For example, just creating a new property descriptor and append `conf` in
the body when it is set.
[image: screenshot_code_diff.png]
I hope to contribute this feature to the current code base if possible.
Warm regards,
Woojin joe
----------------------------------------------------------------------
[image: --]
Woojin Joe
[image: http://]about.me/notforme
<http://about.me/notforme?promo=email_sig>
Woojin Joe (@JoeWoojin <https://twitter.com/JoeWooJin>, Github
<https://github.com/not-for-me>, LinkedIn
<https://www.linkedin.com/in/notforme>)
----------------------------------------------------------------------
Re: [Suggestion] Add a configuration property for LivySessionController
Posted by Woojin Joe <jw...@gmail.com>.
Thank you for letting me know.
I'll catch up the PR.
----------------------------------------------------------------------
[image: --]
Woojin Joe
[image: http://]about.me/notforme
<http://about.me/notforme?promo=email_sig>
Woojin Joe (@JoeWoojin <https://twitter.com/JoeWooJin>, Github
<https://github.com/not-for-me>, LinkedIn
<https://www.linkedin.com/in/notforme>)
----------------------------------------------------------------------
2019년 11월 1일 (금) 오후 7:59, Otto Fowler <ot...@gmail.com>님이 작성:
> You may want to look at https://github.com/apache/nifi/pull/3511 which is
> open under review now and comment there. Maybe you can even try it and
> review with feedback.
>
>
>
>
> On November 1, 2019 at 05:16:58, Woojin Joe (jwj0831@gmail.com) wrote:
>
> Hi, All
> I'm Woojin Joe, a data analytics engineer in Lineplus.
> Our team is trying to test Apache Nifi for the data ingestion pipeline
> integrated with the Spark cluster.
>
> I found that LivySessionController would be proper to integrate between
> Nifi and Spark through Livy.
> But there is none property to include `conf` for creating a session through
> it.
>
> Thus if possible, I'd like to include `conf` property as a JSON format in
> LivySessionController.
>
> IMO, it would take very trivial changes.
> For example, just creating a new property descriptor and append `conf` in
> the body when it is set.
>
> [image: screenshot_code_diff.png]
>
> I hope to contribute this feature to the current code base if possible.
>
> Warm regards,
> Woojin joe
>
> ----------------------------------------------------------------------
>
>
> [image: --]
> Woojin Joe
> [image: http://]about.me/notforme
> <http://about.me/notforme?promo=email_sig>
>
> Woojin Joe (@JoeWoojin <https://twitter.com/JoeWooJin>, Github
> <https://github.com/not-for-me>, LinkedIn
> <https://www.linkedin.com/in/notforme>)
> ----------------------------------------------------------------------
>
Re: [Suggestion] Add a configuration property for LivySessionController
Posted by Otto Fowler <ot...@gmail.com>.
You may want to look at https://github.com/apache/nifi/pull/3511 which is
open under review now and comment there. Maybe you can even try it and
review with feedback.
On November 1, 2019 at 05:16:58, Woojin Joe (jwj0831@gmail.com) wrote:
Hi, All
I'm Woojin Joe, a data analytics engineer in Lineplus.
Our team is trying to test Apache Nifi for the data ingestion pipeline
integrated with the Spark cluster.
I found that LivySessionController would be proper to integrate between
Nifi and Spark through Livy.
But there is none property to include `conf` for creating a session through
it.
Thus if possible, I'd like to include `conf` property as a JSON format in
LivySessionController.
IMO, it would take very trivial changes.
For example, just creating a new property descriptor and append `conf` in
the body when it is set.
[image: screenshot_code_diff.png]
I hope to contribute this feature to the current code base if possible.
Warm regards,
Woojin joe
----------------------------------------------------------------------
[image: --]
Woojin Joe
[image: http://]about.me/notforme
<http://about.me/notforme?promo=email_sig>
Woojin Joe (@JoeWoojin <https://twitter.com/JoeWooJin>, Github
<https://github.com/not-for-me>, LinkedIn
<https://www.linkedin.com/in/notforme>)
----------------------------------------------------------------------