You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Russell Whitaker <ru...@gmail.com> on 2015/09/22 19:20:51 UTC
Status of "Add SCP Processor" NIFI-539?
https://issues.apache.org/jira/browse/NIFI-539
Hey Mark and Edgardo,
This was marked in July as having been moved to 0.4.0; any chance
there might be a .nar available now I could add in /lib and start using?
I was delighted to see Tim Reardon's note from May where he said:
"I have recently written a PutScp processor that is holding up well to
moderate volumes. It is modeled after the PutFTP/PutSFTP processors,
using the JCraft JSch library. I will clean up and submit a pull
request."
The abovementioned Jira ticket doesn't show a notation about what happened with
Tim's PR, if it ever happened. I have my own JSch-based solution for
this (written in
Clojure) which is run out of cron (ugh) and I'd love to replace it
with a GetScp
processor solution.
Thanks, R
--
Russell Whitaker
http://twitter.com/OrthoNormalRuss
http://www.linkedin.com/pub/russell-whitaker/0/b86/329
Re: Status of "Add SCP Processor" NIFI-539?
Posted by Edgardo Vega <ed...@gmail.com>.
Russell,
I just put the ticket it because I saw the need for the processor. Glad to
see someone taking it up.
Cheers,
Edgardo
On Tue, Sep 22, 2015 at 3:30 PM, Russell Whitaker <
russell.whitaker@gmail.com> wrote:
> Very cool Joe. OK, I'll start the ball rolling at work - I post from my
> personal
> address but all my traffic here is in fact work-related :-) - and
> account for this
> effort in our work tracking system. Not having a GetScp processor already
> available means this will add some front-loaded effort to some of the
> ingestion
> work in our pipeline, but the payoff is sufficient to make it worth it for
> us.
>
> Thanks for being open to my doing this in Clojure.
>
> Cheers, R
>
> On Tue, Sep 22, 2015 at 12:08 PM, Joe Witt <jo...@gmail.com> wrote:
> > In my personal view they are welcome and an important part of growth
> > of NiFi. These are of course non-trivial contributions both to do and
> > to bring in but that is ok provided both sides know it will take time.
> > JVM friendly ones are less of a big deal of course but never the less
> > this is the sort of things lots of folks will have strong views on I
> > suspect.
> >
> > Let's pick one off the list that you care about the most and see where
> > we can go with it. You like Clojure and it is JVM friendly. Let's
> > start there ?
> >
> > Thanks
> > Joe
> >
> > On Tue, Sep 22, 2015 at 2:17 PM, Russell Whitaker
> > <ru...@gmail.com> wrote:
> >> Hi Joe,
> >>
> >> My SCP code is not at all a processor, it's standalone Clojure code
> >> importing JSch libs. I
> >> suppose I could adapt it and make it a Nifi processor, sure, and yes,
> >> I'd certainly both need
> >> and welcome guidance. If this is the case, I'd like to revisit a
> >> question I asked a few months
> >> ago on this list: would a non-Java but JVM-targetted language like
> >> Clojure fit in with the
> >> core project's build & deployment systems? I've done mixed
> >> Java/Clojure projects at work, but I also
> >> have complete control over pom.xml & project.clj Maven/Leiningen
> artifacts.
> >>
> >> TL;DR: are Clojure/Scala/etc. non-Java JVM language contributions
> welcome or an
> >> administrative burden?
> >>
> >> Thanks, R
> >>
> >> On Tue, Sep 22, 2015 at 10:44 AM, Joe Witt <jo...@gmail.com> wrote:
> >>> Russell,
> >>>
> >>> It is possible that the PR was closed as we moved from 'develop' to
> >>> 'master'. I had reached out but I think Tim has been busy with other
> >>> things. If you have an SCP oriented processor we'd happily help guide
> >>> it in as a contribution. Folks have asked for this fairly often over
> >>> the years so this would be cool.
> >>>
> >>> Thanks
> >>> Joe
> >>>
> >>> On Tue, Sep 22, 2015 at 1:20 PM, Russell Whitaker
> >>> <ru...@gmail.com> wrote:
> >>>> https://issues.apache.org/jira/browse/NIFI-539
> >>>>
> >>>> Hey Mark and Edgardo,
> >>>>
> >>>> This was marked in July as having been moved to 0.4.0; any chance
> >>>> there might be a .nar available now I could add in /lib and start
> using?
> >>>> I was delighted to see Tim Reardon's note from May where he said:
> >>>>
> >>>> "I have recently written a PutScp processor that is holding up well to
> >>>> moderate volumes. It is modeled after the PutFTP/PutSFTP processors,
> >>>> using the JCraft JSch library. I will clean up and submit a pull
> >>>> request."
> >>>>
> >>>> The abovementioned Jira ticket doesn't show a notation about what
> happened with
> >>>> Tim's PR, if it ever happened. I have my own JSch-based solution for
> >>>> this (written in
> >>>> Clojure) which is run out of cron (ugh) and I'd love to replace it
> >>>> with a GetScp
> >>>> processor solution.
> >>>>
> >>>> Thanks, R
> >>>>
> >>>> --
> >>>> Russell Whitaker
> >>>> http://twitter.com/OrthoNormalRuss
> >>>> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
> >>
> >>
> >>
> >> --
> >> Russell Whitaker
> >> http://twitter.com/OrthoNormalRuss
> >> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
>
>
>
> --
> Russell Whitaker
> http://twitter.com/OrthoNormalRuss
> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
>
--
Cheers,
Edgardo
Re: Status of "Add SCP Processor" NIFI-539?
Posted by Russell Whitaker <ru...@gmail.com>.
Very cool Joe. OK, I'll start the ball rolling at work - I post from my personal
address but all my traffic here is in fact work-related :-) - and
account for this
effort in our work tracking system. Not having a GetScp processor already
available means this will add some front-loaded effort to some of the ingestion
work in our pipeline, but the payoff is sufficient to make it worth it for us.
Thanks for being open to my doing this in Clojure.
Cheers, R
On Tue, Sep 22, 2015 at 12:08 PM, Joe Witt <jo...@gmail.com> wrote:
> In my personal view they are welcome and an important part of growth
> of NiFi. These are of course non-trivial contributions both to do and
> to bring in but that is ok provided both sides know it will take time.
> JVM friendly ones are less of a big deal of course but never the less
> this is the sort of things lots of folks will have strong views on I
> suspect.
>
> Let's pick one off the list that you care about the most and see where
> we can go with it. You like Clojure and it is JVM friendly. Let's
> start there ?
>
> Thanks
> Joe
>
> On Tue, Sep 22, 2015 at 2:17 PM, Russell Whitaker
> <ru...@gmail.com> wrote:
>> Hi Joe,
>>
>> My SCP code is not at all a processor, it's standalone Clojure code
>> importing JSch libs. I
>> suppose I could adapt it and make it a Nifi processor, sure, and yes,
>> I'd certainly both need
>> and welcome guidance. If this is the case, I'd like to revisit a
>> question I asked a few months
>> ago on this list: would a non-Java but JVM-targetted language like
>> Clojure fit in with the
>> core project's build & deployment systems? I've done mixed
>> Java/Clojure projects at work, but I also
>> have complete control over pom.xml & project.clj Maven/Leiningen artifacts.
>>
>> TL;DR: are Clojure/Scala/etc. non-Java JVM language contributions welcome or an
>> administrative burden?
>>
>> Thanks, R
>>
>> On Tue, Sep 22, 2015 at 10:44 AM, Joe Witt <jo...@gmail.com> wrote:
>>> Russell,
>>>
>>> It is possible that the PR was closed as we moved from 'develop' to
>>> 'master'. I had reached out but I think Tim has been busy with other
>>> things. If you have an SCP oriented processor we'd happily help guide
>>> it in as a contribution. Folks have asked for this fairly often over
>>> the years so this would be cool.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Tue, Sep 22, 2015 at 1:20 PM, Russell Whitaker
>>> <ru...@gmail.com> wrote:
>>>> https://issues.apache.org/jira/browse/NIFI-539
>>>>
>>>> Hey Mark and Edgardo,
>>>>
>>>> This was marked in July as having been moved to 0.4.0; any chance
>>>> there might be a .nar available now I could add in /lib and start using?
>>>> I was delighted to see Tim Reardon's note from May where he said:
>>>>
>>>> "I have recently written a PutScp processor that is holding up well to
>>>> moderate volumes. It is modeled after the PutFTP/PutSFTP processors,
>>>> using the JCraft JSch library. I will clean up and submit a pull
>>>> request."
>>>>
>>>> The abovementioned Jira ticket doesn't show a notation about what happened with
>>>> Tim's PR, if it ever happened. I have my own JSch-based solution for
>>>> this (written in
>>>> Clojure) which is run out of cron (ugh) and I'd love to replace it
>>>> with a GetScp
>>>> processor solution.
>>>>
>>>> Thanks, R
>>>>
>>>> --
>>>> Russell Whitaker
>>>> http://twitter.com/OrthoNormalRuss
>>>> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
>>
>>
>>
>> --
>> Russell Whitaker
>> http://twitter.com/OrthoNormalRuss
>> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
--
Russell Whitaker
http://twitter.com/OrthoNormalRuss
http://www.linkedin.com/pub/russell-whitaker/0/b86/329
Re: Status of "Add SCP Processor" NIFI-539?
Posted by Joe Witt <jo...@gmail.com>.
In my personal view they are welcome and an important part of growth
of NiFi. These are of course non-trivial contributions both to do and
to bring in but that is ok provided both sides know it will take time.
JVM friendly ones are less of a big deal of course but never the less
this is the sort of things lots of folks will have strong views on I
suspect.
Let's pick one off the list that you care about the most and see where
we can go with it. You like Clojure and it is JVM friendly. Let's
start there ?
Thanks
Joe
On Tue, Sep 22, 2015 at 2:17 PM, Russell Whitaker
<ru...@gmail.com> wrote:
> Hi Joe,
>
> My SCP code is not at all a processor, it's standalone Clojure code
> importing JSch libs. I
> suppose I could adapt it and make it a Nifi processor, sure, and yes,
> I'd certainly both need
> and welcome guidance. If this is the case, I'd like to revisit a
> question I asked a few months
> ago on this list: would a non-Java but JVM-targetted language like
> Clojure fit in with the
> core project's build & deployment systems? I've done mixed
> Java/Clojure projects at work, but I also
> have complete control over pom.xml & project.clj Maven/Leiningen artifacts.
>
> TL;DR: are Clojure/Scala/etc. non-Java JVM language contributions welcome or an
> administrative burden?
>
> Thanks, R
>
> On Tue, Sep 22, 2015 at 10:44 AM, Joe Witt <jo...@gmail.com> wrote:
>> Russell,
>>
>> It is possible that the PR was closed as we moved from 'develop' to
>> 'master'. I had reached out but I think Tim has been busy with other
>> things. If you have an SCP oriented processor we'd happily help guide
>> it in as a contribution. Folks have asked for this fairly often over
>> the years so this would be cool.
>>
>> Thanks
>> Joe
>>
>> On Tue, Sep 22, 2015 at 1:20 PM, Russell Whitaker
>> <ru...@gmail.com> wrote:
>>> https://issues.apache.org/jira/browse/NIFI-539
>>>
>>> Hey Mark and Edgardo,
>>>
>>> This was marked in July as having been moved to 0.4.0; any chance
>>> there might be a .nar available now I could add in /lib and start using?
>>> I was delighted to see Tim Reardon's note from May where he said:
>>>
>>> "I have recently written a PutScp processor that is holding up well to
>>> moderate volumes. It is modeled after the PutFTP/PutSFTP processors,
>>> using the JCraft JSch library. I will clean up and submit a pull
>>> request."
>>>
>>> The abovementioned Jira ticket doesn't show a notation about what happened with
>>> Tim's PR, if it ever happened. I have my own JSch-based solution for
>>> this (written in
>>> Clojure) which is run out of cron (ugh) and I'd love to replace it
>>> with a GetScp
>>> processor solution.
>>>
>>> Thanks, R
>>>
>>> --
>>> Russell Whitaker
>>> http://twitter.com/OrthoNormalRuss
>>> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
>
>
>
> --
> Russell Whitaker
> http://twitter.com/OrthoNormalRuss
> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
Re: Status of "Add SCP Processor" NIFI-539?
Posted by Russell Whitaker <ru...@gmail.com>.
Hi Joe,
My SCP code is not at all a processor, it's standalone Clojure code
importing JSch libs. I
suppose I could adapt it and make it a Nifi processor, sure, and yes,
I'd certainly both need
and welcome guidance. If this is the case, I'd like to revisit a
question I asked a few months
ago on this list: would a non-Java but JVM-targetted language like
Clojure fit in with the
core project's build & deployment systems? I've done mixed
Java/Clojure projects at work, but I also
have complete control over pom.xml & project.clj Maven/Leiningen artifacts.
TL;DR: are Clojure/Scala/etc. non-Java JVM language contributions welcome or an
administrative burden?
Thanks, R
On Tue, Sep 22, 2015 at 10:44 AM, Joe Witt <jo...@gmail.com> wrote:
> Russell,
>
> It is possible that the PR was closed as we moved from 'develop' to
> 'master'. I had reached out but I think Tim has been busy with other
> things. If you have an SCP oriented processor we'd happily help guide
> it in as a contribution. Folks have asked for this fairly often over
> the years so this would be cool.
>
> Thanks
> Joe
>
> On Tue, Sep 22, 2015 at 1:20 PM, Russell Whitaker
> <ru...@gmail.com> wrote:
>> https://issues.apache.org/jira/browse/NIFI-539
>>
>> Hey Mark and Edgardo,
>>
>> This was marked in July as having been moved to 0.4.0; any chance
>> there might be a .nar available now I could add in /lib and start using?
>> I was delighted to see Tim Reardon's note from May where he said:
>>
>> "I have recently written a PutScp processor that is holding up well to
>> moderate volumes. It is modeled after the PutFTP/PutSFTP processors,
>> using the JCraft JSch library. I will clean up and submit a pull
>> request."
>>
>> The abovementioned Jira ticket doesn't show a notation about what happened with
>> Tim's PR, if it ever happened. I have my own JSch-based solution for
>> this (written in
>> Clojure) which is run out of cron (ugh) and I'd love to replace it
>> with a GetScp
>> processor solution.
>>
>> Thanks, R
>>
>> --
>> Russell Whitaker
>> http://twitter.com/OrthoNormalRuss
>> http://www.linkedin.com/pub/russell-whitaker/0/b86/329
--
Russell Whitaker
http://twitter.com/OrthoNormalRuss
http://www.linkedin.com/pub/russell-whitaker/0/b86/329
Re: Status of "Add SCP Processor" NIFI-539?
Posted by Joe Witt <jo...@gmail.com>.
Russell,
It is possible that the PR was closed as we moved from 'develop' to
'master'. I had reached out but I think Tim has been busy with other
things. If you have an SCP oriented processor we'd happily help guide
it in as a contribution. Folks have asked for this fairly often over
the years so this would be cool.
Thanks
Joe
On Tue, Sep 22, 2015 at 1:20 PM, Russell Whitaker
<ru...@gmail.com> wrote:
> https://issues.apache.org/jira/browse/NIFI-539
>
> Hey Mark and Edgardo,
>
> This was marked in July as having been moved to 0.4.0; any chance
> there might be a .nar available now I could add in /lib and start using?
> I was delighted to see Tim Reardon's note from May where he said:
>
> "I have recently written a PutScp processor that is holding up well to
> moderate volumes. It is modeled after the PutFTP/PutSFTP processors,
> using the JCraft JSch library. I will clean up and submit a pull
> request."
>
> The abovementioned Jira ticket doesn't show a notation about what happened with
> Tim's PR, if it ever happened. I have my own JSch-based solution for
> this (written in
> Clojure) which is run out of cron (ugh) and I'd love to replace it
> with a GetScp
> processor solution.
>
> Thanks, R
>
> --
> Russell Whitaker
> http://twitter.com/OrthoNormalRuss
> http://www.linkedin.com/pub/russell-whitaker/0/b86/329