You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Richard Beare <ri...@gmail.com> on 2023/01/30 10:49:41 UTC
Best practice for configuring jolt processor?
Hi,
What is the preferred way of managing the jolttransformrecord processor
specification? Is there a convenient way to store the spec in a file that
can be version controlled etc?
Re: Best practice for configuring jolt processor?
Posted by Richard Beare <ri...@gmail.com>.
What are the options for populating context variables? e.g. is there a
pattern to load all files from a folder, create variables based on
filename, and populate those variables with the file content?
My problem isn't reusing the transform on multiple nodes, although that may
happen in the future, but simplifying the management of the transform
content.
Thanks
On Fri, Feb 24, 2023 at 4:08 AM Matt Burgess <ma...@apache.org> wrote:
> I'm resurrecting the PR to add a property to specify a file location
> for the Jolt spec. In the meantime you could maintain the spec as the
> value of a context variable and point to it in the Jolt Spec property.
> Then you can share amongst the nodes and maintain the spec in one
> place.
>
> Regards,
> Matt
>
> On Wed, Feb 22, 2023 at 3:17 PM Richard Beare <ri...@gmail.com>
> wrote:
> >
> > Hi,
> > I'm hoping to be able to look at this further with someone I'm working
> with who has more java background. I notice there's a reference to another
> PR from Matt, so perhaps the bulk of the work has been done? What is the
> status and what is the next step required?
> >
> > On Thu, Feb 2, 2023 at 11:28 AM Dirk Arends <di...@fontis.com.au>
> wrote:
> >>
> >> Hi Richard,
> >>
> >> Some work was previously done to the similar processor
> JoltTransformJSON but from what I can see, hasn't been merged into main or
> included in a release.
> >>
> >> NIFI-4957: Enable JoltTransformJSON to pickup a Jolt Spec file from a
> file location
> >>
> >> https://issues.apache.org/jira/browse/NIFI-4957
> >>
> >> I understand this isn't really helpful, but if you have the capacity,
> it would be worth having another crack at these changes.
> >>
> >> Regards,
> >> Dirk Arends
> >>
> >>
> >> On Mon, 30 Jan 2023 at 21:50, Richard Beare <ri...@gmail.com>
> wrote:
> >>>
> >>> Hi,
> >>> What is the preferred way of managing the jolttransformrecord
> processor specification? Is there a convenient way to store the spec in a
> file that can be version controlled etc?
> >>
> >>
> >>
> >> --
> >> Regards,
> >>
> >> --
> >> Dirk Arends
>
Re: Best practice for configuring jolt processor?
Posted by Matt Burgess <ma...@apache.org>.
I'm resurrecting the PR to add a property to specify a file location
for the Jolt spec. In the meantime you could maintain the spec as the
value of a context variable and point to it in the Jolt Spec property.
Then you can share amongst the nodes and maintain the spec in one
place.
Regards,
Matt
On Wed, Feb 22, 2023 at 3:17 PM Richard Beare <ri...@gmail.com> wrote:
>
> Hi,
> I'm hoping to be able to look at this further with someone I'm working with who has more java background. I notice there's a reference to another PR from Matt, so perhaps the bulk of the work has been done? What is the status and what is the next step required?
>
> On Thu, Feb 2, 2023 at 11:28 AM Dirk Arends <di...@fontis.com.au> wrote:
>>
>> Hi Richard,
>>
>> Some work was previously done to the similar processor JoltTransformJSON but from what I can see, hasn't been merged into main or included in a release.
>>
>> NIFI-4957: Enable JoltTransformJSON to pickup a Jolt Spec file from a file location
>>
>> https://issues.apache.org/jira/browse/NIFI-4957
>>
>> I understand this isn't really helpful, but if you have the capacity, it would be worth having another crack at these changes.
>>
>> Regards,
>> Dirk Arends
>>
>>
>> On Mon, 30 Jan 2023 at 21:50, Richard Beare <ri...@gmail.com> wrote:
>>>
>>> Hi,
>>> What is the preferred way of managing the jolttransformrecord processor specification? Is there a convenient way to store the spec in a file that can be version controlled etc?
>>
>>
>>
>> --
>> Regards,
>>
>> --
>> Dirk Arends
Re: Best practice for configuring jolt processor?
Posted by Richard Beare <ri...@gmail.com>.
Hi,
I'm hoping to be able to look at this further with someone I'm working with
who has more java background. I notice there's a reference to another PR
from Matt, so perhaps the bulk of the work has been done? What is the
status and what is the next step required?
On Thu, Feb 2, 2023 at 11:28 AM Dirk Arends <di...@fontis.com.au>
wrote:
> Hi Richard,
>
> Some work was previously done to the similar processor JoltTransformJSON
> but from what I can see, hasn't been merged into main or included in a
> release.
>
> NIFI-4957: Enable JoltTransformJSON to pickup a Jolt Spec file from a file
> location
>
> https://issues.apache.org/jira/browse/NIFI-4957
>
> I understand this isn't really helpful, but if you have the capacity, it
> would be worth having another crack at these changes.
>
> Regards,
> Dirk Arends
>
>
> On Mon, 30 Jan 2023 at 21:50, Richard Beare <ri...@gmail.com>
> wrote:
>
>> Hi,
>> What is the preferred way of managing the jolttransformrecord processor
>> specification? Is there a convenient way to store the spec in a file that
>> can be version controlled etc?
>>
>
>
> --
> Regards,
>
> --
> Dirk Arends
>
Re: Best practice for configuring jolt processor?
Posted by Richard Beare <ri...@gmail.com>.
I'm hoping to be able to look at this further with someone I'm working with
who has more java background. I notice there's a reference to another PR
from Matt, so perhaps the bulk of the work has been done? What is the
status and what is the next step required?
On Thu, Feb 2, 2023 at 11:28 AM Dirk Arends <di...@fontis.com.au>
wrote:
> Hi Richard,
>
> Some work was previously done to the similar processor JoltTransformJSON
> but from what I can see, hasn't been merged into main or included in a
> release.
>
> NIFI-4957: Enable JoltTransformJSON to pickup a Jolt Spec file from a file
> location
>
> https://issues.apache.org/jira/browse/NIFI-4957
>
> I understand this isn't really helpful, but if you have the capacity, it
> would be worth having another crack at these changes.
>
> Regards,
> Dirk Arends
>
>
> On Mon, 30 Jan 2023 at 21:50, Richard Beare <ri...@gmail.com>
> wrote:
>
>> Hi,
>> What is the preferred way of managing the jolttransformrecord processor
>> specification? Is there a convenient way to store the spec in a file that
>> can be version controlled etc?
>>
>
>
> --
> Regards,
>
> --
> Dirk Arends
>
Re: Best practice for configuring jolt processor?
Posted by Dirk Arends <di...@fontis.com.au>.
Hi Richard,
Some work was previously done to the similar processor JoltTransformJSON
but from what I can see, hasn't been merged into main or included in a
release.
NIFI-4957: Enable JoltTransformJSON to pickup a Jolt Spec file from a file
location
https://issues.apache.org/jira/browse/NIFI-4957
I understand this isn't really helpful, but if you have the capacity, it
would be worth having another crack at these changes.
Regards,
Dirk Arends
On Mon, 30 Jan 2023 at 21:50, Richard Beare <ri...@gmail.com> wrote:
> Hi,
> What is the preferred way of managing the jolttransformrecord processor
> specification? Is there a convenient way to store the spec in a file that
> can be version controlled etc?
>
--
Regards,
--
Dirk Arends