You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Andrew Christianson <an...@nextcentury.com> on 2017/05/04 12:07:30 UTC
MiNiFi C++ Expression Language
All,
I see that we do not have support for the expression language yet in MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA? If no one is working on it, is there a general plan for how it should be implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
> I am not aware of it being actively worked [1]. This would require using
> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
> correct me if I'm wrong, but there would have to be changes to both sides.
Both sides as in ANTLR4 and MiNiFi, or as in MiNiFi and NiFi?
> It generated code, but I'm
> uncertain of cross platform compatibility with the expression language. If
> that's not expected or required that will remove some limitations as a
> result of moving to ANTLR4.
Similarly, do you mean compatibility between NiFi and MiNiFi, or the OS/CPU platform? If we're talking compatibility between NiFi and MiNiFi on the expr language, maybe Aldrin or Joe has the answer on whether this is a design goal/requirement.
On a side note, have we looked into porting the grammar to bison? Bison is significantly more tested/supported. It is GNU, but my understanding is that the generated code is not tied to GNU, just the generator (bison). Calling it as part of the build process should be compatible with Apache.
- Andy
________________________________________
From: Marc P. <ma...@gmail.com>
Sent: Thursday, May 4, 2017 8:13:15 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
Andrew,
I am not aware of it being actively worked [1]. This would require using
ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
correct me if I'm wrong, but there would have to be changes to both sides.
I attempted a quick straw man with grammars, but didn't take it very far
after making initial changes to the grammar. It generated code, but I'm
uncertain of cross platform compatibility with the expression language. If
that's not expected or required that will remove some limitations as a
result of moving to ANTLR4.
[1] https://issues.apache.org/jira/browse/MINIFI-140
[2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
andrew.christianson@nextcentury.com> wrote:
> All,
>
> I see that we do not have support for the expression language yet in
> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
> If no one is working on it, is there a general plan for how it should be
> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@apache.org>.
Although I still think the ANTLR4 upgrade would be a
considerably-sized effort, I did make some headway at lunch and pushed
up an "antlr4" branch to my fork [1]. For now I kept the antlr3 stuff
side-by-side with the antlr4 stuff, mostly so I can refer to the
original as reference. It still builds and runs fine if you don't
touch Query or StandardPreparedQuery. However I started making some
updates in those files as well. Query is the workhorse, it recursively
builds Evaluators while walking the tree. In ANTLR4 the idiom is
either to use visitors or listeners (the latter is generated by
default). So the lion's share of the work seems to be converting Query
to use Listeners. If any ANTLR fans out there want to pick this up
and run with it, by all means feel free :) Otherwise I will tinker
with it during rare bouts of spare time.
Regards,
Matt
[1] https://github.com/mattyb149/nifi/tree/antlr4
On Thu, May 4, 2017 at 11:52 AM, Matt Burgess <ma...@gmail.com> wrote:
> Good point, we should select a parser generator that produces tight code. Having said that, the MVP grammar is so simple that it could easily be put into a parser generator DSL later, and the MVP implementation could be hand-written for the first iteration (I'm picturing a couple dozen lines of code, but that's just a knee-jerk estimation).
>
>
>> On May 4, 2017, at 11:25 AM, Marc <ph...@apache.org> wrote:
>>
>> Andy,
>> Depending on the type of parser chosen you may run into memory
>> limitations on devices that run MiNiFi-C++. Since that's a deployment
>> concern -- and one that few will face, I would add that we can make this a
>> conditional build and only including the necessary components within the
>> binary if a build option is not specified to exclude these components.
>>
>> On Thu, May 4, 2017 at 11:14 AM, Andrew Christianson <
>> andrew.christianson@nextcentury.com> wrote:
>>
>>> The island grammar sounds very appealing for an MVP. Simple to implement,
>>> yet it covers a very common EL use case (dynamically inserting attr vals
>>> into property values). If we have general consensus I would love to see a
>>> MINIFI JIRA ticket added for this.
>>>
>>> -Andy
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 11:09:46 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> What you have sounds good to me. IMO minimum viable product would be an
>>> island grammar (meaning you can have any characters outside a ${}
>>> expression) and inside would support an attribute name. Next steps could be
>>> nested expressions and/or support for functions, added piecemeal as the
>>> contributor sees fit. I wasn't involved in the early development of the EL
>>> grammar, so maybe someone who was has some thoughts on a natural evolution.
>>>
>>> Regards,
>>> Matt
>>>
>>>> On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@
>>> nextcentury.com> wrote:
>>>>
>>>> My bad, what does the sketch of the plan *look like*?
>>>>
>>>> -Andy
>>>> ________________________________________
>>>> From: Andrew Christianson
>>>> Sent: Thursday, May 4, 2017 10:59:07 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>
>>>> What does the sketch of the plan to do the separate implementation?
>>> Write a flex/bison grammar, hook it into the cmake build, and start using
>>> it? Any constraints on features or syntax that this separate implementation
>>> must support?
>>>>
>>>> -Andy
>>>> ________________________________________
>>>> From: Matt Burgess <ma...@gmail.com>
>>>> Sent: Thursday, May 4, 2017 9:37:34 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>
>>>> No plans that I know of. In the meantime, EL support for MiNiFi is kind
>>> of held hostage, so maybe the separate implementation is more viable in the
>>> nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever
>>> exists by then with the cross-platform ANTLR target generation, and test
>>> the whole kit and caboodle.
>>>>
>>>> Anyone out there familiar with the aforementioned tools (or willing to
>>> hand-roll one)? What do you think about this "EL bootstrapping" approach?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>>> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@
>>> nextcentury.com> wrote:
>>>>>
>>>>> Got it. So the crux of the problem is porting from v3 to v4, plus the
>>> added uncertainty of the C++ v4 target.
>>>>>
>>>>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If
>>> that's the case, then porting to v4 is probably the ticket. Are there any
>>> concrete plans to do so in the NiFi mother project yet?
>>>>>
>>>>> -Andy
>>>>> ________________________________________
>>>>> From: Matt Burgess <ma...@gmail.com>
>>>>> Sent: Thursday, May 4, 2017 9:18:00 AM
>>>>> To: dev@nifi.apache.org
>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>>
>>>>> Correct, the current NiFi EL grammar is ANTLR3.
>>>>>
>>>>>
>>>>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@
>>> nextcentury.com> wrote:
>>>>>>
>>>>>> Do I understand correctly that NiFi is currently using ANTLRv3?
>>>>>> ________________________________________
>>>>>> From: Matt Burgess <ma...@gmail.com>
>>>>>> Sent: Thursday, May 4, 2017 9:05:35 AM
>>>>>> To: dev@nifi.apache.org
>>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>>>
>>>>>> I haven't used Flex/Bison since a trivial example in college, so I'm
>>> not sure about the LOE for getting that set up, maybe there's a Maven-built
>>> project out there that we could look at for inspiration, but that seems
>>> unlikely :)
>>>>>>
>>>>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would
>>> give us NiFi/MiNiFi EL compatibility (and full-featured EL support in
>>> MiNiFi C++), but we'd have to accept the risks of introducing bugs,
>>> regressions, etc. as a result of the refactor. Basically we'd just need to
>>> test the heck out of it on all platforms, which isn't a bad thing but adds
>>> to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for
>>> incremental development of a C/C++ based grammar.
>>>>>>
>>>>>>
>>>>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@
>>> nextcentury.com> wrote:
>>>>>>
>>>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
>>> refactor the existing grammar. Since the source and target for MiNiFi C++
>>> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
>>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
>>> are doing that with all the MiNiFi components already. The upside is being
>>> able to support EL incrementally as the grammar is developed. What do you
>>> think?
>>>>>>>
>>>>>>> This seems like a pragmatic approach. What's the level-of-effort
>>> required to do the initial grammar port and set up the build tooling? Less
>>> than refactoring for ANTLR4? I'm not as familiar with the EL grammar
>>> situation.
>>>>>>>
>>>>>>> -Andy
>>>>>>> ________________________________________
>>>>>>> From: Matt Burgess <ma...@gmail.com>
>>>>>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>>>>>> To: dev@nifi.apache.org
>>>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>>>>
>>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
>>> refactor the existing grammar. Since the source and target for MiNiFi C++
>>> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
>>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
>>> are doing that with all the MiNiFi components already. The upside is being
>>> able to support EL incrementally as the grammar is developed. What do you
>>> think?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Matt
>>>>>>>
>>>>>>> [1] http://dinosaur.compilertools.net/
>>>>>>> [2] http://www.hwaci.com/sw/lemon/
>>>>>>> [3] http://www.colm.net/open-source/ragel/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Sent from my iPhone
>>>>>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Andrew,
>>>>>>>> I am not aware of it being actively worked [1]. This would require
>>> using
>>>>>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone
>>> can
>>>>>>>> correct me if I'm wrong, but there would have to be changes to both
>>> sides.
>>>>>>>> I attempted a quick straw man with grammars, but didn't take it very
>>> far
>>>>>>>> after making initial changes to the grammar. It generated code, but
>>> I'm
>>>>>>>> uncertain of cross platform compatibility with the expression
>>> language. If
>>>>>>>> that's not expected or required that will remove some limitations as
>>> a
>>>>>>>> result of moving to ANTLR4.
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-
>>> c-target-is-here
>>>>>>>>
>>>>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>>>>>> andrew.christianson@nextcentury.com> wrote:
>>>>>>>>
>>>>>>>>> All,
>>>>>>>>>
>>>>>>>>> I see that we do not have support for the expression language yet in
>>>>>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there
>>> an ETA?
>>>>>>>>> If no one is working on it, is there a general plan for how it
>>> should be
>>>>>>>>> implemented? I think I recall seeing references to ANTLR
>>>
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@gmail.com>.
Good point, we should select a parser generator that produces tight code. Having said that, the MVP grammar is so simple that it could easily be put into a parser generator DSL later, and the MVP implementation could be hand-written for the first iteration (I'm picturing a couple dozen lines of code, but that's just a knee-jerk estimation).
> On May 4, 2017, at 11:25 AM, Marc <ph...@apache.org> wrote:
>
> Andy,
> Depending on the type of parser chosen you may run into memory
> limitations on devices that run MiNiFi-C++. Since that's a deployment
> concern -- and one that few will face, I would add that we can make this a
> conditional build and only including the necessary components within the
> binary if a build option is not specified to exclude these components.
>
> On Thu, May 4, 2017 at 11:14 AM, Andrew Christianson <
> andrew.christianson@nextcentury.com> wrote:
>
>> The island grammar sounds very appealing for an MVP. Simple to implement,
>> yet it covers a very common EL use case (dynamically inserting attr vals
>> into property values). If we have general consensus I would love to see a
>> MINIFI JIRA ticket added for this.
>>
>> -Andy
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 11:09:46 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> What you have sounds good to me. IMO minimum viable product would be an
>> island grammar (meaning you can have any characters outside a ${}
>> expression) and inside would support an attribute name. Next steps could be
>> nested expressions and/or support for functions, added piecemeal as the
>> contributor sees fit. I wasn't involved in the early development of the EL
>> grammar, so maybe someone who was has some thoughts on a natural evolution.
>>
>> Regards,
>> Matt
>>
>>> On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>
>>> My bad, what does the sketch of the plan *look like*?
>>>
>>> -Andy
>>> ________________________________________
>>> From: Andrew Christianson
>>> Sent: Thursday, May 4, 2017 10:59:07 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> What does the sketch of the plan to do the separate implementation?
>> Write a flex/bison grammar, hook it into the cmake build, and start using
>> it? Any constraints on features or syntax that this separate implementation
>> must support?
>>>
>>> -Andy
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 9:37:34 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> No plans that I know of. In the meantime, EL support for MiNiFi is kind
>> of held hostage, so maybe the separate implementation is more viable in the
>> nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever
>> exists by then with the cross-platform ANTLR target generation, and test
>> the whole kit and caboodle.
>>>
>>> Anyone out there familiar with the aforementioned tools (or willing to
>> hand-roll one)? What do you think about this "EL bootstrapping" approach?
>>>
>>> Thanks,
>>> Matt
>>>
>>>> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>>
>>>> Got it. So the crux of the problem is porting from v3 to v4, plus the
>> added uncertainty of the C++ v4 target.
>>>>
>>>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If
>> that's the case, then porting to v4 is probably the ticket. Are there any
>> concrete plans to do so in the NiFi mother project yet?
>>>>
>>>> -Andy
>>>> ________________________________________
>>>> From: Matt Burgess <ma...@gmail.com>
>>>> Sent: Thursday, May 4, 2017 9:18:00 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>
>>>> Correct, the current NiFi EL grammar is ANTLR3.
>>>>
>>>>
>>>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>>>
>>>>> Do I understand correctly that NiFi is currently using ANTLRv3?
>>>>> ________________________________________
>>>>> From: Matt Burgess <ma...@gmail.com>
>>>>> Sent: Thursday, May 4, 2017 9:05:35 AM
>>>>> To: dev@nifi.apache.org
>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>>
>>>>> I haven't used Flex/Bison since a trivial example in college, so I'm
>> not sure about the LOE for getting that set up, maybe there's a Maven-built
>> project out there that we could look at for inspiration, but that seems
>> unlikely :)
>>>>>
>>>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would
>> give us NiFi/MiNiFi EL compatibility (and full-featured EL support in
>> MiNiFi C++), but we'd have to accept the risks of introducing bugs,
>> regressions, etc. as a result of the refactor. Basically we'd just need to
>> test the heck out of it on all platforms, which isn't a bad thing but adds
>> to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for
>> incremental development of a C/C++ based grammar.
>>>>>
>>>>>
>>>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@
>> nextcentury.com> wrote:
>>>>>
>>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
>> refactor the existing grammar. Since the source and target for MiNiFi C++
>> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
>> are doing that with all the MiNiFi components already. The upside is being
>> able to support EL incrementally as the grammar is developed. What do you
>> think?
>>>>>>
>>>>>> This seems like a pragmatic approach. What's the level-of-effort
>> required to do the initial grammar port and set up the build tooling? Less
>> than refactoring for ANTLR4? I'm not as familiar with the EL grammar
>> situation.
>>>>>>
>>>>>> -Andy
>>>>>> ________________________________________
>>>>>> From: Matt Burgess <ma...@gmail.com>
>>>>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>>>>> To: dev@nifi.apache.org
>>>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>>>
>>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
>> refactor the existing grammar. Since the source and target for MiNiFi C++
>> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
>> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
>> are doing that with all the MiNiFi components already. The upside is being
>> able to support EL incrementally as the grammar is developed. What do you
>> think?
>>>>>>
>>>>>> Regards,
>>>>>> Matt
>>>>>>
>>>>>> [1] http://dinosaur.compilertools.net/
>>>>>> [2] http://www.hwaci.com/sw/lemon/
>>>>>> [3] http://www.colm.net/open-source/ragel/
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>>>>
>>>>>>> Andrew,
>>>>>>> I am not aware of it being actively worked [1]. This would require
>> using
>>>>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone
>> can
>>>>>>> correct me if I'm wrong, but there would have to be changes to both
>> sides.
>>>>>>> I attempted a quick straw man with grammars, but didn't take it very
>> far
>>>>>>> after making initial changes to the grammar. It generated code, but
>> I'm
>>>>>>> uncertain of cross platform compatibility with the expression
>> language. If
>>>>>>> that's not expected or required that will remove some limitations as
>> a
>>>>>>> result of moving to ANTLR4.
>>>>>>>
>>>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-
>> c-target-is-here
>>>>>>>
>>>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>>>>> andrew.christianson@nextcentury.com> wrote:
>>>>>>>
>>>>>>>> All,
>>>>>>>>
>>>>>>>> I see that we do not have support for the expression language yet in
>>>>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there
>> an ETA?
>>>>>>>> If no one is working on it, is there a general plan for how it
>> should be
>>>>>>>> implemented? I think I recall seeing references to ANTLR
>>
Re: MiNiFi C++ Expression Language
Posted by Marc <ph...@apache.org>.
Andy,
Depending on the type of parser chosen you may run into memory
limitations on devices that run MiNiFi-C++. Since that's a deployment
concern -- and one that few will face, I would add that we can make this a
conditional build and only including the necessary components within the
binary if a build option is not specified to exclude these components.
On Thu, May 4, 2017 at 11:14 AM, Andrew Christianson <
andrew.christianson@nextcentury.com> wrote:
> The island grammar sounds very appealing for an MVP. Simple to implement,
> yet it covers a very common EL use case (dynamically inserting attr vals
> into property values). If we have general consensus I would love to see a
> MINIFI JIRA ticket added for this.
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 11:09:46 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> What you have sounds good to me. IMO minimum viable product would be an
> island grammar (meaning you can have any characters outside a ${}
> expression) and inside would support an attribute name. Next steps could be
> nested expressions and/or support for functions, added piecemeal as the
> contributor sees fit. I wasn't involved in the early development of the EL
> grammar, so maybe someone who was has some thoughts on a natural evolution.
>
> Regards,
> Matt
>
> > On May 4, 2017, at 11:00 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >
> > My bad, what does the sketch of the plan *look like*?
> >
> > -Andy
> > ________________________________________
> > From: Andrew Christianson
> > Sent: Thursday, May 4, 2017 10:59:07 AM
> > To: dev@nifi.apache.org
> > Subject: Re: MiNiFi C++ Expression Language
> >
> > What does the sketch of the plan to do the separate implementation?
> Write a flex/bison grammar, hook it into the cmake build, and start using
> it? Any constraints on features or syntax that this separate implementation
> must support?
> >
> > -Andy
> > ________________________________________
> > From: Matt Burgess <ma...@gmail.com>
> > Sent: Thursday, May 4, 2017 9:37:34 AM
> > To: dev@nifi.apache.org
> > Subject: Re: MiNiFi C++ Expression Language
> >
> > No plans that I know of. In the meantime, EL support for MiNiFi is kind
> of held hostage, so maybe the separate implementation is more viable in the
> nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever
> exists by then with the cross-platform ANTLR target generation, and test
> the whole kit and caboodle.
> >
> > Anyone out there familiar with the aforementioned tools (or willing to
> hand-roll one)? What do you think about this "EL bootstrapping" approach?
> >
> > Thanks,
> > Matt
> >
> >> On May 4, 2017, at 9:22 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >>
> >> Got it. So the crux of the problem is porting from v3 to v4, plus the
> added uncertainty of the C++ v4 target.
> >>
> >> I'm assuming that NiFi wants to eventually get onto v4 anyway. If
> that's the case, then porting to v4 is probably the ticket. Are there any
> concrete plans to do so in the NiFi mother project yet?
> >>
> >> -Andy
> >> ________________________________________
> >> From: Matt Burgess <ma...@gmail.com>
> >> Sent: Thursday, May 4, 2017 9:18:00 AM
> >> To: dev@nifi.apache.org
> >> Subject: Re: MiNiFi C++ Expression Language
> >>
> >> Correct, the current NiFi EL grammar is ANTLR3.
> >>
> >>
> >>> On May 4, 2017, at 9:12 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >>>
> >>> Do I understand correctly that NiFi is currently using ANTLRv3?
> >>> ________________________________________
> >>> From: Matt Burgess <ma...@gmail.com>
> >>> Sent: Thursday, May 4, 2017 9:05:35 AM
> >>> To: dev@nifi.apache.org
> >>> Subject: Re: MiNiFi C++ Expression Language
> >>>
> >>> I haven't used Flex/Bison since a trivial example in college, so I'm
> not sure about the LOE for getting that set up, maybe there's a Maven-built
> project out there that we could look at for inspiration, but that seems
> unlikely :)
> >>>
> >>> An ANTLR4 refactor (assuming the C++ target is in good shape) would
> give us NiFi/MiNiFi EL compatibility (and full-featured EL support in
> MiNiFi C++), but we'd have to accept the risks of introducing bugs,
> regressions, etc. as a result of the refactor. Basically we'd just need to
> test the heck out of it on all platforms, which isn't a bad thing but adds
> to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for
> incremental development of a C/C++ based grammar.
> >>>
> >>>
> >>> On May 4, 2017, at 8:51 AM, Andrew Christianson <andrew.christianson@
> nextcentury.com> wrote:
> >>>
> >>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
> refactor the existing grammar. Since the source and target for MiNiFi C++
> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
> are doing that with all the MiNiFi components already. The upside is being
> able to support EL incrementally as the grammar is developed. What do you
> think?
> >>>>
> >>>> This seems like a pragmatic approach. What's the level-of-effort
> required to do the initial grammar port and set up the build tooling? Less
> than refactoring for ANTLR4? I'm not as familiar with the EL grammar
> situation.
> >>>>
> >>>> -Andy
> >>>> ________________________________________
> >>>> From: Matt Burgess <ma...@gmail.com>
> >>>> Sent: Thursday, May 4, 2017 8:46:20 AM
> >>>> To: dev@nifi.apache.org
> >>>> Subject: Re: MiNiFi C++ Expression Language
> >>>>
> >>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to
> refactor the existing grammar. Since the source and target for MiNiFi C++
> is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1],
> Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we
> are doing that with all the MiNiFi components already. The upside is being
> able to support EL incrementally as the grammar is developed. What do you
> think?
> >>>>
> >>>> Regards,
> >>>> Matt
> >>>>
> >>>> [1] http://dinosaur.compilertools.net/
> >>>> [2] http://www.hwaci.com/sw/lemon/
> >>>> [3] http://www.colm.net/open-source/ragel/
> >>>>
> >>>>
> >>>>
> >>>> Sent from my iPhone
> >>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
> >>>>>
> >>>>> Andrew,
> >>>>> I am not aware of it being actively worked [1]. This would require
> using
> >>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone
> can
> >>>>> correct me if I'm wrong, but there would have to be changes to both
> sides.
> >>>>> I attempted a quick straw man with grammars, but didn't take it very
> far
> >>>>> after making initial changes to the grammar. It generated code, but
> I'm
> >>>>> uncertain of cross platform compatibility with the expression
> language. If
> >>>>> that's not expected or required that will remove some limitations as
> a
> >>>>> result of moving to ANTLR4.
> >>>>>
> >>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
> >>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-
> c-target-is-here
> >>>>>
> >>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
> >>>>> andrew.christianson@nextcentury.com> wrote:
> >>>>>
> >>>>>> All,
> >>>>>>
> >>>>>> I see that we do not have support for the expression language yet in
> >>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there
> an ETA?
> >>>>>> If no one is working on it, is there a general plan for how it
> should be
> >>>>>> implemented? I think I recall seeing references to ANTLR
>
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
The island grammar sounds very appealing for an MVP. Simple to implement, yet it covers a very common EL use case (dynamically inserting attr vals into property values). If we have general consensus I would love to see a MINIFI JIRA ticket added for this.
-Andy
________________________________________
From: Matt Burgess <ma...@gmail.com>
Sent: Thursday, May 4, 2017 11:09:46 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
What you have sounds good to me. IMO minimum viable product would be an island grammar (meaning you can have any characters outside a ${} expression) and inside would support an attribute name. Next steps could be nested expressions and/or support for functions, added piecemeal as the contributor sees fit. I wasn't involved in the early development of the EL grammar, so maybe someone who was has some thoughts on a natural evolution.
Regards,
Matt
> On May 4, 2017, at 11:00 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> My bad, what does the sketch of the plan *look like*?
>
> -Andy
> ________________________________________
> From: Andrew Christianson
> Sent: Thursday, May 4, 2017 10:59:07 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> What does the sketch of the plan to do the separate implementation? Write a flex/bison grammar, hook it into the cmake build, and start using it? Any constraints on features or syntax that this separate implementation must support?
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:37:34 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage, so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target generation, and test the whole kit and caboodle.
>
> Anyone out there familiar with the aforementioned tools (or willing to hand-roll one)? What do you think about this "EL bootstrapping" approach?
>
> Thanks,
> Matt
>
>> On May 4, 2017, at 9:22 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>> Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target.
>>
>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother project yet?
>>
>> -Andy
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 9:18:00 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> Correct, the current NiFi EL grammar is ANTLR3.
>>
>>
>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>>
>>> Do I understand correctly that NiFi is currently using ANTLRv3?
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 9:05:35 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>>>
>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>>>
>>>
>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>>
>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>>
>>>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>>>
>>>> -Andy
>>>> ________________________________________
>>>> From: Matt Burgess <ma...@gmail.com>
>>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>
>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>>
>>>> Regards,
>>>> Matt
>>>>
>>>> [1] http://dinosaur.compilertools.net/
>>>> [2] http://www.hwaci.com/sw/lemon/
>>>> [3] http://www.colm.net/open-source/ragel/
>>>>
>>>>
>>>>
>>>> Sent from my iPhone
>>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>>
>>>>> Andrew,
>>>>> I am not aware of it being actively worked [1]. This would require using
>>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>>>> I attempted a quick straw man with grammars, but didn't take it very far
>>>>> after making initial changes to the grammar. It generated code, but I'm
>>>>> uncertain of cross platform compatibility with the expression language. If
>>>>> that's not expected or required that will remove some limitations as a
>>>>> result of moving to ANTLR4.
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>>>
>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>>> andrew.christianson@nextcentury.com> wrote:
>>>>>
>>>>>> All,
>>>>>>
>>>>>> I see that we do not have support for the expression language yet in
>>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>>>> If no one is working on it, is there a general plan for how it should be
>>>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@gmail.com>.
What you have sounds good to me. IMO minimum viable product would be an island grammar (meaning you can have any characters outside a ${} expression) and inside would support an attribute name. Next steps could be nested expressions and/or support for functions, added piecemeal as the contributor sees fit. I wasn't involved in the early development of the EL grammar, so maybe someone who was has some thoughts on a natural evolution.
Regards,
Matt
> On May 4, 2017, at 11:00 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> My bad, what does the sketch of the plan *look like*?
>
> -Andy
> ________________________________________
> From: Andrew Christianson
> Sent: Thursday, May 4, 2017 10:59:07 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> What does the sketch of the plan to do the separate implementation? Write a flex/bison grammar, hook it into the cmake build, and start using it? Any constraints on features or syntax that this separate implementation must support?
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:37:34 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage, so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target generation, and test the whole kit and caboodle.
>
> Anyone out there familiar with the aforementioned tools (or willing to hand-roll one)? What do you think about this "EL bootstrapping" approach?
>
> Thanks,
> Matt
>
>> On May 4, 2017, at 9:22 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>> Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target.
>>
>> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother project yet?
>>
>> -Andy
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 9:18:00 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> Correct, the current NiFi EL grammar is ANTLR3.
>>
>>
>>> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>>
>>> Do I understand correctly that NiFi is currently using ANTLRv3?
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 9:05:35 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>>>
>>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>>>
>>>
>>> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>>
>>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>>
>>>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>>>
>>>> -Andy
>>>> ________________________________________
>>>> From: Matt Burgess <ma...@gmail.com>
>>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>>> To: dev@nifi.apache.org
>>>> Subject: Re: MiNiFi C++ Expression Language
>>>>
>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>>
>>>> Regards,
>>>> Matt
>>>>
>>>> [1] http://dinosaur.compilertools.net/
>>>> [2] http://www.hwaci.com/sw/lemon/
>>>> [3] http://www.colm.net/open-source/ragel/
>>>>
>>>>
>>>>
>>>> Sent from my iPhone
>>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>>
>>>>> Andrew,
>>>>> I am not aware of it being actively worked [1]. This would require using
>>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>>>> I attempted a quick straw man with grammars, but didn't take it very far
>>>>> after making initial changes to the grammar. It generated code, but I'm
>>>>> uncertain of cross platform compatibility with the expression language. If
>>>>> that's not expected or required that will remove some limitations as a
>>>>> result of moving to ANTLR4.
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>>>
>>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>>> andrew.christianson@nextcentury.com> wrote:
>>>>>
>>>>>> All,
>>>>>>
>>>>>> I see that we do not have support for the expression language yet in
>>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>>>> If no one is working on it, is there a general plan for how it should be
>>>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
My bad, what does the sketch of the plan *look like*?
-Andy
________________________________________
From: Andrew Christianson
Sent: Thursday, May 4, 2017 10:59:07 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
What does the sketch of the plan to do the separate implementation? Write a flex/bison grammar, hook it into the cmake build, and start using it? Any constraints on features or syntax that this separate implementation must support?
-Andy
________________________________________
From: Matt Burgess <ma...@gmail.com>
Sent: Thursday, May 4, 2017 9:37:34 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage, so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target generation, and test the whole kit and caboodle.
Anyone out there familiar with the aforementioned tools (or willing to hand-roll one)? What do you think about this "EL bootstrapping" approach?
Thanks,
Matt
> On May 4, 2017, at 9:22 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target.
>
> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother project yet?
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:18:00 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> Correct, the current NiFi EL grammar is ANTLR3.
>
>
>> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>> Do I understand correctly that NiFi is currently using ANTLRv3?
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 9:05:35 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>>
>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>>
>>
>> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>
>>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>>
>>> -Andy
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>
>>> Regards,
>>> Matt
>>>
>>> [1] http://dinosaur.compilertools.net/
>>> [2] http://www.hwaci.com/sw/lemon/
>>> [3] http://www.colm.net/open-source/ragel/
>>>
>>>
>>>
>>> Sent from my iPhone
>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>
>>>> Andrew,
>>>> I am not aware of it being actively worked [1]. This would require using
>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>>> I attempted a quick straw man with grammars, but didn't take it very far
>>>> after making initial changes to the grammar. It generated code, but I'm
>>>> uncertain of cross platform compatibility with the expression language. If
>>>> that's not expected or required that will remove some limitations as a
>>>> result of moving to ANTLR4.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>>
>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>> andrew.christianson@nextcentury.com> wrote:
>>>>
>>>>> All,
>>>>>
>>>>> I see that we do not have support for the expression language yet in
>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>>> If no one is working on it, is there a general plan for how it should be
>>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
What does the sketch of the plan to do the separate implementation? Write a flex/bison grammar, hook it into the cmake build, and start using it? Any constraints on features or syntax that this separate implementation must support?
-Andy
________________________________________
From: Matt Burgess <ma...@gmail.com>
Sent: Thursday, May 4, 2017 9:37:34 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage, so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target generation, and test the whole kit and caboodle.
Anyone out there familiar with the aforementioned tools (or willing to hand-roll one)? What do you think about this "EL bootstrapping" approach?
Thanks,
Matt
> On May 4, 2017, at 9:22 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target.
>
> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother project yet?
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:18:00 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> Correct, the current NiFi EL grammar is ANTLR3.
>
>
>> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>> Do I understand correctly that NiFi is currently using ANTLRv3?
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 9:05:35 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>>
>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>>
>>
>> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>
>>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>>
>>> -Andy
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>
>>> Regards,
>>> Matt
>>>
>>> [1] http://dinosaur.compilertools.net/
>>> [2] http://www.hwaci.com/sw/lemon/
>>> [3] http://www.colm.net/open-source/ragel/
>>>
>>>
>>>
>>> Sent from my iPhone
>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>
>>>> Andrew,
>>>> I am not aware of it being actively worked [1]. This would require using
>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>>> I attempted a quick straw man with grammars, but didn't take it very far
>>>> after making initial changes to the grammar. It generated code, but I'm
>>>> uncertain of cross platform compatibility with the expression language. If
>>>> that's not expected or required that will remove some limitations as a
>>>> result of moving to ANTLR4.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>>
>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>> andrew.christianson@nextcentury.com> wrote:
>>>>
>>>>> All,
>>>>>
>>>>> I see that we do not have support for the expression language yet in
>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>>> If no one is working on it, is there a general plan for how it should be
>>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@gmail.com>.
No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage, so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target generation, and test the whole kit and caboodle.
Anyone out there familiar with the aforementioned tools (or willing to hand-roll one)? What do you think about this "EL bootstrapping" approach?
Thanks,
Matt
> On May 4, 2017, at 9:22 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target.
>
> I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother project yet?
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:18:00 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> Correct, the current NiFi EL grammar is ANTLR3.
>
>
>> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>> Do I understand correctly that NiFi is currently using ANTLRv3?
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 9:05:35 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>>
>> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>>
>>
>> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>>
>>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>
>>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>>
>>> -Andy
>>> ________________________________________
>>> From: Matt Burgess <ma...@gmail.com>
>>> Sent: Thursday, May 4, 2017 8:46:20 AM
>>> To: dev@nifi.apache.org
>>> Subject: Re: MiNiFi C++ Expression Language
>>>
>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>>
>>> Regards,
>>> Matt
>>>
>>> [1] http://dinosaur.compilertools.net/
>>> [2] http://www.hwaci.com/sw/lemon/
>>> [3] http://www.colm.net/open-source/ragel/
>>>
>>>
>>>
>>> Sent from my iPhone
>>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>>
>>>> Andrew,
>>>> I am not aware of it being actively worked [1]. This would require using
>>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>>> I attempted a quick straw man with grammars, but didn't take it very far
>>>> after making initial changes to the grammar. It generated code, but I'm
>>>> uncertain of cross platform compatibility with the expression language. If
>>>> that's not expected or required that will remove some limitations as a
>>>> result of moving to ANTLR4.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>>
>>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>>> andrew.christianson@nextcentury.com> wrote:
>>>>
>>>>> All,
>>>>>
>>>>> I see that we do not have support for the expression language yet in
>>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>>> If no one is working on it, is there a general plan for how it should be
>>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target.
I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother project yet?
-Andy
________________________________________
From: Matt Burgess <ma...@gmail.com>
Sent: Thursday, May 4, 2017 9:18:00 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
Correct, the current NiFi EL grammar is ANTLR3.
> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> Do I understand correctly that NiFi is currently using ANTLRv3?
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:05:35 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>
> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>
>
> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>
>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>
>> -Andy
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 8:46:20 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>
>> Regards,
>> Matt
>>
>> [1] http://dinosaur.compilertools.net/
>> [2] http://www.hwaci.com/sw/lemon/
>> [3] http://www.colm.net/open-source/ragel/
>>
>>
>>
>> Sent from my iPhone
>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>
>>> Andrew,
>>> I am not aware of it being actively worked [1]. This would require using
>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>> I attempted a quick straw man with grammars, but didn't take it very far
>>> after making initial changes to the grammar. It generated code, but I'm
>>> uncertain of cross platform compatibility with the expression language. If
>>> that's not expected or required that will remove some limitations as a
>>> result of moving to ANTLR4.
>>>
>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>
>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>> andrew.christianson@nextcentury.com> wrote:
>>>
>>>> All,
>>>>
>>>> I see that we do not have support for the expression language yet in
>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>> If no one is working on it, is there a general plan for how it should be
>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@gmail.com>.
Correct, the current NiFi EL grammar is ANTLR3.
> On May 4, 2017, at 9:12 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
> Do I understand correctly that NiFi is currently using ANTLRv3?
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 9:05:35 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
>
> An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
>
>
> On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>
>>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>
>> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>>
>> -Andy
>> ________________________________________
>> From: Matt Burgess <ma...@gmail.com>
>> Sent: Thursday, May 4, 2017 8:46:20 AM
>> To: dev@nifi.apache.org
>> Subject: Re: MiNiFi C++ Expression Language
>>
>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>>
>> Regards,
>> Matt
>>
>> [1] http://dinosaur.compilertools.net/
>> [2] http://www.hwaci.com/sw/lemon/
>> [3] http://www.colm.net/open-source/ragel/
>>
>>
>>
>> Sent from my iPhone
>>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>>
>>> Andrew,
>>> I am not aware of it being actively worked [1]. This would require using
>>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>>> correct me if I'm wrong, but there would have to be changes to both sides.
>>> I attempted a quick straw man with grammars, but didn't take it very far
>>> after making initial changes to the grammar. It generated code, but I'm
>>> uncertain of cross platform compatibility with the expression language. If
>>> that's not expected or required that will remove some limitations as a
>>> result of moving to ANTLR4.
>>>
>>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>>
>>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>>> andrew.christianson@nextcentury.com> wrote:
>>>
>>>> All,
>>>>
>>>> I see that we do not have support for the expression language yet in
>>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>>> If no one is working on it, is there a general plan for how it should be
>>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
Do I understand correctly that NiFi is currently using ANTLRv3?
________________________________________
From: Matt Burgess <ma...@gmail.com>
Sent: Thursday, May 4, 2017 9:05:35 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>
> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 8:46:20 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>
> Regards,
> Matt
>
> [1] http://dinosaur.compilertools.net/
> [2] http://www.hwaci.com/sw/lemon/
> [3] http://www.colm.net/open-source/ragel/
>
>
>
> Sent from my iPhone
>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>
>> Andrew,
>> I am not aware of it being actively worked [1]. This would require using
>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>> correct me if I'm wrong, but there would have to be changes to both sides.
>> I attempted a quick straw man with grammars, but didn't take it very far
>> after making initial changes to the grammar. It generated code, but I'm
>> uncertain of cross platform compatibility with the expression language. If
>> that's not expected or required that will remove some limitations as a
>> result of moving to ANTLR4.
>>
>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>
>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>> andrew.christianson@nextcentury.com> wrote:
>>
>>> All,
>>>
>>> I see that we do not have support for the expression language yet in
>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>> If no one is working on it, is there a general plan for how it should be
>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@gmail.com>.
I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :)
An ANTLR4 refactor (assuming the C++ target is in good shape) would give us NiFi/MiNiFi EL compatibility (and full-featured EL support in MiNiFi C++), but we'd have to accept the risks of introducing bugs, regressions, etc. as a result of the refactor. Basically we'd just need to test the heck out of it on all platforms, which isn't a bad thing but adds to the LOE for the ANTLR4 upgrade, versus a smaller testing "surface" for incremental development of a C/C++ based grammar.
On May 4, 2017, at 8:51 AM, Andrew Christianson <an...@nextcentury.com> wrote:
>> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>
> This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
>
> -Andy
> ________________________________________
> From: Matt Burgess <ma...@gmail.com>
> Sent: Thursday, May 4, 2017 8:46:20 AM
> To: dev@nifi.apache.org
> Subject: Re: MiNiFi C++ Expression Language
>
> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
>
> Regards,
> Matt
>
> [1] http://dinosaur.compilertools.net/
> [2] http://www.hwaci.com/sw/lemon/
> [3] http://www.colm.net/open-source/ragel/
>
>
>
> Sent from my iPhone
>> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>>
>> Andrew,
>> I am not aware of it being actively worked [1]. This would require using
>> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
>> correct me if I'm wrong, but there would have to be changes to both sides.
>> I attempted a quick straw man with grammars, but didn't take it very far
>> after making initial changes to the grammar. It generated code, but I'm
>> uncertain of cross platform compatibility with the expression language. If
>> that's not expected or required that will remove some limitations as a
>> result of moving to ANTLR4.
>>
>> [1] https://issues.apache.org/jira/browse/MINIFI-140
>> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>>
>> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
>> andrew.christianson@nextcentury.com> wrote:
>>
>>> All,
>>>
>>> I see that we do not have support for the expression language yet in
>>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>>> If no one is working on it, is there a general plan for how it should be
>>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Andrew Christianson <an...@nextcentury.com>.
> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
This seems like a pragmatic approach. What's the level-of-effort required to do the initial grammar port and set up the build tooling? Less than refactoring for ANTLR4? I'm not as familiar with the EL grammar situation.
-Andy
________________________________________
From: Matt Burgess <ma...@gmail.com>
Sent: Thursday, May 4, 2017 8:46:20 AM
To: dev@nifi.apache.org
Subject: Re: MiNiFi C++ Expression Language
I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
Regards,
Matt
[1] http://dinosaur.compilertools.net/
[2] http://www.hwaci.com/sw/lemon/
[3] http://www.colm.net/open-source/ragel/
Sent from my iPhone
> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>
> Andrew,
> I am not aware of it being actively worked [1]. This would require using
> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
> correct me if I'm wrong, but there would have to be changes to both sides.
> I attempted a quick straw man with grammars, but didn't take it very far
> after making initial changes to the grammar. It generated code, but I'm
> uncertain of cross platform compatibility with the expression language. If
> that's not expected or required that will remove some limitations as a
> result of moving to ANTLR4.
>
> [1] https://issues.apache.org/jira/browse/MINIFI-140
> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>
> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
> andrew.christianson@nextcentury.com> wrote:
>
>> All,
>>
>> I see that we do not have support for the expression language yet in
>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>> If no one is working on it, is there a general plan for how it should be
>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by Matt Burgess <ma...@gmail.com>.
I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing that with all the MiNiFi components already. The upside is being able to support EL incrementally as the grammar is developed. What do you think?
Regards,
Matt
[1] http://dinosaur.compilertools.net/
[2] http://www.hwaci.com/sw/lemon/
[3] http://www.colm.net/open-source/ragel/
Sent from my iPhone
> On May 4, 2017, at 8:13 AM, Marc P. <ma...@gmail.com> wrote:
>
> Andrew,
> I am not aware of it being actively worked [1]. This would require using
> ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
> correct me if I'm wrong, but there would have to be changes to both sides.
> I attempted a quick straw man with grammars, but didn't take it very far
> after making initial changes to the grammar. It generated code, but I'm
> uncertain of cross platform compatibility with the expression language. If
> that's not expected or required that will remove some limitations as a
> result of moving to ANTLR4.
>
> [1] https://issues.apache.org/jira/browse/MINIFI-140
> [2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
>
> On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
> andrew.christianson@nextcentury.com> wrote:
>
>> All,
>>
>> I see that we do not have support for the expression language yet in
>> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
>> If no one is working on it, is there a general plan for how it should be
>> implemented? I think I recall seeing references to ANTLR
Re: MiNiFi C++ Expression Language
Posted by "Marc P." <ma...@gmail.com>.
Andrew,
I am not aware of it being actively worked [1]. This would require using
ANTLR4, but I don't believe C++ support is well tested [2]. Someone can
correct me if I'm wrong, but there would have to be changes to both sides.
I attempted a quick straw man with grammars, but didn't take it very far
after making initial changes to the grammar. It generated code, but I'm
uncertain of cross platform compatibility with the expression language. If
that's not expected or required that will remove some limitations as a
result of moving to ANTLR4.
[1] https://issues.apache.org/jira/browse/MINIFI-140
[2] http://www.soft-gems.net/index.php/tools/49-the-antlr4-c-target-is-here
On Thu, May 4, 2017 at 8:07 AM, Andrew Christianson <
andrew.christianson@nextcentury.com> wrote:
> All,
>
> I see that we do not have support for the expression language yet in
> MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA?
> If no one is working on it, is there a general plan for how it should be
> implemented? I think I recall seeing references to ANTLR