You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Romain Manni-Bucau <rm...@gmail.com> on 2018/04/25 07:09:56 UTC
Beam to move to JSON-B
Hi guys,
Currently Beam uses jackson, I opened BEAM-3992 to move to JSON-B since
beam has no choice
on the dependencies it uses by nature (it is embedded in an environment
with its own stack) and I'd like to see the current workaround to shade the
world to stop because it leads to way too big deliveries for what it does
and it slows down builds for no end users gain.
The move is straight forward for all clean code and shouldn't be a big deal
for beam.
I'm happy if somebody wants to take the lead on that and I'd be happy to
support anyone new to JSON-B if needed.
To be concrete (or give some rational), I already have some environment
where jackson will be dropped in favor of existing alternative
implementations and it enables to "submit" (run) a pipeline with one
implementation and execute it in a cluster with another without having to
duplicate and mess up the libs in both. This is critical to let ops handle
vulnerabilities properly and without impacting the whole company ecosystem
in a lot of cases.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Re: Beam to move to JSON-B
Posted by Kenneth Knowles <kl...@google.com>.
On Wed, Apr 25, 2018 at 1:26 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Shading also introduces issues companies don't want (like a bad tracking
> of vulnerabilities opening backdoors in beam etc.
>
This is a very interesting point that I hadn't heard before.
Thanks for the rest of the write up. That clarifies a lot. Seems reasonable
and useful to me. It actually sounds like how users interact with it is
"they don't unless they want to" which is good, and they can suppress and
replace if they want to.
Kenn
> IMHO shading must be abandonned ause it is not reliable for end users (not
> speaking just of the runtime here but more of company concerns more widely)
>
>
>> - What is the scope of change to Beam?
>>
>
> If you list beam dependencies there shouldnt be jackson as anything
> "compile" but only potentially "runtime".
>
>
>>
>> I'll leave off the shading discussion since we already had one. You can
>> continue this one:
>>
>> https://lists.apache.org/thread.html/12383d2e5d70026427df43294e30d6524334e16f03d86c9a5860792f@%3Cdev.beam.apache.org%3E
>> or work on https://issues.apache.org/jira/browse/BEAM-3608 and similar
>> to improve things. Should be very easy now. I didn't work on it because
>> work on poms is throwaway. Now is a good time to pick it up.
>>
>> Kenn
>>
>> On Wed, Apr 25, 2018 at 12:10 AM Romain Manni-Bucau <
>> rmannibucau@gmail.com> wrote:
>>
>>> Hi guys,
>>>
>>> Currently Beam uses jackson, I opened BEAM-3992 to move to JSON-B since
>>> beam has no choice
>>> on the dependencies it uses by nature (it is embedded in an environment
>>> with its own stack) and I'd like to see the current workaround to shade the
>>> world to stop because it leads to way too big deliveries for what it does
>>> and it slows down builds for no end users gain.
>>>
>>> The move is straight forward for all clean code and shouldn't be a big
>>> deal for beam.
>>>
>>> I'm happy if somebody wants to take the lead on that and I'd be happy to
>>> support anyone new to JSON-B if needed.
>>>
>>> To be concrete (or give some rational), I already have some environment
>>> where jackson will be dropped in favor of existing alternative
>>> implementations and it enables to "submit" (run) a pipeline with one
>>> implementation and execute it in a cluster with another without having to
>>> duplicate and mess up the libs in both. This is critical to let ops handle
>>> vulnerabilities properly and without impacting the whole company ecosystem
>>> in a lot of cases.
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>>> <https://rmannibucau.metawerx.net/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github
>>> <https://github.com/rmannibucau> | LinkedIn
>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>>
>>
>
Re: Beam to move to JSON-B
Posted by Romain Manni-Bucau <rm...@gmail.com>.
2018-04-25 21:50 GMT+02:00 Kenneth Knowles <kl...@google.com>:
> I'm not super familiar with JSON-B. The JIRA makes it sound like using
> slf4j instead of JUL directly. In the logging case it is a clear win. In
> this case I do have some questions:
>
> - How do users interact with JSON-B?
>
JSON-B is the API so you:
1. import the api + impl in the pom (sorry build.gradle ;))
2. instantiate an instance: JsonbBuilder.create(/*optional config*/) which
gives you a Jsonb instance (ObjectMapper)
3. use javax.json API instead of jackson one (@JsonbProperty etc)
> - Is it really just a facade/API? The site makes it look like yet another
> JSON library.
>
This is just an API - and jackson implements it ;).
It is a portable API - as any EE API and as Beam should be at the end. It
is not an implementation (jackson, johnzon etc are)
> - What are the reasons we do not shade Jackson?
>
Hope you don't do that it would create way more issues than the ones it
would solve.
On another side beam already shades way too much (and BTW not enough to fix
real conflicts ATM). Shading also introduces issues companies don't want
(like a bad tracking of vulnerabilities opening backdoors in beam etc. IMHO
shading must be abandonned ause it is not reliable for end users (not
speaking just of the runtime here but more of company concerns more widely)
> - What is the scope of change to Beam?
>
If you list beam dependencies there shouldnt be jackson as anything
"compile" but only potentially "runtime".
>
> I'll leave off the shading discussion since we already had one. You can
> continue this one:
> https://lists.apache.org/thread.html/12383d2e5d70026427df43294e30d6
> 524334e16f03d86c9a5860792f@%3Cdev.beam.apache.org%3E or work on
> https://issues.apache.org/jira/browse/BEAM-3608 and similar to improve
> things. Should be very easy now. I didn't work on it because work on poms
> is throwaway. Now is a good time to pick it up.
>
> Kenn
>
> On Wed, Apr 25, 2018 at 12:10 AM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> Hi guys,
>>
>> Currently Beam uses jackson, I opened BEAM-3992 to move to JSON-B since
>> beam has no choice
>> on the dependencies it uses by nature (it is embedded in an environment
>> with its own stack) and I'd like to see the current workaround to shade the
>> world to stop because it leads to way too big deliveries for what it does
>> and it slows down builds for no end users gain.
>>
>> The move is straight forward for all clean code and shouldn't be a big
>> deal for beam.
>>
>> I'm happy if somebody wants to take the lead on that and I'd be happy to
>> support anyone new to JSON-B if needed.
>>
>> To be concrete (or give some rational), I already have some environment
>> where jackson will be dropped in favor of existing alternative
>> implementations and it enables to "submit" (run) a pipeline with one
>> implementation and execute it in a cluster with another without having to
>> duplicate and mess up the libs in both. This is critical to let ops handle
>> vulnerabilities properly and without impacting the whole company ecosystem
>> in a lot of cases.
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>
>
Re: Beam to move to JSON-B
Posted by Kenneth Knowles <kl...@google.com>.
I'm not super familiar with JSON-B. The JIRA makes it sound like using
slf4j instead of JUL directly. In the logging case it is a clear win. In
this case I do have some questions:
- How do users interact with JSON-B?
- Is it really just a facade/API? The site makes it look like yet another
JSON library.
- What are the reasons we do not shade Jackson?
- What is the scope of change to Beam?
I'll leave off the shading discussion since we already had one. You can
continue this one:
https://lists.apache.org/thread.html/12383d2e5d70026427df43294e30d6524334e16f03d86c9a5860792f@%3Cdev.beam.apache.org%3E
or work on https://issues.apache.org/jira/browse/BEAM-3608 and similar to
improve things. Should be very easy now. I didn't work on it because work
on poms is throwaway. Now is a good time to pick it up.
Kenn
On Wed, Apr 25, 2018 at 12:10 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi guys,
>
> Currently Beam uses jackson, I opened BEAM-3992 to move to JSON-B since
> beam has no choice
> on the dependencies it uses by nature (it is embedded in an environment
> with its own stack) and I'd like to see the current workaround to shade the
> world to stop because it leads to way too big deliveries for what it does
> and it slows down builds for no end users gain.
>
> The move is straight forward for all clean code and shouldn't be a big
> deal for beam.
>
> I'm happy if somebody wants to take the lead on that and I'd be happy to
> support anyone new to JSON-B if needed.
>
> To be concrete (or give some rational), I already have some environment
> where jackson will be dropped in favor of existing alternative
> implementations and it enables to "submit" (run) a pipeline with one
> implementation and execute it in a cluster with another without having to
> duplicate and mess up the libs in both. This is critical to let ops handle
> vulnerabilities properly and without impacting the whole company ecosystem
> in a lot of cases.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>