Serialization Formats, Previously: OSGi

[Peter <ji...@zeus.net.au>]

I think we're getting off topic, can we create a new thread, as we're no 
longer discussing OSGi, but the virtues of various serialization formats?

Cheers,

Peter.

On 4/02/2017 9:09 PM, Niclas Hedhman wrote:
> see below
>
> On Sat, Feb 4, 2017 at 6:21 PM, "Micha\u0142 K\u0142eczek (XPro Sp. z o. o.)"<
> michal.kleczek@xpro.biz>  wrote:
>> Once you transfer the code with your data - the issue of code version
> synchronization disappears, doesn't it?
>> It also makes the wire data format irrelevant. At least for "short lived
> serialized states".
>
> Only works if you have no exchange with the environment it is executing.
> And this is where "sandboxing" concern kicks in. What is the sandbox? In a
> web browser they try to define it to DOM + handful of other well-defined
> objects. In case of Java Serialization, it is all classes reachable from
> the loading classloader. And I think Gregg is trying to argue that if one
> is very prudent, one need to manage this well.
>
>> I fail to understand how JSON or XML changes anything here.
> It doesn't. It makes the issues more visible, and more easy to recover when
> something goes wrong. Java Serialization is sold "too generic" and hence
> abused by everyone (myself included) and we (except Gregg and other gurus)
> get burned badly somewhere along the way.
>
>> In the end all of the arguments against Java Object Serialization boil
> down to:
>> "It is easy to use but if not used carefully it will bite you - so it is
> too easy to use"
>
> Well, kind of...
> If you ever need to deserialize a serialVersionUid=1 with a codebase where
> it is now serialVersionUid != 1, I wouldn't call it "easy to use" anymore.
> Back in the days when I used this stuff heavily, I ended up never change
> serialVersionUid. If I needed to refactor it enough to loose compatibility,
> I would create a new class and make an adapter.
>
>> What I do not like about Java Object Serialization has nothing to do with
> the format of persistent data
>> but rather with the APIs - it is inherently blocking by design.
> I am not sure I understand what you mean by that.
>
> Cheers
> --
> Niclas Hedhman, Software Developer
> http://polygene.apache.org - New Energy for Java
>

Re: Serialization Formats, Previously: OSGi

[Niclas Hedhman <ni...@hedhman.org>]

Thumbs Up! Agree...

On Sun, Feb 5, 2017 at 1:33 AM, "Michał Kłeczek (XPro Sp. z o. o.)" <
michal.kleczek@xpro.biz> wrote:

> I cannot disagree with rants about software industry state.
>
> Let's get back to technical solutions to non-technical problems. I am
> interested in providing tools - whether will be used... is a different
> story.
>
> That said...
> IMHO Jini - in all its greatness - DID NOT solve the problem of Java code
> mobility in any way.
> As has been discussed on this list several time the way how it "solved" it
> is:
> - inherently insecure (because object validation is done _after_ code
> execution)
> - is not capable of transferring complicated object graphs - hence it
> cannot be used in many different interesting scenarios.
>
> Partial solutions are worse than lack of solutions - they confuse users
> (in our case programmers) and in the end people loose interest.
>
> I am not a big fan of Java containers - be it JEE or any other (OSGI
> included)
> The industry seems to understand they are a dead end - escpecially in the
> age of Docker etc - and is moving away from them (not that in a very
> meaningful direction :) ).
>
> I have worked with OSGI for several years and it was a difficult
> relationship :)
> Today I prefer simpler solutions: "java -jar my-very-complicated-and-important-service.jar"
> is the way to go.
>
> Thanks,
> Michal
>
>
>
> Niclas Hedhman wrote:
>
> (I think wrong thread, so to please Peter, I copied it all into here)
>
> Correct, it is not different. But you are missing the point; CONSTRAINTS.
> And without constraints, developers are capable of doing these good deeds
> (such as your example) and many very poor ones. The knife cuts your meat or
> butcher your neighbor... It is all about the constraints, something that
> few developers are willing to admit that makes our work better.
>
> As for the "leasable and you have..."; The problem is that you are probably
> wrong on that front too, like the OSGi community have learned the hard way.
> There are too many ways software entangle classloading. All kinds of shit
> "registers itself" in the bowels of the runtime, such as with the
> DriverManager, Loggers, URLHandlers or PermGenSpace (might be gone in Java
> 8). Then add 100s of common libraries that also does a poor job in
> releasing "permanent" resources/instances/classes... The stain sticks, but
> the smell is weak, so often we can't tell other than memory leak during
> class updates.
> And why do we have all this mess? IMHO; Lack of constraints, lack of
> lifecycle management in "everything Java" (and most languages) and lack of
> discipline (something Gregg has, and I plus 5 million other Java devs don't
> have). OSGi is not as successful as it "should" (SpringSource gave up)
> because it makes the "stain" stink really badly. OSGi introduces
> constraints and fails spectacular when we try to break or circumvent them.
>
> River as it currently stands has "solved" Java code mobility, Java leases,
> dynamic service registry with query capabilities and much more. But as with
> a lot of good technology, the masses don't buy it. The ignorant masses are
> now in Peter Deutsch's Fallacies of Distributed Computing territory,
> thinking that microservices on JAX-RS is going to save the day (it isn't, I
> am rescuing a project out of it now).
> Distributed OSGi tried to solve this problem, and AFAICT has serious
> problems to work reliably in production environments. What do I learn? This
> is hard, but every 5 years we double in numbers, so half the developer
> population is inexperienced and repeat the same mistakes again and again.
>
> Sorry for highlighting problems, mostly psychological/philosophical rather
> than technological. I don't have the answers, other than; Without
> Constraints Technology Fails. And the better the constraints are defined,
> the better likelihood that it can succeed.
>
>
>
>
> On Sat, Feb 4, 2017 at 8:59 PM, "Michał Kłeczek (XPro Sp. z o. o.)" <mi...@xpro.biz> wrote:
>
>
> Comments below.
>
> Niclas Hedhman wrote:
>
> see below
>
> On Sat, Feb 4, 2017 at 6:21 PM, "Michał Kłeczek (XPro Sp. z o. o.)" <mi...@xpro.biz> <mi...@xpro.biz> wrote:
>
> Once you transfer the code with your data - the issue of code version
>
> synchronization disappears, doesn't it?
>
> It also makes the wire data format irrelevant. At least for "short lived
>
> serialized states".
>
> Only works if you have no exchange with the environment it is executing.
> And this is where "sandboxing" concern kicks in. What is the sandbox? In a
> web browser they try to define it to DOM + handful of other well-defined
> objects. In case of Java Serialization, it is all classes reachable from
> the loading classloader. And I think Gregg is trying to argue that if one
> is very prudent, one need to manage this well.
>
>
> But how is "exchange with the environment it is executing"
> actually different when installing code on demand from installing it in
> advance???
>
> The whole point IMHO is to shift thinking from "moving data" to "exchange
> configured software" -
> think Java specific Docker on steroids.
>
> Transferable objects allow you for example to do things like
> downloading your JDBC driver automagically without the fuss of installing
> it and managing upgrades.
> Just publish a DataSource object in your ServiceRegistrar and you are done.
> Make it leasable and you have automatic upgrades and/or reconfiguration.
>
> Thanks,
> Michal
>
>
>
>
>


-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org <http://zest.apache.org> - New Energy for Java

Re: Serialization Formats, Previously: OSGi

["Michał Kłeczek (XPro Sp. z o. o.)" <mi...@xpro.biz>]

I cannot disagree with rants about software industry state.

Let's get back to technical solutions to non-technical problems. I am 
interested in providing tools - whether will be used... is a different 
story.

That said...
IMHO Jini - in all its greatness - DID NOT solve the problem of Java 
code mobility in any way.
As has been discussed on this list several time the way how it "solved" 
it is:
- inherently insecure (because object validation is done _after_ code 
execution)
- is not capable of transferring complicated object graphs - hence it 
cannot be used in many different interesting scenarios.

Partial solutions are worse than lack of solutions - they confuse users 
(in our case programmers) and in the end people loose interest.

I am not a big fan of Java containers - be it JEE or any other (OSGI 
included)
The industry seems to understand they are a dead end - escpecially in 
the age of Docker etc - and is moving away from them (not that in a very 
meaningful direction :) ).

I have worked with OSGI for several years and it was a difficult 
relationship :)
Today I prefer simpler solutions: "java -jar 
my-very-complicated-and-important-service.jar" is the way to go.

Thanks,
Michal


Niclas Hedhman wrote:
> (I think wrong thread, so to please Peter, I copied it all into here)
>
> Correct, it is not different. But you are missing the point; CONSTRAINTS.
> And without constraints, developers are capable of doing these good deeds
> (such as your example) and many very poor ones. The knife cuts your meat or
> butcher your neighbor... It is all about the constraints, something that
> few developers are willing to admit that makes our work better.
>
> As for the "leasable and you have..."; The problem is that you are probably
> wrong on that front too, like the OSGi community have learned the hard way.
> There are too many ways software entangle classloading. All kinds of shit
> "registers itself" in the bowels of the runtime, such as with the
> DriverManager, Loggers, URLHandlers or PermGenSpace (might be gone in Java
> 8). Then add 100s of common libraries that also does a poor job in
> releasing "permanent" resources/instances/classes... The stain sticks, but
> the smell is weak, so often we can't tell other than memory leak during
> class updates.
> And why do we have all this mess? IMHO; Lack of constraints, lack of
> lifecycle management in "everything Java" (and most languages) and lack of
> discipline (something Gregg has, and I plus 5 million other Java devs don't
> have). OSGi is not as successful as it "should" (SpringSource gave up)
> because it makes the "stain" stink really badly. OSGi introduces
> constraints and fails spectacular when we try to break or circumvent them.
>
> River as it currently stands has "solved" Java code mobility, Java leases,
> dynamic service registry with query capabilities and much more. But as with
> a lot of good technology, the masses don't buy it. The ignorant masses are
> now in Peter Deutsch's Fallacies of Distributed Computing territory,
> thinking that microservices on JAX-RS is going to save the day (it isn't, I
> am rescuing a project out of it now).
> Distributed OSGi tried to solve this problem, and AFAICT has serious
> problems to work reliably in production environments. What do I learn? This
> is hard, but every 5 years we double in numbers, so half the developer
> population is inexperienced and repeat the same mistakes again and again.
>
> Sorry for highlighting problems, mostly psychological/philosophical rather
> than technological. I don't have the answers, other than; Without
> Constraints Technology Fails. And the better the constraints are defined,
> the better likelihood that it can succeed.
>
>
>
>
> On Sat, Feb 4, 2017 at 8:59 PM, "Micha\u0142 K\u0142eczek (XPro Sp. z o. o.)"<
> michal.kleczek@xpro.biz>  wrote:
>
>> Comments below.
>>
>> Niclas Hedhman wrote:
>>
>> see below
>>
>> On Sat, Feb 4, 2017 at 6:21 PM, "Micha\u0142 K\u0142eczek (XPro Sp. z o. o.)"<mi...@xpro.biz>  wrote:
>>
>> Once you transfer the code with your data - the issue of code version
>>
>> synchronization disappears, doesn't it?
>>
>> It also makes the wire data format irrelevant. At least for "short lived
>>
>> serialized states".
>>
>> Only works if you have no exchange with the environment it is executing.
>> And this is where "sandboxing" concern kicks in. What is the sandbox? In a
>> web browser they try to define it to DOM + handful of other well-defined
>> objects. In case of Java Serialization, it is all classes reachable from
>> the loading classloader. And I think Gregg is trying to argue that if one
>> is very prudent, one need to manage this well.
>>
>>
>> But how is "exchange with the environment it is executing"
>> actually different when installing code on demand from installing it in
>> advance???
>>
>> The whole point IMHO is to shift thinking from "moving data" to "exchange
>> configured software" -
>> think Java specific Docker on steroids.
>>
>> Transferable objects allow you for example to do things like
>> downloading your JDBC driver automagically without the fuss of installing
>> it and managing upgrades.
>> Just publish a DataSource object in your ServiceRegistrar and you are done.
>> Make it leasable and you have automatic upgrades and/or reconfiguration.
>>
>> Thanks,
>> Michal
>>
>
>
>

Re: Serialization Formats, Previously: OSGi

[Niclas Hedhman <ni...@hedhman.org>]

(I think wrong thread, so to please Peter, I copied it all into here)

Correct, it is not different. But you are missing the point; CONSTRAINTS.
And without constraints, developers are capable of doing these good deeds
(such as your example) and many very poor ones. The knife cuts your meat or
butcher your neighbor... It is all about the constraints, something that
few developers are willing to admit that makes our work better.

As for the "leasable and you have..."; The problem is that you are probably
wrong on that front too, like the OSGi community have learned the hard way.
There are too many ways software entangle classloading. All kinds of shit
"registers itself" in the bowels of the runtime, such as with the
DriverManager, Loggers, URLHandlers or PermGenSpace (might be gone in Java
8). Then add 100s of common libraries that also does a poor job in
releasing "permanent" resources/instances/classes... The stain sticks, but
the smell is weak, so often we can't tell other than memory leak during
class updates.
And why do we have all this mess? IMHO; Lack of constraints, lack of
lifecycle management in "everything Java" (and most languages) and lack of
discipline (something Gregg has, and I plus 5 million other Java devs don't
have). OSGi is not as successful as it "should" (SpringSource gave up)
because it makes the "stain" stink really badly. OSGi introduces
constraints and fails spectacular when we try to break or circumvent them.

River as it currently stands has "solved" Java code mobility, Java leases,
dynamic service registry with query capabilities and much more. But as with
a lot of good technology, the masses don't buy it. The ignorant masses are
now in Peter Deutsch's Fallacies of Distributed Computing territory,
thinking that microservices on JAX-RS is going to save the day (it isn't, I
am rescuing a project out of it now).
Distributed OSGi tried to solve this problem, and AFAICT has serious
problems to work reliably in production environments. What do I learn? This
is hard, but every 5 years we double in numbers, so half the developer
population is inexperienced and repeat the same mistakes again and again.

Sorry for highlighting problems, mostly psychological/philosophical rather
than technological. I don't have the answers, other than; Without
Constraints Technology Fails. And the better the constraints are defined,
the better likelihood that it can succeed.




On Sat, Feb 4, 2017 at 8:59 PM, "Michał Kłeczek (XPro Sp. z o. o.)" <
michal.kleczek@xpro.biz> wrote:

> Comments below.
>
> Niclas Hedhman wrote:
>
> see below
>
> On Sat, Feb 4, 2017 at 6:21 PM, "Michał Kłeczek (XPro Sp. z o. o.)" <mi...@xpro.biz> wrote:
>
> Once you transfer the code with your data - the issue of code version
>
> synchronization disappears, doesn't it?
>
> It also makes the wire data format irrelevant. At least for "short lived
>
> serialized states".
>
> Only works if you have no exchange with the environment it is executing.
> And this is where "sandboxing" concern kicks in. What is the sandbox? In a
> web browser they try to define it to DOM + handful of other well-defined
> objects. In case of Java Serialization, it is all classes reachable from
> the loading classloader. And I think Gregg is trying to argue that if one
> is very prudent, one need to manage this well.
>
>
> But how is "exchange with the environment it is executing"
> actually different when installing code on demand from installing it in
> advance???
>
> The whole point IMHO is to shift thinking from "moving data" to "exchange
> configured software" -
> think Java specific Docker on steroids.
>
> Transferable objects allow you for example to do things like
> downloading your JDBC driver automagically without the fuss of installing
> it and managing upgrades.
> Just publish a DataSource object in your ServiceRegistrar and you are done.
> Make it leasable and you have automatic upgrades and/or reconfiguration.
>
> Thanks,
> Michal
>



-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org <http://zest.apache.org/> - New Energy for Java