You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Adam Jacobs <ja...@hotmail.com> on 2017/11/14 00:12:35 UTC
Immutability
The subject of immutability was raised in JENA-1391 (https://issues.apache.org/jira/browse/JENA-1391).
Specifically, the `getUnionModel` method in Jena 3.4 returns an immutable model view, and the implementation of the aforementioned story includes methods that will return an immutable dataset view.
The question is whether these immutable views deserve their own interfaces. Currently, the views are returned using what I called "unexpected immutability" because they implement mutable interfaces. This introduces the potential for `UnsupportedOperationException`s.
Unfortunately, that (degenerate) pattern is used in Java's `Collections` utility as well (https://docs.oracle.com/javase/8/docs/api/java/util/Collections.html) but Scala is a clean example to draw inspiration from: by implementing immutable interfaces as parents to their mutable counterparts (rather than vice verse) we can satisfy the Liskov Substitution Principle.
Obviously, implementing that solution is easier to do from scratch than in an existing code base; but I imagine it could be done in multiple phases, by introducing the new interfaces and using them in new methods (with easy conversion to mutability via union) while gradually retrofitting older methods.
The question then, is whether such a change is worthwhile...
Re: potential new API Was: Immutability
Posted by ajs6f <aj...@apache.org>.
Progress on 1391 hasn't been blocked on a new API, not at all. Just taking a good while because of general busy-ness. I was able to write some more tests over the last few days and I think I should be able to finish up very early in January. You can track progress at:
https://github.com/apache/jena/compare/master...ajs6f:JENA-1391
ajs6f
> On Dec 17, 2017, at 8:15 PM, Adam Jacobs <ja...@hotmail.com> wrote:
>
> I just want to mention that it was not my intent to derail progress on Jena-1391 (https://issues.apache.org/jira/browse/JENA-1391) by raising the issue of immutability,
> particularly if immutability would be implemented via a new API. I don't think it should be considered a blocker to that story. Apologies if I threw a wrench into the plan.
>
>
> ________________________________________
> From: ajs6f <aj...@apache.org>
> Sent: Wednesday, December 13, 2017 10:39 AM
> To: dev@jena.apache.org
> Subject: potential new API Was: Immutability
>
> Yes, the SPI is likely fine, or we will find out otherwise if we try a new API. :)
>
> A new API is clearly no small thing. I suppose one way to approach the thing would be to look at what Clerezza and RDF4J 4 and Commons RDF have come up with, what worked and what didn't, and then bring in our own use cases and ideas...
>
> The two "asks" I've heard recently from Jena users are: immutability and Streams. I'm guessing there are others. I would like to take advantage of Optional for some of the attributes of a node (e.g. return the lang of a literal as Optional).
>
> Other thoughts? It's fun to make wish lists at the end of the year. :grin:
>
> ajs6f
>
>> On Nov 16, 2017, at 11:47 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> Yes, I think it is best done as a new API.
>>
>> Modifications to the existing one are likely to run into problems somewhere and given the number of users, and the change over time on upgrades, having compatibility matters as does gradual migration
>>
>> A complicates case: Dataset.getModel(String) returns a model. No idea whether it is immutable or not or how it will be used. Assigning to a ImmutableModel does not guaratee anything - it helps the caller not make mistakes but the model is not immutable. Some code calling getModel can change it. And the union model isn't immutable! - it's a dynamic union and any name graph change will be reflected. The world is multithreaded.
>>
>> But the main reason for a new API is that this is just one of several features to include so to make it a success these different features need to come together into one *agreed* design.
>>
>> The good news is that it can target the SPI.
>> (I see no value in changing the SPI for this - that's not it's role)
>>
>> ARQ's API is really very small. A design that took a dataset+model view of the world would be a good thing.
>>
>> Andy
>>
>> See also commons-rdf.
>> That is RDF 1.1 compliant
>>
>> On 15/11/17 15:52, ajs6f@apache.org wrote:
>>> So it seems to me that if we want to introduce immutable types, we might want to do that in the context of a completely new API. The use of the Java 8 Streams API is also something that has been mooted as something that might merit a new Jena API. (Instead of mixing things up in the current one.)
>>> I'm not sure how that plays out with ARQ, though. We would want people to be able to use the new types with ARQ without much difficulty.
>
Re: potential new API Was: Immutability
Posted by Adam Jacobs <ja...@hotmail.com>.
I just want to mention that it was not my intent to derail progress on Jena-1391 (https://issues.apache.org/jira/browse/JENA-1391) by raising the issue of immutability,
particularly if immutability would be implemented via a new API. I don't think it should be considered a blocker to that story. Apologies if I threw a wrench into the plan.
________________________________________
From: ajs6f <aj...@apache.org>
Sent: Wednesday, December 13, 2017 10:39 AM
To: dev@jena.apache.org
Subject: potential new API Was: Immutability
Yes, the SPI is likely fine, or we will find out otherwise if we try a new API. :)
A new API is clearly no small thing. I suppose one way to approach the thing would be to look at what Clerezza and RDF4J 4 and Commons RDF have come up with, what worked and what didn't, and then bring in our own use cases and ideas...
The two "asks" I've heard recently from Jena users are: immutability and Streams. I'm guessing there are others. I would like to take advantage of Optional for some of the attributes of a node (e.g. return the lang of a literal as Optional).
Other thoughts? It's fun to make wish lists at the end of the year. :grin:
ajs6f
> On Nov 16, 2017, at 11:47 AM, Andy Seaborne <an...@apache.org> wrote:
>
> Yes, I think it is best done as a new API.
>
> Modifications to the existing one are likely to run into problems somewhere and given the number of users, and the change over time on upgrades, having compatibility matters as does gradual migration
>
> A complicates case: Dataset.getModel(String) returns a model. No idea whether it is immutable or not or how it will be used. Assigning to a ImmutableModel does not guaratee anything - it helps the caller not make mistakes but the model is not immutable. Some code calling getModel can change it. And the union model isn't immutable! - it's a dynamic union and any name graph change will be reflected. The world is multithreaded.
>
> But the main reason for a new API is that this is just one of several features to include so to make it a success these different features need to come together into one *agreed* design.
>
> The good news is that it can target the SPI.
> (I see no value in changing the SPI for this - that's not it's role)
>
> ARQ's API is really very small. A design that took a dataset+model view of the world would be a good thing.
>
> Andy
>
> See also commons-rdf.
> That is RDF 1.1 compliant
>
> On 15/11/17 15:52, ajs6f@apache.org wrote:
>> So it seems to me that if we want to introduce immutable types, we might want to do that in the context of a completely new API. The use of the Java 8 Streams API is also something that has been mooted as something that might merit a new Jena API. (Instead of mixing things up in the current one.)
>> I'm not sure how that plays out with ARQ, though. We would want people to be able to use the new types with ARQ without much difficulty.
potential new API Was: Immutability
Posted by ajs6f <aj...@apache.org>.
Yes, the SPI is likely fine, or we will find out otherwise if we try a new API. :)
A new API is clearly no small thing. I suppose one way to approach the thing would be to look at what Clerezza and RDF4J 4 and Commons RDF have come up with, what worked and what didn't, and then bring in our own use cases and ideas...
The two "asks" I've heard recently from Jena users are: immutability and Streams. I'm guessing there are others. I would like to take advantage of Optional for some of the attributes of a node (e.g. return the lang of a literal as Optional).
Other thoughts? It's fun to make wish lists at the end of the year. :grin:
ajs6f
> On Nov 16, 2017, at 11:47 AM, Andy Seaborne <an...@apache.org> wrote:
>
> Yes, I think it is best done as a new API.
>
> Modifications to the existing one are likely to run into problems somewhere and given the number of users, and the change over time on upgrades, having compatibility matters as does gradual migration
>
> A complicates case: Dataset.getModel(String) returns a model. No idea whether it is immutable or not or how it will be used. Assigning to a ImmutableModel does not guaratee anything - it helps the caller not make mistakes but the model is not immutable. Some code calling getModel can change it. And the union model isn't immutable! - it's a dynamic union and any name graph change will be reflected. The world is multithreaded.
>
> But the main reason for a new API is that this is just one of several features to include so to make it a success these different features need to come together into one *agreed* design.
>
> The good news is that it can target the SPI.
> (I see no value in changing the SPI for this - that's not it's role)
>
> ARQ's API is really very small. A design that took a dataset+model view of the world would be a good thing.
>
> Andy
>
> See also commons-rdf.
> That is RDF 1.1 compliant
>
> On 15/11/17 15:52, ajs6f@apache.org wrote:
>> So it seems to me that if we want to introduce immutable types, we might want to do that in the context of a completely new API. The use of the Java 8 Streams API is also something that has been mooted as something that might merit a new Jena API. (Instead of mixing things up in the current one.)
>> I'm not sure how that plays out with ARQ, though. We would want people to be able to use the new types with ARQ without much difficulty.
Re: Immutability
Posted by Andy Seaborne <an...@apache.org>.
Yes, I think it is best done as a new API.
Modifications to the existing one are likely to run into problems
somewhere and given the number of users, and the change over time on
upgrades, having compatibility matters as does gradual migration
A complicates case: Dataset.getModel(String) returns a model. No idea
whether it is immutable or not or how it will be used. Assigning to a
ImmutableModel does not guaratee anything - it helps the caller not make
mistakes but the model is not immutable. Some code calling getModel can
change it. And the union model isn't immutable! - it's a dynamic union
and any name graph change will be reflected. The world is multithreaded.
But the main reason for a new API is that this is just one of several
features to include so to make it a success these different features
need to come together into one *agreed* design.
The good news is that it can target the SPI.
(I see no value in changing the SPI for this - that's not it's role)
ARQ's API is really very small. A design that took a dataset+model view
of the world would be a good thing.
Andy
See also commons-rdf.
That is RDF 1.1 compliant
On 15/11/17 15:52, ajs6f@apache.org wrote:
> So it seems to me that if we want to introduce immutable types, we might
> want to do that in the context of a completely new API. The use of the
> Java 8 Streams API is also something that has been mooted as something
> that might merit a new Jena API. (Instead of mixing things up in the
> current one.)
>
> I'm not sure how that plays out with ARQ, though. We would want people
> to be able to use the new types with ARQ without much difficulty.
Re: Immutability
Posted by aj...@apache.org.
I think this is one reason that Clerezza introduced a new RDF API:
http://clerezza.apache.org/apidocs/org/apache/clerezza/rdf/core/package-summary.html
So it seems to me that if we want to introduce immutable types, we might want to do that in the context of a completely
new API. The use of the Java 8 Streams API is also something that has been mooted as something that might merit a new
Jena API. (Instead of mixing things up in the current one.)
I'm not sure how that plays out with ARQ, though. We would want people to be able to use the new types with ARQ without
much difficulty.
ajs6f
Claude Warren wrote on 11/14/17 2:43 AM:
> In most cases I prefer immutable interfaces. However, immutable interfaces
> pose an interesting problem for contract testing and for the permissions
> implementation.
>
> In contract testing you get have a producer to create instances of an
> interface and tests you run against it.
>
> However, since you don't have any setters to call on the instance you can
> not know what the result of any particular getter should be.
>
> The only choices that I see for this case are:
>
> 1. Don't test the immutable interface separately and therefore miss some
> implementations. That is only test the immutable interface when paired
> with the mutable interface.
> 2. Modify the producer interface so that the producer will create the
> data necessary to execute the tests. This results in complicated producers.
>
> Keeping in mind that contract tests allow us to write tests for the Graph
> interface and then create very simple implementations of suites for each
> implementation on Graph. This means that when we add ad method or detect
> an incorrectly implemented method we can modify the contract test and all
> implementations are then properly tested.
>
> In the permissions implementation we wrap the interfaces with dynamic
> proxies that intercept calls to verify if the user has permission to make
> those calls before execution, wrap the results with "secured" versions, and
> in some case filter results (e.g. iterators). This system will be
> perfectly happy running against immutable interfaces. The interesting part
> is that the system can take mutable objects and return objects that throw
> exceptions when the user does not have access (much the same as the current
> read only implementations do). But you can not know *a priori* which
> methods will throw exceptions.
>
> This leads me to one more observation. When building the permissions layer
> I learned that simple objects like RDFNode can return complex objects like
> Model. I believe that an immutable model would have to return an
> immutable RDFNode. The signature of the Immutable RDFNode should indicate a
> return of an Immutable Model. But to be a drop in replacement for a
> standard RDFNode it will need to return a Model. Classes like RDF lists
> also pose interesting problems.
>
> So while I like immutable interfaces in general, I think that back fitting
> them here is problematic. Scan through the permissions layer for some idea
> of the complexity.
>
> Having written all of this I think I have come to believe that low level
> tools like Jena or data stores in general, should not have immutable
> interfaces. Immutable interfaces belong at a slightly higher architectural
> level or at the extreme boundary of the project. For example if Jena had a
> webservice API that retrieved Models and such then it might makes sense for
> the deserialized versions to be immutable.
>
> Claude
>
>
> On Tue, Nov 14, 2017 at 12:12 AM, Adam Jacobs <ja...@hotmail.com>
> wrote:
>
>> The subject of immutability was raised in JENA-1391 (
>> https://issues.apache.org/jira/browse/JENA-1391).
>>
>> Specifically, the `getUnionModel` method in Jena 3.4 returns an immutable
>> model view, and the implementation of the aforementioned story includes
>> methods that will return an immutable dataset view.
>>
>> The question is whether these immutable views deserve their own
>> interfaces. Currently, the views are returned using what I called
>> "unexpected immutability" because they implement mutable interfaces. This
>> introduces the potential for `UnsupportedOperationException`s.
>>
>> Unfortunately, that (degenerate) pattern is used in Java's `Collections`
>> utility as well (https://docs.oracle.com/javase/8/docs/api/java/util/
>> Collections.html) but Scala is a clean example to draw inspiration from:
>> by implementing immutable interfaces as parents to their mutable
>> counterparts (rather than vice verse) we can satisfy the Liskov
>> Substitution Principle.
>>
>> Obviously, implementing that solution is easier to do from scratch than in
>> an existing code base; but I imagine it could be done in multiple phases,
>> by introducing the new interfaces and using them in new methods (with easy
>> conversion to mutability via union) while gradually retrofitting older
>> methods.
>>
>> The question then, is whether such a change is worthwhile...
>
>
>
>
Re: Immutability
Posted by Claude Warren <cl...@xenei.com>.
In most cases I prefer immutable interfaces. However, immutable interfaces
pose an interesting problem for contract testing and for the permissions
implementation.
In contract testing you get have a producer to create instances of an
interface and tests you run against it.
However, since you don't have any setters to call on the instance you can
not know what the result of any particular getter should be.
The only choices that I see for this case are:
1. Don't test the immutable interface separately and therefore miss some
implementations. That is only test the immutable interface when paired
with the mutable interface.
2. Modify the producer interface so that the producer will create the
data necessary to execute the tests. This results in complicated producers.
Keeping in mind that contract tests allow us to write tests for the Graph
interface and then create very simple implementations of suites for each
implementation on Graph. This means that when we add ad method or detect
an incorrectly implemented method we can modify the contract test and all
implementations are then properly tested.
In the permissions implementation we wrap the interfaces with dynamic
proxies that intercept calls to verify if the user has permission to make
those calls before execution, wrap the results with "secured" versions, and
in some case filter results (e.g. iterators). This system will be
perfectly happy running against immutable interfaces. The interesting part
is that the system can take mutable objects and return objects that throw
exceptions when the user does not have access (much the same as the current
read only implementations do). But you can not know *a priori* which
methods will throw exceptions.
This leads me to one more observation. When building the permissions layer
I learned that simple objects like RDFNode can return complex objects like
Model. I believe that an immutable model would have to return an
immutable RDFNode. The signature of the Immutable RDFNode should indicate a
return of an Immutable Model. But to be a drop in replacement for a
standard RDFNode it will need to return a Model. Classes like RDF lists
also pose interesting problems.
So while I like immutable interfaces in general, I think that back fitting
them here is problematic. Scan through the permissions layer for some idea
of the complexity.
Having written all of this I think I have come to believe that low level
tools like Jena or data stores in general, should not have immutable
interfaces. Immutable interfaces belong at a slightly higher architectural
level or at the extreme boundary of the project. For example if Jena had a
webservice API that retrieved Models and such then it might makes sense for
the deserialized versions to be immutable.
Claude
On Tue, Nov 14, 2017 at 12:12 AM, Adam Jacobs <ja...@hotmail.com>
wrote:
> The subject of immutability was raised in JENA-1391 (
> https://issues.apache.org/jira/browse/JENA-1391).
>
> Specifically, the `getUnionModel` method in Jena 3.4 returns an immutable
> model view, and the implementation of the aforementioned story includes
> methods that will return an immutable dataset view.
>
> The question is whether these immutable views deserve their own
> interfaces. Currently, the views are returned using what I called
> "unexpected immutability" because they implement mutable interfaces. This
> introduces the potential for `UnsupportedOperationException`s.
>
> Unfortunately, that (degenerate) pattern is used in Java's `Collections`
> utility as well (https://docs.oracle.com/javase/8/docs/api/java/util/
> Collections.html) but Scala is a clean example to draw inspiration from:
> by implementing immutable interfaces as parents to their mutable
> counterparts (rather than vice verse) we can satisfy the Liskov
> Substitution Principle.
>
> Obviously, implementing that solution is easier to do from scratch than in
> an existing code base; but I imagine it could be done in multiple phases,
> by introducing the new interfaces and using them in new methods (with easy
> conversion to mutability via union) while gradually retrofitting older
> methods.
>
> The question then, is whether such a change is worthwhile...
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Immutability
Posted by Claude Warren <cl...@xenei.com>.
One more note:
I have an project where we are using in memory Models as the storage for
object data (i.e. all/most instance variables are stored in the model). In
some cases we want to lock the model so that no further changes can occur
(specifically when we pass the model through a listener interface) To do
this locking we have created a permissions implementation that prohibits
all changes to the model and pass that down.
In doing so we found several interesting corner cases where unanticipated
modification was occurring.
Claude
On Tue, Nov 14, 2017 at 12:12 AM, Adam Jacobs <ja...@hotmail.com>
wrote:
> The subject of immutability was raised in JENA-1391 (
> https://issues.apache.org/jira/browse/JENA-1391).
>
> Specifically, the `getUnionModel` method in Jena 3.4 returns an immutable
> model view, and the implementation of the aforementioned story includes
> methods that will return an immutable dataset view.
>
> The question is whether these immutable views deserve their own
> interfaces. Currently, the views are returned using what I called
> "unexpected immutability" because they implement mutable interfaces. This
> introduces the potential for `UnsupportedOperationException`s.
>
> Unfortunately, that (degenerate) pattern is used in Java's `Collections`
> utility as well (https://docs.oracle.com/javase/8/docs/api/java/util/
> Collections.html) but Scala is a clean example to draw inspiration from:
> by implementing immutable interfaces as parents to their mutable
> counterparts (rather than vice verse) we can satisfy the Liskov
> Substitution Principle.
>
> Obviously, implementing that solution is easier to do from scratch than in
> an existing code base; but I imagine it could be done in multiple phases,
> by introducing the new interfaces and using them in new methods (with easy
> conversion to mutability via union) while gradually retrofitting older
> methods.
>
> The question then, is whether such a change is worthwhile...
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren