Generics question

[Saravanan Palanichamy <ch...@gmail.com>]

Hello folks

I am building an AST transformation that will allow me to define a concept
of mutable and immutable objects. Java lets me hold mutable objects in
immutable object lists (as per the code below). But my DSL also wants to
allow immutable objects to be held in mutable lists. The meaning it is
conveying is that adding to mutable lists will clone the immutable object
and not affect the original ( I use the << operator to denote this. It
denotes that the object is being streamed into the list and will not affect
the object if list contents are modified) . What is the best way to achieve
this? I dont see a type checking extension that will catch this missing
"method" and let me override it.

I dont want to create an extension function, because this could be a model
I want to follow for several classes and not just Blah. Which is why I want
to use the AST route to solve this problem. Any pointers will be very
helpful.

Assume that I can convert immutable to mutable behind the scenes. Also
assume that 100s of classes will follow this model of mutability

static class Blah {
> static class Immutable {
> String getProperty1() { "PROP1" }
> }
> static class Mutable extends Immutable {
> void setProperty1(String value) {
> }
> }
> }
> void fn() {
> // This works
> List<Blah.Immutable> myOtherThings = []
> myOtherThings << new Blah.Immutable()
> myOtherThings << new Blah.Mutable()
>
> // This does not (I want an AST that will make this work)
> List<Blah.Mutable> myThings = []
> myThings << new Blah.Immutable()
>
> // This works
> myThings << new Blah.Mutable()
> }
>

regards
Saravanan

Re: Generics question

[Jochen Theodorou <bl...@gmx.org>]

On 28.08.23 02:25, Saravanan Palanichamy wrote:
[...]
I think I understand a bit better now...

[...]
> The problem I run into however is when I use lists of mutable with
> immutables. For example, This entire code should be valid
> void fn(@ReadOnly Person me) {
>   List<Person> myList = []
>   myList << me
>   Map<String, Person> myMap = [:]
>   myMap << me
> }
>
> This gets converted into
> void fn(Person.Immutable me) {
>   List<Person.Mutable> myList = []
>   myList << me // Fails with me cannot be assigned to mutable list
>   Map<String, Person.Mutable> myMap = [:]
>   myMap["ME"] = me // Also fails because types dont match
> }
>
> How do I solve this? Are there other issues I am not foreseeing?

I think the substitution for List<Person> should then be
List<Person.Mutable | Person Immutable>. While this cannot be expressed
in source we have UnionTypeClassNode for this in the AST. Though that
node is mainly for a different context and could cause compiler problems.

bye Jochen

Re: Generics question

[Saravanan Palanichamy <ch...@gmail.com>]

Hello Jochen

Maybe its good for me to state the original problem separately from the
solution I am pursuing to see if there are other ways to do it. Apologies
if the email is too long. I tried to stick to a lot of example rather than
text

The problem. Entities in the DSL are either mutable or immutable. Meaning I
can either just read from them or update their internals. These include
lists, maps etc. For example,

    class Person {
        String name
        Person spouse
        List<String> address
        List<Person> relations
    }

1. I must not modify read only objects
        @ReadOnly Person me = new Person().tap { ... }
        me.name = "MyName" // Fails with name is readonly
        me.address << "Other details" // Fails with list is readonly
        me.relations << new Person() // Fails with list is readonly
        me.relations[0].name = "NEWRELATION" // Fails with name is readonly

2. To modify an object it must be mutable
        Person me = new Person()
        me.name = "MYNAME"
        me.address << "ADDR"

3. A mutable object accessed as read only must not be able to modify content
        @ReadOnly Person safeMe = me
        safeMe.name = "MYNAME" // Fails with name is read only
        me.name = "ICANCHANGETHIS"

4. An immutable list should not allow list and content modification
        @ReadOnly List<Person> persons = [ me, you, him, her]
        persons << everyone // Fails with list is readonly
        persons[0].name = "MyName" // Fails with name is readonly
        def me = persons[0]; me.name = "MyName" // Fails with name is
readonly

5. A mutable list can allow modification of list or content, A mutable list
can accept immutable content but it clones it to make it mutable
        // All of this should work
        List<Person> mutableListOfPersons = [me, you]
        mutableListOfPersons << everyone
        mutableListOfPersons[0].name = "MyName"
        // Watch immutable me being changed because it was cloned into the
list. The original immutableme is still unchanged
        mutableListOfPersons << myImmutableMe
        mutableListOfPersons[1].name = "ChangingTheImmutable"

I will never have a use case where I will need List<@ReadOnly ...> because
either the entire list including its contents is read only or it is
mutable. This explained, I am trying to do this

1. I run a second compile on the code just to check for mutability. These
class files are discarded (so in this compile I am only interested in
semantic validation and not runtime safety. All temporary immutable and
mutable classes generated here are never used in the runtime)
2. During compile, I create Person.Mutable and Person.Immutable. Mutable
derives from immutable.
3. Immutable has only get functions and Mutable has set/get functions (the
overriden get further qualifies the return type to be mutable)
4. I then replace code that uses Person. Everywhere a local variable is
declared, I make it mutable. Everywhere I call a function, I make its
parameters read onlyI then let the Groovy compiler do its magic. This works
perfectly for persons not used inside generic collections

So after compile, my entity looks like this

    class Person {

        interface class Immutable {
            String getName()
            Person.Immutable getSpouse()
            List<String> getAddress()
            List<Person.Immutable> getRelations()
        }

        interface Mutable extends Immutable {

        // All gets with mutable signatures
        String getName()
        Person.Mutable getSpouse()
        List<String> getAddress()
        List<Person.Mutable> getRelations()

        // All sets with immutable signatures to allow both mutable and
immutable
        void setName(String value)
        void setSpouse(Person.Immutable value)
        void setAddress(List<String value)
        void setRelations(List<? extends Person.Immutable> value)
        }
    }

My code then looks like this and works as expected without any major
incidents (newing up the interface is handled through another AST that
replaces constructor call, so ignore that)

1. I must not modify read only objects
        @ReadOnly Person.Immutable me = new Person.Mutable().tap { ... }
        me.name = "MyName" // Fails with name is readonly
        me.address << "Other details" // Fails with list is readonly
        me.relations << new Person() // Fails with list is readonly
        me.relations[0].name = "NEWRELATION" // Fails with name is readonly

2. To modify an object it must be mutable
        Person.Mutable me = new Person.Mutable()
        me.name = "MYNAME"
        me.address << "ADDR"

3. A mutable object accessed as read only must not be able to modify content
        @ReadOnly Person.Immutable safeMe = me // me is mutable
        safeMe.name = "MYNAME" // Fails with name is read only
        me.name = "ICANCHANGETHIS"

4. An immutable list should not allow list and content modification
        @ReadOnly List<Person> persons = [ me, you, him, her]
        persons[0].name = "MyName" // Fails with name is readonly
        def me = persons[0]; me.name = "MyName" // Fails with name is
readonly

5. A mutable list can allow modification of list or content
// All of this should work
        List<Person> mutableListOfPersons = [me, you]
        mutableListOfPersons[0].name = "MyName"

I feel like I can handle the list mutability by further qualifying lists as
MutableList vs ImmutableList and checking after static compile what method
was called on what object (and I can compile error on putAt being used in
ImmutableList etc ...)

The problem I run into however is when I use lists of mutable with
immutables. For example, This entire code should be valid
        void fn(@ReadOnly Person me) {
List<Person> myList = []
myList << me
Map<String, Person> myMap = [:]
myMap << me
}

This gets converted into
void fn(Person.Immutable me) {
List<Person.Mutable> myList = []
myList << me // Fails with me cannot be assigned to mutable list
Map<String, Person.Mutable> myMap = [:]
myMap["ME"] = me // Also fails because types dont match
}

How do I solve this? Are there other issues I am not foreseeing?

regards
Saravanan

Re: Generics question

[Jochen Theodorou <bl...@gmx.org>]

On 27.08.23 16:36, Saravanan Palanichamy wrote:
> Jochen,
>
> Thank you for your quick response.
>
> I did consider this, but I want to be able to use a mutable where-ever
> immutable is allowed. For example, if a function can take all immutable
> parameters, I want to be able to call it with mutable types as well. So
> my choices are either bypass typing when I try to use mutable where
> immutables are needed (in addition to dealing with the generics issue I
> called out) or just deal with the generics issue I am looking to solve.
> Either way, I'll have to do something (or am I thinking too much :D)
>
> I specifically want this to work, because my DSL is trying to implement
> ReadOnly constructs so Mutable/Immutable are not really types seen by
> the programmer, its just used during the type checking mechanism to make
> sure something marked @ReadOnly is not modified

for me they are kind of exclusive, but ok.

> So the programmer will type
>
> void fn1(@ReadOnly Entity myInputParameter) {
>      List<Entity> things = []
>      things << myInputParameter
>
>      // This should cause a compile error
>      myInputParameter.myProperty = "1234
> }

that should be possible without much problem. A bit more tricky is

def x = myInputParameter
x.myProperty = "1234"

but still possible I would say. Where it gets really problematic is then

things[0].myProperty = "1234"

> and I want to see
>
> void fn1(Entity.Immutable myInputParameter) {
>      // I want this to work but it wont because of groovy type checking
>      List<Entity.Mutable> things = []
>      things << myInputParameter
>
>      // This will automatically fail (like I want it to) because
> setMyProperty is not a method in immutable and is only available in mutable
>      myInputParameter.myProperty = "1234"
> }

but what, except that you replaced the annotations with two types is now
really different?

But why not extend the annotation idea?

void fn1(@ReadOnly Entity myInputParameter) {
      List<@ReadOnly Entity> things = []
      // only allow if T(myInputParameter) is
      // @ReadOnly Entity
      things << myInputParameter

      // fails because of @ReadOnly
      myInputParameter.myProperty = "1234
      // fails because of propagated @ReadOnly
      things[0].myProperty = "1234"
      // could be allowed or forbidden based
      // on @Readonly
      Entity x = things[0]
      things[0].myProperty = "1234"
}


Frankly I currently don't see how to solve this based on the type system
if you do not want to do it like I suggested.

In general adding restrictions or making something found that is not
found otherwise is easy with the type checking extensions. But to tell
the compiler that something is found but wrong or that something should
be propagated in a different way than usual... that is difficult with
the current infrastructure. One way to solve this would then imho be to
find a version that passes compilation and let that the typechecker
check that instead. And then add your checks for immutability.

bye Jochen

Re: Generics question

[Saravanan Palanichamy <ch...@gmail.com>]

Jochen,

Thank you for your quick response.

I did consider this, but I want to be able to use a mutable where-ever
immutable is allowed. For example, if a function can take all immutable
parameters, I want to be able to call it with mutable types as well. So my
choices are either bypass typing when I try to use mutable where immutables
are needed (in addition to dealing with the generics issue I called out) or
just deal with the generics issue I am looking to solve. Either way, I'll
have to do something (or am I thinking too much :D)

I specifically want this to work, because my DSL is trying to implement
ReadOnly constructs so Mutable/Immutable are not really types seen by the
programmer, its just used during the type checking mechanism to make sure
something marked @ReadOnly is not modified

So the programmer will type

void fn1(@ReadOnly Entity myInputParameter) {
    List<Entity> things = []
    things << myInputParameter

    // This should cause a compile error
    myInputParameter.myProperty = "1234
}

and I want to see

void fn1(Entity.Immutable myInputParameter) {
    // I want this to work but it wont because of groovy type checking
    List<Entity.Mutable> things = []
    things << myInputParameter

    // This will automatically fail (like I want it to) because
setMyProperty is not a method in immutable and is only available in mutable
    myInputParameter.myProperty = "1234"
}

// Now both these will compile fine
fn1(new Entity.Mutable())
fn1(new Entity.Immutable())

It does look like I am bypassing the type system, but only in certain cases
specifically generics

regards
Saravanan

On Sun, Aug 27, 2023 at 6:58 PM Jochen Theodorou <bl...@gmx.org> wrote:

> hi,
>
> Basically you want to use an AST transform to bypass the type system? I
> don't have enough data, but I wonder if you cannot solve the problem
> without this.
>
> I mean... why does Mutable extend Immutable? How about...
>
> interface Entity {}
> class Mutable implement Entity [}
> class Immutable implements Entity {}
>
> if you use sealed, then those two classes can be designated as the only
> implementations as well. Though I assume that is not what you intend.
>
> Then you should be able to do /I have tested nothing):
>
> List<? extends Entity> myOtherThings = []
> myOtherThings << new Immutable()
> myOtherThings << new Mutable()
>
> List<Immutale> myThings = []
> myThings << new Immutable()
> // myThings << new Mutable() forbidden
>
> List<Mutable> myMutables = []
> // myMutables << new Immutable() forbiddden
> myMutables << new Mutable()
>
> On 27.08.23 14:59, Saravanan Palanichamy wrote:
> > Hello folks
> >
> > I am building an AST transformation that will allow me to define a
> > concept of mutable and immutable objects. Java lets me hold mutable
> > objects in immutable object lists (as per the code below). But my DSL
> > also wants to allow immutable objects to be held in mutable lists. The
> > meaning it is conveying is that adding to mutable lists will clone the
> > immutable object and not affect the original ( I use the << operator to
> > denote this. It denotes that the object is being streamed into the list
> > and will not affect the object if list contents are modified) . What is
> > the best way to achieve this? I dont see a type checking extension that
> > will catch this missing "method" and let me override it.
> >
> > I dont want to create an extension function, because this could be a
> > model I want to follow for several classes and not just Blah. Which is
> > why I want to use the AST route to solve this problem. Any pointers will
> > be very helpful.
> >
> > Assume that I can convert immutable to mutable behind the scenes. Also
> > assume that 100s of classes will follow this model of mutability
> >
> >     static class Blah {
> >     static class Immutable {
> >     String getProperty1() { "PROP1" }
> >     }
> >     static class Mutable extends Immutable {
> >     void setProperty1(String value) {
> >     }
> >     }
> >     }
> >     void fn() {
> >     // This works
> >     List<Blah.Immutable> myOtherThings = []
> >     myOtherThings << new Blah.Immutable()
> >     myOtherThings << new Blah.Mutable()
> >
> >     // This does not (I want an AST that will make this work)
> >     List<Blah.Mutable> myThings = []
> >     myThings << new Blah.Immutable()
> >
> >     // This works
> >     myThings << new Blah.Mutable()
> >     }
> >
> >
> > regards
> > Saravanan
>
>

Re: Generics question

[Jochen Theodorou <bl...@gmx.org>]

hi,

Basically you want to use an AST transform to bypass the type system? I
don't have enough data, but I wonder if you cannot solve the problem
without this.

I mean... why does Mutable extend Immutable? How about...

interface Entity {}
class Mutable implement Entity [}
class Immutable implements Entity {}

if you use sealed, then those two classes can be designated as the only
implementations as well. Though I assume that is not what you intend.

Then you should be able to do /I have tested nothing):

List<? extends Entity> myOtherThings = []
myOtherThings << new Immutable()
myOtherThings << new Mutable()

List<Immutale> myThings = []
myThings << new Immutable()
// myThings << new Mutable() forbidden

List<Mutable> myMutables = []
// myMutables << new Immutable() forbiddden
myMutables << new Mutable()

On 27.08.23 14:59, Saravanan Palanichamy wrote:
> Hello folks
>
> I am building an AST transformation that will allow me to define a
> concept of mutable and immutable objects. Java lets me hold mutable
> objects in immutable object lists (as per the code below). But my DSL
> also wants to allow immutable objects to be held in mutable lists. The
> meaning it is conveying is that adding to mutable lists will clone the
> immutable object and not affect the original ( I use the << operator to
> denote this. It denotes that the object is being streamed into the list
> and will not affect the object if list contents are modified) . What is
> the best way to achieve this? I dont see a type checking extension that
> will catch this missing "method" and let me override it.
>
> I dont want to create an extension function, because this could be a
> model I want to follow for several classes and not just Blah. Which is
> why I want to use the AST route to solve this problem. Any pointers will
> be very helpful.
>
> Assume that I can convert immutable to mutable behind the scenes. Also
> assume that 100s of classes will follow this model of mutability
>
>     static class Blah {
>     static class Immutable {
>     String getProperty1() { "PROP1" }
>     }
>     static class Mutable extends Immutable {
>     void setProperty1(String value) {
>     }
>     }
>     }
>     void fn() {
>     // This works
>     List<Blah.Immutable> myOtherThings = []
>     myOtherThings << new Blah.Immutable()
>     myOtherThings << new Blah.Mutable()
>
>     // This does not (I want an AST that will make this work)
>     List<Blah.Mutable> myThings = []
>     myThings << new Blah.Immutable()
>
>     // This works
>     myThings << new Blah.Mutable()
>     }
>
>
> regards
> Saravanan