You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Andrew Whitaker <an...@braintreepayments.com> on 2016/03/03 21:06:43 UTC
Compilation error with Scala case class with private constructor
Hi,
I've run up against a compilation error involving a case class with a
private constructor:
[error]
/Users/anwhitaker/code/flink-fold-issue/src/main/scala/TestApp.scala:18:
could not find implicit value for evidence parameter of type
org.apache.flink.api.common.typeinfo.TypeInformation[List[scala.util.Try[TestApp.Wrapper]]]
[error] .fold(List[Try[Wrapper]](), new FoldFunction[Tuple2[Int,
Int], List[Try[Wrapper]]] {
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
If I make the constructor public again, the error goes away. I've set up a
simple example that demonstrates the problem here:
https://github.com/AndrewWhitaker/flink-case-class-private-ctor
I've read this article on Flink's website:
https://flink.apache.org/faq.html#why-am-i-getting-a-nonserializableexception-
but I think my issue is slightly different.
I'm just trying to understand this behavior and if there's a way I can work
around it.
Thanks!
--
Andrew Whitaker | andrew.whitaker@braintreepayments.com
Re: Compilation error with Scala case class with private constructor
Posted by Aljoscha Krettek <al...@apache.org>.
Yes Stephan, you’re spot on. We generate code for serialization/deserialization of the types and in there it creates code like this:
new YourCaseClass(field1, field2, …)
to create the value that is given to the user function.
If the constructor is private compilation of that generated code will fail and we don’t get the TypeInformation.
I’m afraid there is no workaround right now, except for not making the constructor private.
> On 03 Mar 2016, at 21:17, Andrew Whitaker <an...@braintreepayments.com> wrote:
>
> Thanks for the quick response, Stephen, that makes sense.
>
> > Is it a problem for your use case to make the constructor public?
>
> It's not a huge problem, I was mostly just curious as to what the problem was exactly.
>
> On Thu, Mar 3, 2016 at 2:14 PM, Stephan Ewen <se...@apache.org> wrote:
> Hi!
>
> My guess is that this error is indirectly reporting that no TypeInformation could be generated for the case class. That TypeInformation is generated using macros during program compilation.
>
> The generated TypeInformation will contain a partially code-generated serializer including the code to instantiate the case class upon deserialization. That generated code at some point calls the constructor and thus probably fails to compile when the constructor is private. As a result, no TypeInformation will be created, and then Scala cannot provide one for the implicit parameter.
>
> I think Aljoscha can probably give a deeper insight into the TypeInformation generator for Scala.
>
> Is it a problem for your use case to make the constructor public?
>
> Greetings,
> Stephan
>
>
> On Thu, Mar 3, 2016 at 9:06 PM, Andrew Whitaker <an...@braintreepayments.com> wrote:
> Hi,
>
> I've run up against a compilation error involving a case class with a private constructor:
>
> [error] /Users/anwhitaker/code/flink-fold-issue/src/main/scala/TestApp.scala:18: could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[List[scala.util.Try[TestApp.Wrapper]]]
> [error] .fold(List[Try[Wrapper]](), new FoldFunction[Tuple2[Int, Int], List[Try[Wrapper]]] {
> [error] ^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> If I make the constructor public again, the error goes away. I've set up a simple example that demonstrates the problem here: https://github.com/AndrewWhitaker/flink-case-class-private-ctor
> I've read this article on Flink's website: https://flink.apache.org/faq.html#why-am-i-getting-a-nonserializableexception- but I think my issue is slightly different.
>
> I'm just trying to understand this behavior and if there's a way I can work around it.
>
> Thanks!
>
> --
> Andrew Whitaker | andrew.whitaker@braintreepayments.com
>
>
>
>
> --
> Andrew Whitaker | andrew.whitaker@braintreepayments.com
> --
> Note: this information is confidential. It is prohibited to share, post online or otherwise publicize without Braintree's prior written consent.
Re: Compilation error with Scala case class with private constructor
Posted by Andrew Whitaker <an...@braintreepayments.com>.
Thanks for the quick response, Stephen, that makes sense.
> Is it a problem for your use case to make the constructor public?
It's not a huge problem, I was mostly just curious as to what the problem
was exactly.
On Thu, Mar 3, 2016 at 2:14 PM, Stephan Ewen <se...@apache.org> wrote:
> Hi!
>
> My guess is that this error is indirectly reporting that no
> TypeInformation could be generated for the case class. That TypeInformation
> is generated using macros during program compilation.
>
> The generated TypeInformation will contain a partially code-generated
> serializer including the code to instantiate the case class upon
> deserialization. That generated code at some point calls the constructor
> and thus probably fails to compile when the constructor is private. As a
> result, no TypeInformation will be created, and then Scala cannot provide
> one for the implicit parameter.
>
> I think Aljoscha can probably give a deeper insight into the
> TypeInformation generator for Scala.
>
> Is it a problem for your use case to make the constructor public?
>
> Greetings,
> Stephan
>
>
> On Thu, Mar 3, 2016 at 9:06 PM, Andrew Whitaker <
> andrew.whitaker@braintreepayments.com> wrote:
>
>> Hi,
>>
>> I've run up against a compilation error involving a case class with a
>> private constructor:
>>
>> [error]
>> /Users/anwhitaker/code/flink-fold-issue/src/main/scala/TestApp.scala:18:
>> could not find implicit value for evidence parameter of type
>> org.apache.flink.api.common.typeinfo.TypeInformation[List[scala.util.Try[TestApp.Wrapper]]]
>> [error] .fold(List[Try[Wrapper]](), new FoldFunction[Tuple2[Int,
>> Int], List[Try[Wrapper]]] {
>> [error] ^
>> [error] one error found
>> [error] (compile:compileIncremental) Compilation failed
>>
>> If I make the constructor public again, the error goes away. I've set up
>> a simple example that demonstrates the problem here:
>> https://github.com/AndrewWhitaker/flink-case-class-private-ctor
>> I've read this article on Flink's website:
>> https://flink.apache.org/faq.html#why-am-i-getting-a-nonserializableexception-
>> but I think my issue is slightly different.
>>
>> I'm just trying to understand this behavior and if there's a way I can
>> work around it.
>>
>> Thanks!
>>
>> --
>> Andrew Whitaker | andrew.whitaker@braintreepayments.com
>>
>
>
--
Andrew Whitaker | andrew.whitaker@braintreepayments.com
--
Note: this information is confidential. It is prohibited to share, post
online or otherwise publicize without Braintree's prior written consent.
Re: Compilation error with Scala case class with private constructor
Posted by Stephan Ewen <se...@apache.org>.
Hi!
My guess is that this error is indirectly reporting that no TypeInformation
could be generated for the case class. That TypeInformation is generated
using macros during program compilation.
The generated TypeInformation will contain a partially code-generated
serializer including the code to instantiate the case class upon
deserialization. That generated code at some point calls the constructor
and thus probably fails to compile when the constructor is private. As a
result, no TypeInformation will be created, and then Scala cannot provide
one for the implicit parameter.
I think Aljoscha can probably give a deeper insight into the
TypeInformation generator for Scala.
Is it a problem for your use case to make the constructor public?
Greetings,
Stephan
On Thu, Mar 3, 2016 at 9:06 PM, Andrew Whitaker <
andrew.whitaker@braintreepayments.com> wrote:
> Hi,
>
> I've run up against a compilation error involving a case class with a
> private constructor:
>
> [error]
> /Users/anwhitaker/code/flink-fold-issue/src/main/scala/TestApp.scala:18:
> could not find implicit value for evidence parameter of type
> org.apache.flink.api.common.typeinfo.TypeInformation[List[scala.util.Try[TestApp.Wrapper]]]
> [error] .fold(List[Try[Wrapper]](), new FoldFunction[Tuple2[Int,
> Int], List[Try[Wrapper]]] {
> [error] ^
> [error] one error found
> [error] (compile:compileIncremental) Compilation failed
>
> If I make the constructor public again, the error goes away. I've set up a
> simple example that demonstrates the problem here:
> https://github.com/AndrewWhitaker/flink-case-class-private-ctor
> I've read this article on Flink's website:
> https://flink.apache.org/faq.html#why-am-i-getting-a-nonserializableexception-
> but I think my issue is slightly different.
>
> I'm just trying to understand this behavior and if there's a way I can
> work around it.
>
> Thanks!
>
> --
> Andrew Whitaker | andrew.whitaker@braintreepayments.com
>