You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Bernd Wiswedel <be...@knime.com> on 2020/08/24 21:52:52 UTC
[Java] Issues with "Split Packages" and OSGi (and probably Java 9+)
We at KNIME (https://www.knime.com/) are currently looking at Arrow as an
alternative data representation. Our platform is based on Eclipse and uses
OSGi for dependency management. As part of that we use the Arrow artifacts
available from maven and convert them to OSGI bundles.
We noticed that at least one java package (org.apache.arrow.util) is
provided in two different artifacts
- arrow-memory-core: (Link to GitHub
<https://github.com/apache/arrow/tree/master/java/memory/memory-core/src/main/java/org/apache/arrow/util>
)
- arrow-vectors: (Link to GitHub
<https://github.com/apache/arrow/tree/master/java/vector/src/main/java/org/apache/arrow/util>
)
This is causing us trouble as the package is available via two chains. I
believe this is also going to be troublesome once it's used in a Java 9+
environment where different modules(*) can't share the same package ('split
package' -- some more details are here
<https://www.logicbig.com/tutorials/core-java-tutorial/modules/split-packages.html>
).
The simplest solution is to move classes from one package to the other
(same) package. If that's not an accepted option it could also be moved
into a new artifact(*) arrow-util or so.
Please advise!
Thanks,
Bernd
(*) Bundle, Module, Plug-In, Artefact ... different terms used in different
contexts but they mostly mean the same thing.
Re: [Java] Issues with "Split Packages" and OSGi (and probably Java 9+)
Posted by Micah Kornfield <em...@gmail.com>.
Arrow is still in a mode that most releases will be a major release
according to SemVer [1], so the breakage is acceptable from a project
standpoint if it is better for long term java compatibility.
That being said, taking a quick look at these classes, consolidating them
into memory-core might be OK since the vector ones don't seem to have
dependencies on other the only one that is questionable to me is the the
rounding helper class since that seems like it has tighter coupling to
Vector.
[1] https://arrow.apache.org/docs/format/Versioning.html
On Tue, Aug 25, 2020 at 12:51 AM Bernd Wiswedel <be...@knime.com>
wrote:
> Jokes aside and on second thought: Package renaming is considered an API
> break and would usually require a major release according to semantic
> versioning.
>
> I believe any change to fix this issue is going to cause an API break
> (including moving classes within the same package to a different artifact).
> As it stands right now I don't think the artifacts can be used in a Java 9+
> module system (anyone there to prove me wrong?), and it's going to be
> painful in an OSGi environment. (In our project we currently really only
> care about the latter right now.)
>
> On the plus side the API change is easy to understand and fix.
>
> Thanks,
> Bernd
>
> On Tue, Aug 25, 2020 at 12:07 AM Bernd Wiswedel <be...@knime.com>
> wrote:
>
>> Nah, they had plenty of time to object. Will look into it in the coming
>> days.
>>
>> Thanks,
>> Bernd
>>
>> Jacques Nadeau <ja...@apache.org> schrieb am Di., 25. Aug. 2020, 00:03:
>>
>>> I suggest renaming the vector related util package to
>>> org.apache.arrow.vector.util and memory to org.apache.arrow.memory.util. It
>>> would be great if you could open a ticket and propose a patch (assuming
>>> others agree with the solution).
>>>
>>> On Mon, Aug 24, 2020 at 2:53 PM Bernd Wiswedel <be...@knime.com>
>>> wrote:
>>>
>>>>
>>>> We at KNIME (https://www.knime.com/) are currently looking at Arrow as
>>>> an alternative data representation. Our platform is based on Eclipse and
>>>> uses OSGi for dependency management. As part of that we use the Arrow
>>>> artifacts available from maven and convert them to OSGI bundles.
>>>>
>>>> We noticed that at least one java package (org.apache.arrow.util) is
>>>> provided in two different artifacts
>>>>
>>>> - arrow-memory-core: (Link to GitHub
>>>> <https://github.com/apache/arrow/tree/master/java/memory/memory-core/src/main/java/org/apache/arrow/util>
>>>> )
>>>> - arrow-vectors: (Link to GitHub
>>>> <https://github.com/apache/arrow/tree/master/java/vector/src/main/java/org/apache/arrow/util>
>>>> )
>>>>
>>>> This is causing us trouble as the package is available via two chains.
>>>> I believe this is also going to be troublesome once it's used in a Java 9+
>>>> environment where different modules(*) can't share the same package ('split
>>>> package' -- some more details are here
>>>> <https://www.logicbig.com/tutorials/core-java-tutorial/modules/split-packages.html>
>>>> ).
>>>>
>>>> The simplest solution is to move classes from one package to the other
>>>> (same) package. If that's not an accepted option it could also be moved
>>>> into a new artifact(*) arrow-util or so.
>>>>
>>>> Please advise!
>>>>
>>>> Thanks,
>>>> Bernd
>>>>
>>>> (*) Bundle, Module, Plug-In, Artefact ... different terms used in
>>>> different contexts but they mostly mean the same thing.
>>>>
>>>
Re: [Java] Issues with "Split Packages" and OSGi (and probably Java 9+)
Posted by Bernd Wiswedel <be...@knime.com>.
Jokes aside and on second thought: Package renaming is considered an API
break and would usually require a major release according to semantic
versioning.
I believe any change to fix this issue is going to cause an API break
(including moving classes within the same package to a different artifact).
As it stands right now I don't think the artifacts can be used in a Java 9+
module system (anyone there to prove me wrong?), and it's going to be
painful in an OSGi environment. (In our project we currently really only
care about the latter right now.)
On the plus side the API change is easy to understand and fix.
Thanks,
Bernd
On Tue, Aug 25, 2020 at 12:07 AM Bernd Wiswedel <be...@knime.com>
wrote:
> Nah, they had plenty of time to object. Will look into it in the coming
> days.
>
> Thanks,
> Bernd
>
> Jacques Nadeau <ja...@apache.org> schrieb am Di., 25. Aug. 2020, 00:03:
>
>> I suggest renaming the vector related util package to
>> org.apache.arrow.vector.util and memory to org.apache.arrow.memory.util. It
>> would be great if you could open a ticket and propose a patch (assuming
>> others agree with the solution).
>>
>> On Mon, Aug 24, 2020 at 2:53 PM Bernd Wiswedel <be...@knime.com>
>> wrote:
>>
>>>
>>> We at KNIME (https://www.knime.com/) are currently looking at Arrow as
>>> an alternative data representation. Our platform is based on Eclipse and
>>> uses OSGi for dependency management. As part of that we use the Arrow
>>> artifacts available from maven and convert them to OSGI bundles.
>>>
>>> We noticed that at least one java package (org.apache.arrow.util) is
>>> provided in two different artifacts
>>>
>>> - arrow-memory-core: (Link to GitHub
>>> <https://github.com/apache/arrow/tree/master/java/memory/memory-core/src/main/java/org/apache/arrow/util>
>>> )
>>> - arrow-vectors: (Link to GitHub
>>> <https://github.com/apache/arrow/tree/master/java/vector/src/main/java/org/apache/arrow/util>
>>> )
>>>
>>> This is causing us trouble as the package is available via two chains. I
>>> believe this is also going to be troublesome once it's used in a Java 9+
>>> environment where different modules(*) can't share the same package ('split
>>> package' -- some more details are here
>>> <https://www.logicbig.com/tutorials/core-java-tutorial/modules/split-packages.html>
>>> ).
>>>
>>> The simplest solution is to move classes from one package to the other
>>> (same) package. If that's not an accepted option it could also be moved
>>> into a new artifact(*) arrow-util or so.
>>>
>>> Please advise!
>>>
>>> Thanks,
>>> Bernd
>>>
>>> (*) Bundle, Module, Plug-In, Artefact ... different terms used in
>>> different contexts but they mostly mean the same thing.
>>>
>>
Re: [Java] Issues with "Split Packages" and OSGi (and probably Java 9+)
Posted by Bernd Wiswedel <be...@knime.com>.
Nah, they had plenty of time to object. Will look into it in the coming
days.
Thanks,
Bernd
Jacques Nadeau <ja...@apache.org> schrieb am Di., 25. Aug. 2020, 00:03:
> I suggest renaming the vector related util package to
> org.apache.arrow.vector.util and memory to org.apache.arrow.memory.util. It
> would be great if you could open a ticket and propose a patch (assuming
> others agree with the solution).
>
> On Mon, Aug 24, 2020 at 2:53 PM Bernd Wiswedel <be...@knime.com>
> wrote:
>
>>
>> We at KNIME (https://www.knime.com/) are currently looking at Arrow as
>> an alternative data representation. Our platform is based on Eclipse and
>> uses OSGi for dependency management. As part of that we use the Arrow
>> artifacts available from maven and convert them to OSGI bundles.
>>
>> We noticed that at least one java package (org.apache.arrow.util) is
>> provided in two different artifacts
>>
>> - arrow-memory-core: (Link to GitHub
>> <https://github.com/apache/arrow/tree/master/java/memory/memory-core/src/main/java/org/apache/arrow/util>
>> )
>> - arrow-vectors: (Link to GitHub
>> <https://github.com/apache/arrow/tree/master/java/vector/src/main/java/org/apache/arrow/util>
>> )
>>
>> This is causing us trouble as the package is available via two chains. I
>> believe this is also going to be troublesome once it's used in a Java 9+
>> environment where different modules(*) can't share the same package ('split
>> package' -- some more details are here
>> <https://www.logicbig.com/tutorials/core-java-tutorial/modules/split-packages.html>
>> ).
>>
>> The simplest solution is to move classes from one package to the other
>> (same) package. If that's not an accepted option it could also be moved
>> into a new artifact(*) arrow-util or so.
>>
>> Please advise!
>>
>> Thanks,
>> Bernd
>>
>> (*) Bundle, Module, Plug-In, Artefact ... different terms used in
>> different contexts but they mostly mean the same thing.
>>
>
Re: [Java] Issues with "Split Packages" and OSGi (and probably Java 9+)
Posted by Jacques Nadeau <ja...@apache.org>.
I suggest renaming the vector related util package to
org.apache.arrow.vector.util and memory to org.apache.arrow.memory.util. It
would be great if you could open a ticket and propose a patch (assuming
others agree with the solution).
On Mon, Aug 24, 2020 at 2:53 PM Bernd Wiswedel <be...@knime.com>
wrote:
>
> We at KNIME (https://www.knime.com/) are currently looking at Arrow as an
> alternative data representation. Our platform is based on Eclipse and uses
> OSGi for dependency management. As part of that we use the Arrow artifacts
> available from maven and convert them to OSGI bundles.
>
> We noticed that at least one java package (org.apache.arrow.util) is
> provided in two different artifacts
>
> - arrow-memory-core: (Link to GitHub
> <https://github.com/apache/arrow/tree/master/java/memory/memory-core/src/main/java/org/apache/arrow/util>
> )
> - arrow-vectors: (Link to GitHub
> <https://github.com/apache/arrow/tree/master/java/vector/src/main/java/org/apache/arrow/util>
> )
>
> This is causing us trouble as the package is available via two chains. I
> believe this is also going to be troublesome once it's used in a Java 9+
> environment where different modules(*) can't share the same package ('split
> package' -- some more details are here
> <https://www.logicbig.com/tutorials/core-java-tutorial/modules/split-packages.html>
> ).
>
> The simplest solution is to move classes from one package to the other
> (same) package. If that's not an accepted option it could also be moved
> into a new artifact(*) arrow-util or so.
>
> Please advise!
>
> Thanks,
> Bernd
>
> (*) Bundle, Module, Plug-In, Artefact ... different terms used in
> different contexts but they mostly mean the same thing.
>