You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by sebb <se...@gmail.com> on 2022/06/29 10:59:23 UTC
[CRYPTO] Problem with JNA on macOS and Windows
It looks like macOS 10.5+ and Windows (latest) use LibreSSL by default
rather than OpenSSL.
The LibreSSL API does not have the same functions as either 1.0.2 or
1.1.1, so needs its own JNA class. In particular it looks like
ENGINE_load_rdrand is not present, nor is OpenSSL_version_num; 1.0.2
has the former only, and 1.1.1 has the latter only.
This makes it hard to support JNA with the current design.
It would require another OpenSsl<version>NativeJna class, and the
parent class OpenSslNativeJna would need to use yet another condition
in each of its methods.
This is quite tedious and error-prone.
Seems to me it would be better to use something like a set of
singleton classes that implement the required methods. The appropriate
one can then be initialised and used by OpenSslNativeJna to field the
actual methods. i.e. replace the conditional logic with a static
reference to the correct API interface instance. This should only
affect non-public classes.
Any other suggestions?
Sebb
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Bernd Eckenfels <ec...@zusammenkunft.net>.
Hello,
I don’t really understand why we “must” support libreSSL. I mean it is good if we would support multiple implementations to cater for a wider audience, but if commons-crypto requires a certain OpenSSL library and if that library is available for all platforms, why not.
Integrators (aka users) can always ship a compatible one. I think none of the libraries comes out of the box on windows (and even if it did it would not be a showstopper)? So why would libressl be prefered. Other question, why do the symbols differ wasn’t there some form of replaceability?
Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
Von: sebb <se...@gmail.com>
Gesendet: Thursday, June 30, 2022 12:14:06 AM
An: Commons Developers List <de...@commons.apache.org>
Betreff: Re: [CRYPTO] Problem with JNA on macOS and Windows
On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
>
> We cannot remove support for Windows and macOS, that's silly.
AFAICT that means we must support the different set of function names
in LibreSSL.
Note that we only currently use a small proportion of them.
> I have not followed all the branches and commits, so I'm not sure what the
> current problems are, but I know I was able to release all OSs last go
> around. I don't see why we need to rip out anything as fundamental.
Well, I have tried running the Crypto and OpenSslJna main classes on
macOS and Windows, and they both fail with the 1.1.0 release.
With current master, Crypto succeeds, but OpenSslJna fails to find
ENGINE_load_rdrand on both macOS and Windows.
(The job step succeeds, because the code catches the exception)
It looks like the test which would have exposed at least one of the
issues was never enabled because of a failures on macOS; this hid the
same problem on Windows.
I am not suggesting we rip out anything at this point.
The question is whether to do anything about the previously undetected
issues with JNA on macOS and Windows, and if so, whether this needs to
be done for the current or a later release. If we don't fix this
release, obviously it needs some mention in the release notes.
Sebb
https://github.com/apache/commons-crypto/actions/runs/2586011129
> Gary
>
> On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
>
> > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com> wrote:
> > >
> > > I agree with Gary that we just don't support LibreSSL. To my knowledge
> > > we've never advertised LibreSSL support, so I don't see it as an issue.
> >
> > In that case AFAICT we will have to drop *all* support for macOS and
> > Windows, as they both seem to default to LibreSSL.
> >
> > Note that adding support for LibreSSL was much easier for JNI, as it
> > uses far fewer methods.
> > Rather than checking the version, I changed the code to try OpenSSL
> > 1.1 names first, then a fallback.
> > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> >
> > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > macOS fails on 16345bc because it now uses LibreSSL which has a
> > different mix of names.
> >
> > I think it's vital we support JNI as far as possible (and the code
> > already does with commit 3ee3f65).
> >
> > However JNA is more of a backstop, so the fact that it has stopped
> > working for macOS and Windows is less of an issue.
> >
> > But I don't think we can say we are not supporting LibreSSL at all.
> >
> > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > >
> > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <ga...@gmail.com>
> > wrote:
> > > > >
> > > > > The simple solution is to keep doing what we do now: only support
> > OpenSSL
> > > > > and not whatever Apple does with LibreSSL which may or may not be up
> > to
> > > > > date.
> > > >
> > > > I think this also affects Windows.
> > > >
> > > > I don't have Windows box at present, but I have seen this on GH
> > Actions.
> > > >
> > > > If you have a WIndows build, perhaps you could try these tests:
> > > >
> > > > mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > mvn -q exec:java
> > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > >
> > > > The first one should show the SSL details; on GH the output includes:
> > > >
> > > > OpenSSL library loaded OK, version: 0x20000000
> > > > OpenSSL library info: LibreSSL 3.0.2
> > > > Additional OpenSSL_version(n) details:
> > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > >
> > > > The second test crashes with:
> > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > isEnabled(): false
> > > >
> > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error looking
> > > > up function 'ENGINE_load_rdrand': The specified procedure could not be
> > > > found.
> > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > ...
> > > >
> > > > It would certainly be easier to ignore the problem for now.
> > > >
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > default
> > > > > > rather than OpenSSL.
> > > > > >
> > > > > > The LibreSSL API does not have the same functions as either 1.0.2
> > or
> > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > 1.0.2
> > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > >
> > > > > > This makes it hard to support JNA with the current design.
> > > > > > It would require another OpenSsl<version>NativeJna class, and the
> > > > > > parent class OpenSslNativeJna would need to use yet another
> > condition
> > > > > > in each of its methods.
> > > > > >
> > > > > > This is quite tedious and error-prone.
> > > > > >
> > > > > > Seems to me it would be better to use something like a set of
> > > > > > singleton classes that implement the required methods. The
> > appropriate
> > > > > > one can then be initialised and used by OpenSslNativeJna to field
> > the
> > > > > > actual methods. i.e. replace the conditional logic with a static
> > > > > > reference to the correct API interface instance. This should only
> > > > > > affect non-public classes.
> > > > > >
> > > > > > Any other suggestions?
> > > > > >
> > > > > > Sebb
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Matt Sicker <bo...@gmail.com>.
The only OpenSSL fork I know of in macOS is BoringSSL which is also used by Chrome. This fork maintains some level of compatibility though.
—
Matt Sicker
> On Jun 29, 2022, at 20:03, Alex Remily <al...@gmail.com> wrote:
>
> Which Mac OS version did you use? Since I upgraded to BigSur (OS 11) my
> Commons Crypto builds fail. I think Apple moved a bunch of the developer
> libraries, but I haven't taken the time to troubleshoot.
>
>> On Wed, Jun 29, 2022 at 8:55 PM Gary Gregory <ga...@gmail.com> wrote:
>>
>> Arg, idiotic line wrapping removed: https://pastebin.com/raw/nPczrrd8
>>
>> Gary
>>
>>> On Wed, Jun 29, 2022 at 8:49 PM Gary Gregory <ga...@gmail.com>
>>> wrote:
>>>
>>> FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
>>> 'mvn clean verify' and everything built just fine.
>>>
>>> The Maven ant-run output was:
>>>
>>> [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
>>> [INFO] Executing tasks
>>>
>>> make:
>>> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
>>> -classpath target/classes -o
>>>
>> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
>>> org.apache.commons.crypto.random.OpenSslCryptoRandomNative
>>> [exec] gcc -arch x86_64 -Ilib/inc_mac
>>> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
>>> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
>>> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
>>> -I"src/main/native/org/apache/commons/crypto/"
>>> -I"/usr/local/Cellar/openjdk@8
>> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
>>> -I"target/jni-classes/org/apache/commons/crypto/cipher"
>>> -I"target/jni-classes/org/apache/commons/crypto/random" -c
>>>
>> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
>>> -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
>>> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
>>> -classpath target/classes -o
>>> target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
>>> org.apache.commons.crypto.cipher.OpenSslNative
>>> [exec] gcc -arch x86_64 -Ilib/inc_mac
>>> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
>>> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
>>> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
>>> -I"src/main/native/org/apache/commons/crypto/"
>>> -I"/usr/local/Cellar/openjdk@8
>> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
>>> -I"target/jni-classes/org/apache/commons/crypto/cipher"
>>> -I"target/jni-classes/org/apache/commons/crypto/random" -c
>>> src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
>>> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
>>> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
>>> -classpath target/classes -o
>>> target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
>>> org.apache.commons.crypto.OpenSslInfoNative
>>> [exec] gcc -arch x86_64 -Ilib/inc_mac
>>> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
>>> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
>>> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
>>> -I"src/main/native/org/apache/commons/crypto/"
>>> -I"/usr/local/Cellar/openjdk@8
>> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
>>> -I"target/jni-classes/org/apache/commons/crypto/cipher"
>>> -I"target/jni-classes/org/apache/commons/crypto/random"
>>> -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
>>> -I"target/jni-classes/org/apache/commons/crypto" -c
>>> src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
>>> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
>>> [exec] gcc -arch x86_64 -Ilib/inc_mac
>>> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
>>> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
>>> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
>>> -I"/usr/local/Cellar/openjdk@8
>> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
>>> -I"target/jni-classes/org/apache/commons/crypto/cipher"
>>> -I"target/jni-classes/org/apache/commons/crypto/random" -o
>>> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
>>> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
>>> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
>>> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
>>> -L/usr/local/lib
>>> [exec] strip -x
>>> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
>>> [exec] cp
>> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
>>>
>> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
>>> [exec] cp
>> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
>>>
>> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
>>>
>>> Gary
>>>
>>> On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com>
>> wrote:
>>>>
>>>> I agree with Alex.
>>>>
>>>> Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
>>>> tight. Last time I checked I had an antique version of LibreSSL on my
>>>> mac years ago, I just installed OpenSSL and never looked back.
>>>>
>>>> Gary
>>>>
>>>> On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com>
>> wrote:
>>>>>
>>>>> <The question is whether to do anything about the previously
>> undetected
>>>>> issues with JNA on macOS and Windows, and if so, whether this needs
>> to
>>>>> be done for the current or a later release. If we don't fix this
>>>>> release, obviously it needs some mention in the release notes.>
>>>>>
>>>>> I wouldn't characterize the issues running against LibreSSL as
>>>>> "undetected". I also don't see this as an issue with Mac or
>> Windows, but
>>>>> with LibreSSL. Install any supported OpenSSL library on any
>> supported
>>>>> architecture (to include Mac and Windows) and all commons crypto
>>>>> functionality is available. I first encountered the rand issue you
>>>>> describe years ago when I was becoming familiar with commons
>> crypto. I did
>>>>> a little research, discovered that I was running LibreSSL (and an old
>>>>> version at that), installed a supported version of OpenSSL and forgot
>>>>> all about it until this thread. I don't think it's unreasonable to
>> expect
>>>>> users to install a supported version of OpenSSL on a supported
>> architecture
>>>>> as a precondition of using commons crypto. I think it could become
>>>>> cumbersome to try and support every vendor default *SSL install.
>> That
>>>>> said, I don't have a problem committing this particular "fix" to the
>>>>> baseline, particularly since you have already done the work. I just
>> don't
>>>>> think that the project should obligate itself to do so or advertise
>>>>> LibreSSL (or any other non-OpenSSL branch) support as such. I'm
>> advocating
>>>>> a "use at your own risk" approach to anything but OpenSSL proper. I
>> agree
>>>>> that we should update the documentation to reflect whatever we move
>> forward
>>>>> with.
>>>>>
>>>>> Anyway, that's my $0.02 worth.
>>>>>
>>>>> Alex
>>>>>
>>>>> On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
>>>>>
>>>>>> On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com>
>> wrote:
>>>>>>>
>>>>>>> We cannot remove support for Windows and macOS, that's silly.
>>>>>>
>>>>>> AFAICT that means we must support the different set of function
>> names
>>>>>> in LibreSSL.
>>>>>> Note that we only currently use a small proportion of them.
>>>>>>
>>>>>>> I have not followed all the branches and commits, so I'm not
>> sure what
>>>>>> the
>>>>>>> current problems are, but I know I was able to release all OSs
>> last go
>>>>>>> around. I don't see why we need to rip out anything as
>> fundamental.
>>>>>>
>>>>>> Well, I have tried running the Crypto and OpenSslJna main classes
>> on
>>>>>> macOS and Windows, and they both fail with the 1.1.0 release.
>>>>>>
>>>>>> With current master, Crypto succeeds, but OpenSslJna fails to find
>>>>>> ENGINE_load_rdrand on both macOS and Windows.
>>>>>> (The job step succeeds, because the code catches the exception)
>>>>>>
>>>>>> It looks like the test which would have exposed at least one of the
>>>>>> issues was never enabled because of a failures on macOS; this hid
>> the
>>>>>> same problem on Windows.
>>>>>>
>>>>>> I am not suggesting we rip out anything at this point.
>>>>>>
>>>>>> The question is whether to do anything about the previously
>> undetected
>>>>>> issues with JNA on macOS and Windows, and if so, whether this
>> needs to
>>>>>> be done for the current or a later release. If we don't fix this
>>>>>> release, obviously it needs some mention in the release notes.
>>>>>>
>>>>>> Sebb
>>>>>> https://github.com/apache/commons-crypto/actions/runs/2586011129
>>>>>>> Gary
>>>>>>>
>>>>>>> On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
>>>>>>>
>>>>>>>> On Wed, 29 Jun 2022 at 16:11, Alex Remily <
>> alex.remily@gmail.com>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I agree with Gary that we just don't support LibreSSL. To my
>>>>>> knowledge
>>>>>>>>> we've never advertised LibreSSL support, so I don't see it
>> as an
>>>>>> issue.
>>>>>>>>
>>>>>>>> In that case AFAICT we will have to drop *all* support for
>> macOS and
>>>>>>>> Windows, as they both seem to default to LibreSSL.
>>>>>>>>
>>>>>>>> Note that adding support for LibreSSL was much easier for JNI,
>> as it
>>>>>>>> uses far fewer methods.
>>>>>>>> Rather than checking the version, I changed the code to try
>> OpenSSL
>>>>>>>> 1.1 names first, then a fallback.
>>>>>>>> That happens to work for 1.0.x and 1.1.x and LibreSSL.
>>>>>>>>
>>>>>>>> If you want to try it out, compare 16345bc (old) with 3ee3f65
>> (new)
>>>>>>>> macOS fails on 16345bc because it now uses LibreSSL which has a
>>>>>>>> different mix of names.
>>>>>>>>
>>>>>>>> I think it's vital we support JNI as far as possible (and the
>> code
>>>>>>>> already does with commit 3ee3f65).
>>>>>>>>
>>>>>>>> However JNA is more of a backstop, so the fact that it has
>> stopped
>>>>>>>> working for macOS and Windows is less of an issue.
>>>>>>>>
>>>>>>>> But I don't think we can say we are not supporting LibreSSL at
>> all.
>>>>>>>>
>>>>>>>>> On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com>
>> wrote:
>>>>>>>>>
>>>>>>>>>> On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
>> garydgregory@gmail.com
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> The simple solution is to keep doing what we do now:
>> only support
>>>>>>>> OpenSSL
>>>>>>>>>>> and not whatever Apple does with LibreSSL which may or
>> may not
>>>>>> be up
>>>>>>>> to
>>>>>>>>>>> date.
>>>>>>>>>>
>>>>>>>>>> I think this also affects Windows.
>>>>>>>>>>
>>>>>>>>>> I don't have Windows box at present, but I have seen this
>> on GH
>>>>>>>> Actions.
>>>>>>>>>>
>>>>>>>>>> If you have a WIndows build, perhaps you could try these
>> tests:
>>>>>>>>>>
>>>>>>>>>> mvn -q exec:java
>>>>>> -D"exec.mainClass=org.apache.commons.crypto.Crypto"
>>>>>>>>>> mvn -q exec:java
>>>>>>>>>> -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
>>>>>>>>>>
>>>>>>>>>> The first one should show the SSL details; on GH the output
>>>>>> includes:
>>>>>>>>>>
>>>>>>>>>> OpenSSL library loaded OK, version: 0x20000000
>>>>>>>>>> OpenSSL library info: LibreSSL 3.0.2
>>>>>>>>>> Additional OpenSSL_version(n) details:
>>>>>>>>>> 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
>>>>>>>>>>
>>>>>>>>>> The second test crashes with:
>>>>>>>>>> java.lang.UnsatisfiedLinkError: Error looking up function
>>>>>>>>>> 'ENGINE_load_rdrand': The specified procedure could not be
>> found.
>>>>>>>>>> isEnabled(): false
>>>>>>>>>>
>>>>>>>>>> initialisationError(): java.lang.UnsatisfiedLinkError:
>> Error
>>>>>> looking
>>>>>>>>>> up function 'ENGINE_load_rdrand': The specified procedure
>> could
>>>>>> not be
>>>>>>>>>> found.
>>>>>>>>>> at com.sun.jna.Function.<init>(Function.java:252)
>>>>>>>>>> ...
>>>>>>>>>>
>>>>>>>>>> It would certainly be easier to ignore the problem for now.
>>>>>>>>>>
>>>>>>>>>>> Gary
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com>
>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> It looks like macOS 10.5+ and Windows (latest) use
>> LibreSSL by
>>>>>>>> default
>>>>>>>>>>>> rather than OpenSSL.
>>>>>>>>>>>>
>>>>>>>>>>>> The LibreSSL API does not have the same functions as
>> either
>>>>>> 1.0.2
>>>>>>>> or
>>>>>>>>>>>> 1.1.1, so needs its own JNA class. In particular it
>> looks like
>>>>>>>>>>>> ENGINE_load_rdrand is not present, nor is
>> OpenSSL_version_num;
>>>>>>>> 1.0.2
>>>>>>>>>>>> has the former only, and 1.1.1 has the latter only.
>>>>>>>>>>>>
>>>>>>>>>>>> This makes it hard to support JNA with the current
>> design.
>>>>>>>>>>>> It would require another OpenSsl<version>NativeJna
>> class, and
>>>>>> the
>>>>>>>>>>>> parent class OpenSslNativeJna would need to use yet
>> another
>>>>>>>> condition
>>>>>>>>>>>> in each of its methods.
>>>>>>>>>>>>
>>>>>>>>>>>> This is quite tedious and error-prone.
>>>>>>>>>>>>
>>>>>>>>>>>> Seems to me it would be better to use something like a
>> set of
>>>>>>>>>>>> singleton classes that implement the required methods.
>> The
>>>>>>>> appropriate
>>>>>>>>>>>> one can then be initialised and used by
>> OpenSslNativeJna to
>>>>>> field
>>>>>>>> the
>>>>>>>>>>>> actual methods. i.e. replace the conditional logic
>> with a
>>>>>> static
>>>>>>>>>>>> reference to the correct API interface instance. This
>> should
>>>>>> only
>>>>>>>>>>>> affect non-public classes.
>>>>>>>>>>>>
>>>>>>>>>>>> Any other suggestions?
>>>>>>>>>>>>
>>>>>>>>>>>> Sebb
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe, e-mail:
>> dev-unsubscribe@commons.apache.org
>>>>>>>>>>>> For additional commands, e-mail:
>> dev-help@commons.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>>>> For additional commands, e-mail:
>> dev-help@commons.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Thu, 30 Jun 2022 at 02:03, Alex Remily <al...@gmail.com> wrote:
>
> Which Mac OS version did you use? Since I upgraded to BigSur (OS 11) my
> Commons Crypto builds fail. I think Apple moved a bunch of the developer
> libraries, but I haven't taken the time to troubleshoot.
Note that GH Action macos-latest uses OS 11.6.6 and that compiles the
current code fine.
https://github.com/apache/commons-crypto/actions/runs/2586011129
> On Wed, Jun 29, 2022 at 8:55 PM Gary Gregory <ga...@gmail.com> wrote:
>
> > Arg, idiotic line wrapping removed: https://pastebin.com/raw/nPczrrd8
> >
> > Gary
> >
> > On Wed, Jun 29, 2022 at 8:49 PM Gary Gregory <ga...@gmail.com>
> > wrote:
> > >
> > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> > > 'mvn clean verify' and everything built just fine.
> > >
> > > The Maven ant-run output was:
> > >
> > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > > [INFO] Executing tasks
> > >
> > > make:
> > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > -classpath target/classes -o
> > >
> > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"src/main/native/org/apache/commons/crypto/"
> > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > >
> > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > -classpath target/classes -o
> > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > org.apache.commons.crypto.cipher.OpenSslNative
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"src/main/native/org/apache/commons/crypto/"
> > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > -classpath target/classes -o
> > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > org.apache.commons.crypto.OpenSslInfoNative
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"src/main/native/org/apache/commons/crypto/"
> > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> > > -L/usr/local/lib
> > > [exec] strip -x
> > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > [exec] cp
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > >
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > [exec] cp
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > >
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > >
> > > Gary
> > >
> > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com>
> > wrote:
> > > >
> > > > I agree with Alex.
> > > >
> > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > > tight. Last time I checked I had an antique version of LibreSSL on my
> > > > mac years ago, I just installed OpenSSL and never looked back.
> > > >
> > > > Gary
> > > >
> > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com>
> > wrote:
> > > > >
> > > > > <The question is whether to do anything about the previously
> > undetected
> > > > > issues with JNA on macOS and Windows, and if so, whether this needs
> > to
> > > > > be done for the current or a later release. If we don't fix this
> > > > > release, obviously it needs some mention in the release notes.>
> > > > >
> > > > > I wouldn't characterize the issues running against LibreSSL as
> > > > > "undetected". I also don't see this as an issue with Mac or
> > Windows, but
> > > > > with LibreSSL. Install any supported OpenSSL library on any
> > supported
> > > > > architecture (to include Mac and Windows) and all commons crypto
> > > > > functionality is available. I first encountered the rand issue you
> > > > > describe years ago when I was becoming familiar with commons
> > crypto. I did
> > > > > a little research, discovered that I was running LibreSSL (and an old
> > > > > version at that), installed a supported version of OpenSSL and forgot
> > > > > all about it until this thread. I don't think it's unreasonable to
> > expect
> > > > > users to install a supported version of OpenSSL on a supported
> > architecture
> > > > > as a precondition of using commons crypto. I think it could become
> > > > > cumbersome to try and support every vendor default *SSL install.
> > That
> > > > > said, I don't have a problem committing this particular "fix" to the
> > > > > baseline, particularly since you have already done the work. I just
> > don't
> > > > > think that the project should obligate itself to do so or advertise
> > > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm
> > advocating
> > > > > a "use at your own risk" approach to anything but OpenSSL proper. I
> > agree
> > > > > that we should update the documentation to reflect whatever we move
> > forward
> > > > > with.
> > > > >
> > > > > Anyway, that's my $0.02 worth.
> > > > >
> > > > > Alex
> > > > >
> > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > > >
> > > > > > AFAICT that means we must support the different set of function
> > names
> > > > > > in LibreSSL.
> > > > > > Note that we only currently use a small proportion of them.
> > > > > >
> > > > > > > I have not followed all the branches and commits, so I'm not
> > sure what
> > > > > > the
> > > > > > > current problems are, but I know I was able to release all OSs
> > last go
> > > > > > > around. I don't see why we need to rip out anything as
> > fundamental.
> > > > > >
> > > > > > Well, I have tried running the Crypto and OpenSslJna main classes
> > on
> > > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > > >
> > > > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > (The job step succeeds, because the code catches the exception)
> > > > > >
> > > > > > It looks like the test which would have exposed at least one of the
> > > > > > issues was never enabled because of a failures on macOS; this hid
> > the
> > > > > > same problem on Windows.
> > > > > >
> > > > > > I am not suggesting we rip out anything at this point.
> > > > > >
> > > > > > The question is whether to do anything about the previously
> > undetected
> > > > > > issues with JNA on macOS and Windows, and if so, whether this
> > needs to
> > > > > > be done for the current or a later release. If we don't fix this
> > > > > > release, obviously it needs some mention in the release notes.
> > > > > >
> > > > > > Sebb
> > > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > Gary
> > > > > > >
> > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> > alex.remily@gmail.com>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > > > knowledge
> > > > > > > > > we've never advertised LibreSSL support, so I don't see it
> > as an
> > > > > > issue.
> > > > > > > >
> > > > > > > > In that case AFAICT we will have to drop *all* support for
> > macOS and
> > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > >
> > > > > > > > Note that adding support for LibreSSL was much easier for JNI,
> > as it
> > > > > > > > uses far fewer methods.
> > > > > > > > Rather than checking the version, I changed the code to try
> > OpenSSL
> > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > >
> > > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65
> > (new)
> > > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > > > different mix of names.
> > > > > > > >
> > > > > > > > I think it's vital we support JNI as far as possible (and the
> > code
> > > > > > > > already does with commit 3ee3f65).
> > > > > > > >
> > > > > > > > However JNA is more of a backstop, so the fact that it has
> > stopped
> > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > >
> > > > > > > > But I don't think we can say we are not supporting LibreSSL at
> > all.
> > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com>
> > wrote:
> > > > > > > > >
> > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> > garydgregory@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > The simple solution is to keep doing what we do now:
> > only support
> > > > > > > > OpenSSL
> > > > > > > > > > > and not whatever Apple does with LibreSSL which may or
> > may not
> > > > > > be up
> > > > > > > > to
> > > > > > > > > > > date.
> > > > > > > > > >
> > > > > > > > > > I think this also affects Windows.
> > > > > > > > > >
> > > > > > > > > > I don't have Windows box at present, but I have seen this
> > on GH
> > > > > > > > Actions.
> > > > > > > > > >
> > > > > > > > > > If you have a WIndows build, perhaps you could try these
> > tests:
> > > > > > > > > >
> > > > > > > > > > mvn -q exec:java
> > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > mvn -q exec:java
> > > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > >
> > > > > > > > > > The first one should show the SSL details; on GH the output
> > > > > > includes:
> > > > > > > > > >
> > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > >
> > > > > > > > > > The second test crashes with:
> > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be
> > found.
> > > > > > > > > > isEnabled(): false
> > > > > > > > > >
> > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError:
> > Error
> > > > > > looking
> > > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure
> > could
> > > > > > not be
> > > > > > > > > > found.
> > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > ...
> > > > > > > > > >
> > > > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > > > >
> > > > > > > > > > > Gary
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com>
> > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use
> > LibreSSL by
> > > > > > > > default
> > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > >
> > > > > > > > > > > > The LibreSSL API does not have the same functions as
> > either
> > > > > > 1.0.2
> > > > > > > > or
> > > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it
> > looks like
> > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> > OpenSSL_version_num;
> > > > > > > > 1.0.2
> > > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > > >
> > > > > > > > > > > > This makes it hard to support JNA with the current
> > design.
> > > > > > > > > > > > It would require another OpenSsl<version>NativeJna
> > class, and
> > > > > > the
> > > > > > > > > > > > parent class OpenSslNativeJna would need to use yet
> > another
> > > > > > > > condition
> > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > >
> > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > >
> > > > > > > > > > > > Seems to me it would be better to use something like a
> > set of
> > > > > > > > > > > > singleton classes that implement the required methods.
> > The
> > > > > > > > appropriate
> > > > > > > > > > > > one can then be initialised and used by
> > OpenSslNativeJna to
> > > > > > field
> > > > > > > > the
> > > > > > > > > > > > actual methods. i.e. replace the conditional logic
> > with a
> > > > > > static
> > > > > > > > > > > > reference to the correct API interface instance. This
> > should
> > > > > > only
> > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > >
> > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > >
> > > > > > > > > > > > Sebb
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@commons.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > For additional commands, e-mail:
> > dev-help@commons.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
Which Mac OS version did you use? Since I upgraded to BigSur (OS 11) my
Commons Crypto builds fail. I think Apple moved a bunch of the developer
libraries, but I haven't taken the time to troubleshoot.
On Wed, Jun 29, 2022 at 8:55 PM Gary Gregory <ga...@gmail.com> wrote:
> Arg, idiotic line wrapping removed: https://pastebin.com/raw/nPczrrd8
>
> Gary
>
> On Wed, Jun 29, 2022 at 8:49 PM Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> > 'mvn clean verify' and everything built just fine.
> >
> > The Maven ant-run output was:
> >
> > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > [INFO] Executing tasks
> >
> > make:
> > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > -classpath target/classes -o
> >
> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"src/main/native/org/apache/commons/crypto/"
> > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> >
> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > -classpath target/classes -o
> > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > org.apache.commons.crypto.cipher.OpenSslNative
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"src/main/native/org/apache/commons/crypto/"
> > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > -classpath target/classes -o
> > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > org.apache.commons.crypto.OpenSslInfoNative
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"src/main/native/org/apache/commons/crypto/"
> > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random"
> > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > -I"target/jni-classes/org/apache/commons/crypto" -c
> > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> > -L/usr/local/lib
> > [exec] strip -x
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > [exec] cp
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > [exec] cp
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> >
> > Gary
> >
> > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com>
> wrote:
> > >
> > > I agree with Alex.
> > >
> > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > tight. Last time I checked I had an antique version of LibreSSL on my
> > > mac years ago, I just installed OpenSSL and never looked back.
> > >
> > > Gary
> > >
> > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com>
> wrote:
> > > >
> > > > <The question is whether to do anything about the previously
> undetected
> > > > issues with JNA on macOS and Windows, and if so, whether this needs
> to
> > > > be done for the current or a later release. If we don't fix this
> > > > release, obviously it needs some mention in the release notes.>
> > > >
> > > > I wouldn't characterize the issues running against LibreSSL as
> > > > "undetected". I also don't see this as an issue with Mac or
> Windows, but
> > > > with LibreSSL. Install any supported OpenSSL library on any
> supported
> > > > architecture (to include Mac and Windows) and all commons crypto
> > > > functionality is available. I first encountered the rand issue you
> > > > describe years ago when I was becoming familiar with commons
> crypto. I did
> > > > a little research, discovered that I was running LibreSSL (and an old
> > > > version at that), installed a supported version of OpenSSL and forgot
> > > > all about it until this thread. I don't think it's unreasonable to
> expect
> > > > users to install a supported version of OpenSSL on a supported
> architecture
> > > > as a precondition of using commons crypto. I think it could become
> > > > cumbersome to try and support every vendor default *SSL install.
> That
> > > > said, I don't have a problem committing this particular "fix" to the
> > > > baseline, particularly since you have already done the work. I just
> don't
> > > > think that the project should obligate itself to do so or advertise
> > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm
> advocating
> > > > a "use at your own risk" approach to anything but OpenSSL proper. I
> agree
> > > > that we should update the documentation to reflect whatever we move
> forward
> > > > with.
> > > >
> > > > Anyway, that's my $0.02 worth.
> > > >
> > > > Alex
> > > >
> > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > >
> > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com>
> wrote:
> > > > > >
> > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > >
> > > > > AFAICT that means we must support the different set of function
> names
> > > > > in LibreSSL.
> > > > > Note that we only currently use a small proportion of them.
> > > > >
> > > > > > I have not followed all the branches and commits, so I'm not
> sure what
> > > > > the
> > > > > > current problems are, but I know I was able to release all OSs
> last go
> > > > > > around. I don't see why we need to rip out anything as
> fundamental.
> > > > >
> > > > > Well, I have tried running the Crypto and OpenSslJna main classes
> on
> > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > >
> > > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > (The job step succeeds, because the code catches the exception)
> > > > >
> > > > > It looks like the test which would have exposed at least one of the
> > > > > issues was never enabled because of a failures on macOS; this hid
> the
> > > > > same problem on Windows.
> > > > >
> > > > > I am not suggesting we rip out anything at this point.
> > > > >
> > > > > The question is whether to do anything about the previously
> undetected
> > > > > issues with JNA on macOS and Windows, and if so, whether this
> needs to
> > > > > be done for the current or a later release. If we don't fix this
> > > > > release, obviously it needs some mention in the release notes.
> > > > >
> > > > > Sebb
> > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > Gary
> > > > > >
> > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> alex.remily@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > > knowledge
> > > > > > > > we've never advertised LibreSSL support, so I don't see it
> as an
> > > > > issue.
> > > > > > >
> > > > > > > In that case AFAICT we will have to drop *all* support for
> macOS and
> > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > >
> > > > > > > Note that adding support for LibreSSL was much easier for JNI,
> as it
> > > > > > > uses far fewer methods.
> > > > > > > Rather than checking the version, I changed the code to try
> OpenSSL
> > > > > > > 1.1 names first, then a fallback.
> > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > >
> > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65
> (new)
> > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > > different mix of names.
> > > > > > >
> > > > > > > I think it's vital we support JNI as far as possible (and the
> code
> > > > > > > already does with commit 3ee3f65).
> > > > > > >
> > > > > > > However JNA is more of a backstop, so the fact that it has
> stopped
> > > > > > > working for macOS and Windows is less of an issue.
> > > > > > >
> > > > > > > But I don't think we can say we are not supporting LibreSSL at
> all.
> > > > > > >
> > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com>
> wrote:
> > > > > > > >
> > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> garydgregory@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > The simple solution is to keep doing what we do now:
> only support
> > > > > > > OpenSSL
> > > > > > > > > > and not whatever Apple does with LibreSSL which may or
> may not
> > > > > be up
> > > > > > > to
> > > > > > > > > > date.
> > > > > > > > >
> > > > > > > > > I think this also affects Windows.
> > > > > > > > >
> > > > > > > > > I don't have Windows box at present, but I have seen this
> on GH
> > > > > > > Actions.
> > > > > > > > >
> > > > > > > > > If you have a WIndows build, perhaps you could try these
> tests:
> > > > > > > > >
> > > > > > > > > mvn -q exec:java
> > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > mvn -q exec:java
> > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > >
> > > > > > > > > The first one should show the SSL details; on GH the output
> > > > > includes:
> > > > > > > > >
> > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > >
> > > > > > > > > The second test crashes with:
> > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be
> found.
> > > > > > > > > isEnabled(): false
> > > > > > > > >
> > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError:
> Error
> > > > > looking
> > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure
> could
> > > > > not be
> > > > > > > > > found.
> > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > ...
> > > > > > > > >
> > > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > > >
> > > > > > > > > > Gary
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com>
> wrote:
> > > > > > > > > >
> > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use
> LibreSSL by
> > > > > > > default
> > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > >
> > > > > > > > > > > The LibreSSL API does not have the same functions as
> either
> > > > > 1.0.2
> > > > > > > or
> > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it
> looks like
> > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> OpenSSL_version_num;
> > > > > > > 1.0.2
> > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > >
> > > > > > > > > > > This makes it hard to support JNA with the current
> design.
> > > > > > > > > > > It would require another OpenSsl<version>NativeJna
> class, and
> > > > > the
> > > > > > > > > > > parent class OpenSslNativeJna would need to use yet
> another
> > > > > > > condition
> > > > > > > > > > > in each of its methods.
> > > > > > > > > > >
> > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > >
> > > > > > > > > > > Seems to me it would be better to use something like a
> set of
> > > > > > > > > > > singleton classes that implement the required methods.
> The
> > > > > > > appropriate
> > > > > > > > > > > one can then be initialised and used by
> OpenSslNativeJna to
> > > > > field
> > > > > > > the
> > > > > > > > > > > actual methods. i.e. replace the conditional logic
> with a
> > > > > static
> > > > > > > > > > > reference to the correct API interface instance. This
> should
> > > > > only
> > > > > > > > > > > affect non-public classes.
> > > > > > > > > > >
> > > > > > > > > > > Any other suggestions?
> > > > > > > > > > >
> > > > > > > > > > > Sebb
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> dev-help@commons.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > For additional commands, e-mail:
> dev-help@commons.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
Arg, idiotic line wrapping removed: https://pastebin.com/raw/nPczrrd8
Gary
On Wed, Jun 29, 2022 at 8:49 PM Gary Gregory <ga...@gmail.com> wrote:
>
> FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> 'mvn clean verify' and everything built just fine.
>
> The Maven ant-run output was:
>
> [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> [INFO] Executing tasks
>
> make:
> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> -classpath target/classes -o
> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"src/main/native/org/apache/commons/crypto/"
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random" -c
> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> -classpath target/classes -o
> target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> org.apache.commons.crypto.cipher.OpenSslNative
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"src/main/native/org/apache/commons/crypto/"
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random" -c
> src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> -classpath target/classes -o
> target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> org.apache.commons.crypto.OpenSslInfoNative
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"src/main/native/org/apache/commons/crypto/"
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random"
> -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> -I"target/jni-classes/org/apache/commons/crypto" -c
> src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random" -o
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> -L/usr/local/lib
> [exec] strip -x
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
>
> Gary
>
> On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com> wrote:
> >
> > I agree with Alex.
> >
> > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > tight. Last time I checked I had an antique version of LibreSSL on my
> > mac years ago, I just installed OpenSSL and never looked back.
> >
> > Gary
> >
> > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
> > >
> > > <The question is whether to do anything about the previously undetected
> > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > be done for the current or a later release. If we don't fix this
> > > release, obviously it needs some mention in the release notes.>
> > >
> > > I wouldn't characterize the issues running against LibreSSL as
> > > "undetected". I also don't see this as an issue with Mac or Windows, but
> > > with LibreSSL. Install any supported OpenSSL library on any supported
> > > architecture (to include Mac and Windows) and all commons crypto
> > > functionality is available. I first encountered the rand issue you
> > > describe years ago when I was becoming familiar with commons crypto. I did
> > > a little research, discovered that I was running LibreSSL (and an old
> > > version at that), installed a supported version of OpenSSL and forgot
> > > all about it until this thread. I don't think it's unreasonable to expect
> > > users to install a supported version of OpenSSL on a supported architecture
> > > as a precondition of using commons crypto. I think it could become
> > > cumbersome to try and support every vendor default *SSL install. That
> > > said, I don't have a problem committing this particular "fix" to the
> > > baseline, particularly since you have already done the work. I just don't
> > > think that the project should obligate itself to do so or advertise
> > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> > > a "use at your own risk" approach to anything but OpenSSL proper. I agree
> > > that we should update the documentation to reflect whatever we move forward
> > > with.
> > >
> > > Anyway, that's my $0.02 worth.
> > >
> > > Alex
> > >
> > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > >
> > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > > > >
> > > > > We cannot remove support for Windows and macOS, that's silly.
> > > >
> > > > AFAICT that means we must support the different set of function names
> > > > in LibreSSL.
> > > > Note that we only currently use a small proportion of them.
> > > >
> > > > > I have not followed all the branches and commits, so I'm not sure what
> > > > the
> > > > > current problems are, but I know I was able to release all OSs last go
> > > > > around. I don't see why we need to rip out anything as fundamental.
> > > >
> > > > Well, I have tried running the Crypto and OpenSslJna main classes on
> > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > >
> > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > (The job step succeeds, because the code catches the exception)
> > > >
> > > > It looks like the test which would have exposed at least one of the
> > > > issues was never enabled because of a failures on macOS; this hid the
> > > > same problem on Windows.
> > > >
> > > > I am not suggesting we rip out anything at this point.
> > > >
> > > > The question is whether to do anything about the previously undetected
> > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > be done for the current or a later release. If we don't fix this
> > > > release, obviously it needs some mention in the release notes.
> > > >
> > > > Sebb
> > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > knowledge
> > > > > > > we've never advertised LibreSSL support, so I don't see it as an
> > > > issue.
> > > > > >
> > > > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > >
> > > > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > > > uses far fewer methods.
> > > > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > > > 1.1 names first, then a fallback.
> > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > >
> > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > different mix of names.
> > > > > >
> > > > > > I think it's vital we support JNI as far as possible (and the code
> > > > > > already does with commit 3ee3f65).
> > > > > >
> > > > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > > > working for macOS and Windows is less of an issue.
> > > > > >
> > > > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > > > >
> > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > The simple solution is to keep doing what we do now: only support
> > > > > > OpenSSL
> > > > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > > > be up
> > > > > > to
> > > > > > > > > date.
> > > > > > > >
> > > > > > > > I think this also affects Windows.
> > > > > > > >
> > > > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > > > Actions.
> > > > > > > >
> > > > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > > > >
> > > > > > > > mvn -q exec:java
> > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > mvn -q exec:java
> > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > >
> > > > > > > > The first one should show the SSL details; on GH the output
> > > > includes:
> > > > > > > >
> > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > >
> > > > > > > > The second test crashes with:
> > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > > > isEnabled(): false
> > > > > > > >
> > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > > > looking
> > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > > > not be
> > > > > > > > found.
> > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > ...
> > > > > > > >
> > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > >
> > > > > > > > > Gary
> > > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > > > default
> > > > > > > > > > rather than OpenSSL.
> > > > > > > > > >
> > > > > > > > > > The LibreSSL API does not have the same functions as either
> > > > 1.0.2
> > > > > > or
> > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > > > 1.0.2
> > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > >
> > > > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > > > the
> > > > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > > > condition
> > > > > > > > > > in each of its methods.
> > > > > > > > > >
> > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > >
> > > > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > > > singleton classes that implement the required methods. The
> > > > > > appropriate
> > > > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > > > field
> > > > > > the
> > > > > > > > > > actual methods. i.e. replace the conditional logic with a
> > > > static
> > > > > > > > > > reference to the correct API interface instance. This should
> > > > only
> > > > > > > > > > affect non-public classes.
> > > > > > > > > >
> > > > > > > > > > Any other suggestions?
> > > > > > > > > >
> > > > > > > > > > Sebb
> > > > > > > > > >
> > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
On Thu, Jun 30, 2022 at 8:02 AM sebb <se...@gmail.com> wrote:
>
> That looks fine, however I don't see the same, and nor does GH.
>
> What version of macOS are you using?
uname says: Darwin xxx 21.5.0 Darwin Kernel Version 21.5.0: Tue Apr 26
21:08:22 PDT 2022; root:xnu-8020.121.3~4/RELEASE_X86_64 x86_64
UI says 12.4
Gary
>
> On Thu, 30 Jun 2022 at 12:52, Gary Gregory <ga...@gmail.com> wrote:
> >
> > With the 1.1.0 tagm I get:
> >
> > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> > target/classes org.apache.commons.crypto.Crypto
> > Apache Commons Crypto 1.1.0
> > Native code loaded OK: 1.1.0
> > Native name: Apache Commons Crypto
> > Native built: Jun 30 2022
> > OpenSSL library loaded OK, version: 0x1010107f
> > OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> > Random instance created OK:
> > org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> > Cipher AES/CTR/NoPadding instance created OK:
> > org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> > Additional OpenSSL_version(n) details:
> > 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> > -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> > -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
> > -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> > -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> > -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> > 2: built on: Tue Apr 21 13:30:00 2020 UTC
> > 3: platform: darwin64-x86_64-cc
> > 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> > 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1/1.1.1g/lib/engines-1.1"
> >
> > and
> >
> > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q exec:java
> > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > isEnabled(): true
> > initialisationError(): null
> >
> > Gary
> >
> > On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> > >
> > > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> > >
> > > As previously noted, please try:
> > > java -cp target/classes org.apache.commons.crypto.Crypto
> > > and
> > > mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > >
> > > See the GH run I linked yesterday:
> > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > >
> > > I just noticed also that 'openssl version' shows 1.1.1p but the Crypto
> > > class shows
> > > OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0 code crashes)
> > >
> > > Something strange is going on; it looks like dlopen is finding a
> > > different version of the dynamic library from the default CLI.
> > > I have the same issue on my mac.
> > >
> > > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <ga...@gmail.com> wrote:
> > > >
> > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> > > > 'mvn clean verify' and everything built just fine.
> > > >
> > > > The Maven ant-run output was:
> > > >
> > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > > > [INFO] Executing tasks
> > > >
> > > > make:
> > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > -classpath target/classes -o
> > > > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > -classpath target/classes -o
> > > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > > org.apache.commons.crypto.cipher.OpenSslNative
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > -classpath target/classes -o
> > > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > > org.apache.commons.crypto.OpenSslInfoNative
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> > > > -L/usr/local/lib
> > > > [exec] strip -x
> > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > >
> > > > Gary
> > > >
> > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com> wrote:
> > > > >
> > > > > I agree with Alex.
> > > > >
> > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > > > tight. Last time I checked I had an antique version of LibreSSL on my
> > > > > mac years ago, I just installed OpenSSL and never looked back.
> > > > >
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
> > > > > >
> > > > > > <The question is whether to do anything about the previously undetected
> > > > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > > > be done for the current or a later release. If we don't fix this
> > > > > > release, obviously it needs some mention in the release notes.>
> > > > > >
> > > > > > I wouldn't characterize the issues running against LibreSSL as
> > > > > > "undetected". I also don't see this as an issue with Mac or Windows, but
> > > > > > with LibreSSL. Install any supported OpenSSL library on any supported
> > > > > > architecture (to include Mac and Windows) and all commons crypto
> > > > > > functionality is available. I first encountered the rand issue you
> > > > > > describe years ago when I was becoming familiar with commons crypto. I did
> > > > > > a little research, discovered that I was running LibreSSL (and an old
> > > > > > version at that), installed a supported version of OpenSSL and forgot
> > > > > > all about it until this thread. I don't think it's unreasonable to expect
> > > > > > users to install a supported version of OpenSSL on a supported architecture
> > > > > > as a precondition of using commons crypto. I think it could become
> > > > > > cumbersome to try and support every vendor default *SSL install. That
> > > > > > said, I don't have a problem committing this particular "fix" to the
> > > > > > baseline, particularly since you have already done the work. I just don't
> > > > > > think that the project should obligate itself to do so or advertise
> > > > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> > > > > > a "use at your own risk" approach to anything but OpenSSL proper. I agree
> > > > > > that we should update the documentation to reflect whatever we move forward
> > > > > > with.
> > > > > >
> > > > > > Anyway, that's my $0.02 worth.
> > > > > >
> > > > > > Alex
> > > > > >
> > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > > > >
> > > > > > > AFAICT that means we must support the different set of function names
> > > > > > > in LibreSSL.
> > > > > > > Note that we only currently use a small proportion of them.
> > > > > > >
> > > > > > > > I have not followed all the branches and commits, so I'm not sure what
> > > > > > > the
> > > > > > > > current problems are, but I know I was able to release all OSs last go
> > > > > > > > around. I don't see why we need to rip out anything as fundamental.
> > > > > > >
> > > > > > > Well, I have tried running the Crypto and OpenSslJna main classes on
> > > > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > > > >
> > > > > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > > (The job step succeeds, because the code catches the exception)
> > > > > > >
> > > > > > > It looks like the test which would have exposed at least one of the
> > > > > > > issues was never enabled because of a failures on macOS; this hid the
> > > > > > > same problem on Windows.
> > > > > > >
> > > > > > > I am not suggesting we rip out anything at this point.
> > > > > > >
> > > > > > > The question is whether to do anything about the previously undetected
> > > > > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > > > > be done for the current or a later release. If we don't fix this
> > > > > > > release, obviously it needs some mention in the release notes.
> > > > > > >
> > > > > > > Sebb
> > > > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > > Gary
> > > > > > > >
> > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > > > > knowledge
> > > > > > > > > > we've never advertised LibreSSL support, so I don't see it as an
> > > > > > > issue.
> > > > > > > > >
> > > > > > > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > > >
> > > > > > > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > > > > > > uses far fewer methods.
> > > > > > > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > > >
> > > > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > > > > different mix of names.
> > > > > > > > >
> > > > > > > > > I think it's vital we support JNI as far as possible (and the code
> > > > > > > > > already does with commit 3ee3f65).
> > > > > > > > >
> > > > > > > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > > >
> > > > > > > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > > > > > > >
> > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > The simple solution is to keep doing what we do now: only support
> > > > > > > > > OpenSSL
> > > > > > > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > > > > > > be up
> > > > > > > > > to
> > > > > > > > > > > > date.
> > > > > > > > > > >
> > > > > > > > > > > I think this also affects Windows.
> > > > > > > > > > >
> > > > > > > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > > > > > > Actions.
> > > > > > > > > > >
> > > > > > > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > > > > > > >
> > > > > > > > > > > mvn -q exec:java
> > > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > > >
> > > > > > > > > > > The first one should show the SSL details; on GH the output
> > > > > > > includes:
> > > > > > > > > > >
> > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > > >
> > > > > > > > > > > The second test crashes with:
> > > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > > > > > > isEnabled(): false
> > > > > > > > > > >
> > > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > > > > > > looking
> > > > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > > > > > > not be
> > > > > > > > > > > found.
> > > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > > ...
> > > > > > > > > > >
> > > > > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > > > > >
> > > > > > > > > > > > Gary
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > > > > > > default
> > > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The LibreSSL API does not have the same functions as either
> > > > > > > 1.0.2
> > > > > > > > > or
> > > > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > > > > > > 1.0.2
> > > > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > > > > > > the
> > > > > > > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > > > > > > condition
> > > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > > > > > > singleton classes that implement the required methods. The
> > > > > > > > > appropriate
> > > > > > > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > > > > > > field
> > > > > > > > > the
> > > > > > > > > > > > > actual methods. i.e. replace the conditional logic with a
> > > > > > > static
> > > > > > > > > > > > > reference to the correct API interface instance. This should
> > > > > > > only
> > > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Sebb
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
I tinkered with the same issues today and have come to the
same conclusion. We may want to consider optionally prepending a path to
the COMMONS_CRYPTO_OPENSSL_LIBRARY to specify a specific libcrypto.
On Thu, Jun 30, 2022 at 5:35 PM sebb <se...@gmail.com> wrote:
> On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com> wrote:
> >
> > <For example, the standalone code takes note of LD_LIBRARY_PATH when
> > loading the dll, whereas Java apparently does not.>
> >
> > I experience the same behavior. What's more interesting is that when I
> run
> > the main function from Eclipse as a Run Configuration with the
> > LD_LIBRARY_PATH set as an Environment variable appended to the native
> > environment, it runs as expected. As of yet I haven't found a way to get
> > the java CLI to recognize the LD_LIBRARY_PATH environment variable, even
> > though it echos out correctly when queried. Strange.
>
> If you use getenv within the JNI C code, it does not see any of the
> library-related environment variables.
>
> It looks like DYLD_* is dropped, along with LD_LIBRARY_PATH.
>
> There is some strange behaviour with export as well.
> Try the following:
>
> export LD_LIBRART_PATH=/usr/lib
> export LD_LIBRARY_PATH=/usr/lib
> env | grep LIBRAR
>
> I get:
>
> LD_LIBRART_PATH=/usr/lib
>
> Similarly with DYLD_*
>
> I think we may need to allow the user to provide the full path name to
> the crypto library.
>
> > On Thu, Jun 30, 2022 at 9:39 AM sebb <se...@gmail.com> wrote:
> >
> > > I've started copying bits of the C code to create a standalone tool to
> > > load the dll and report some info.
> > >
> > > No idea why yet, but it behaves differently from basically the same
> > > code when run via JNI.
> > >
> > > For example, the standalone code takes note of LD_LIBRARY_PATH when
> > > loading the dll, whereas Java apparently does not.
> > >
> > > On Thu, 30 Jun 2022 at 13:22, Alex Remily <al...@gmail.com>
> wrote:
> > > >
> > > > I also get the "Error looking up function 'ENGINE_load_rdrand'" when
> I
> > > run
> > > > the Jna test class from the command line. Interesting. dlopen
> finds the
> > > > LibreSSL when run from the CLI.
> > > >
> > > > On Thu, Jun 30, 2022 at 8:02 AM sebb <se...@gmail.com> wrote:
> > > >
> > > > > That looks fine, however I don't see the same, and nor does GH.
> > > > >
> > > > > What version of macOS are you using?
> > > > >
> > > > > On Thu, 30 Jun 2022 at 12:52, Gary Gregory <garydgregory@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > > With the 1.1.0 tagm I get:
> > > > > >
> > > > > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> > > > > > target/classes org.apache.commons.crypto.Crypto
> > > > > > Apache Commons Crypto 1.1.0
> > > > > > Native code loaded OK: 1.1.0
> > > > > > Native name: Apache Commons Crypto
> > > > > > Native built: Jun 30 2022
> > > > > > OpenSSL library loaded OK, version: 0x1010107f
> > > > > > OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> > > > > > Random instance created OK:
> > > > > > org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> > > > > > Cipher AES/CTR/NoPadding instance created OK:
> > > > > > org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> > > > > > Additional OpenSSL_version(n) details:
> > > > > > 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> > > > > > -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> > > > > > -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5
> -DOPENSSL_BN_ASM_GF2m
> > > > > > -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> > > > > > -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> > > > > > -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> > > > > > 2: built on: Tue Apr 21 13:30:00 2020 UTC
> > > > > > 3: platform: darwin64-x86_64-cc
> > > > > > 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> > > > > > 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1
> > > /1.1.1g/lib/engines-1.1"
> > > > > >
> > > > > > and
> > > > > >
> > > > > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q
> > > exec:java
> > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > isEnabled(): true
> > > > > > initialisationError(): null
> > > > > >
> > > > > > Gary
> > > > > >
> > > > > > On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> > > > > > >
> > > > > > > As previously noted, please try:
> > > > > > > java -cp target/classes org.apache.commons.crypto.Crypto
> > > > > > > and
> > > > > > > mvn -q exec:java
> > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > >
> > > > > > > See the GH run I linked yesterday:
> > > > > > >
> https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > >
> > > > > > > I just noticed also that 'openssl version' shows 1.1.1p but the
> > > Crypto
> > > > > > > class shows
> > > > > > > OpenSSL library info: LibreSSL 2.2.9 (on current code; the
> 1.11.0
> > > code
> > > > > crashes)
> > > > > > >
> > > > > > > Something strange is going on; it looks like dlopen is finding
> a
> > > > > > > different version of the dynamic library from the default CLI.
> > > > > > > I have the same issue on my mac.
> > > > > > >
> > > > > > > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <
> garydgregory@gmail.com
> > > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on
> macOS
> > > and
> > > > > ran
> > > > > > > > 'mvn clean verify' and everything built just fine.
> > > > > > > >
> > > > > > > > The Maven ant-run output was:
> > > > > > > >
> > > > > > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @
> commons-crypto
> > > ---
> > > > > > > > [INFO] Executing tasks
> > > > > > > >
> > > > > > > > make:
> > > > > > > > [exec] "/usr/local/Cellar/openjdk@8
> /1.8.0+322/bin/javah"
> > > -force
> > > > > > > > -classpath target/classes -o
> > > > > > > >
> > > > >
> > >
> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > > > > > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > > -I/usr/local/include
> > > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > > > >
> > > > >
> > >
> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > > > > > > -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > > > > [exec] "/usr/local/Cellar/openjdk@8
> /1.8.0+322/bin/javah"
> > > -force
> > > > > > > > -classpath target/classes -o
> > > > > > > >
> > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > > > > > > org.apache.commons.crypto.cipher.OpenSslNative
> > > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > > -I/usr/local/include
> > > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > > > >
> src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c
> > > -o
> > > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > > > > [exec] "/usr/local/Cellar/openjdk@8
> /1.8.0+322/bin/javah"
> > > -force
> > > > > > > > -classpath target/classes -o
> > > > > > > >
> target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > > > > > > org.apache.commons.crypto.OpenSslInfoNative
> > > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > > -I/usr/local/include
> > > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > > > > > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > > > > > >
> src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > > -I/usr/local/include
> > > > > > > > -I/usr/local/opt/openssl/include -Ilib/include
> -I/usr/include
> > > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > > > > > >
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > > >
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > > -dynamiclib
> > > > > > > > -L/usr/local/lib
> > > > > > > > [exec] strip -x
> > > > > > > >
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > > > [exec] cp
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > > >
> > > > >
> > >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > > > > [exec] cp
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > > >
> > > > >
> > >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > > > >
> > > > > > > > Gary
> > > > > > > >
> > > > > > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <
> > > garydgregory@gmail.com>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > I agree with Alex.
> > > > > > > > >
> > > > > > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice,
> simple,
> > > and
> > > > > > > > > tight. Last time I checked I had an antique version of
> > > LibreSSL on
> > > > > my
> > > > > > > > > mac years ago, I just installed OpenSSL and never looked
> back.
> > > > > > > > >
> > > > > > > > > Gary
> > > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <
> > > alex.remily@gmail.com>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > <The question is whether to do anything about the
> previously
> > > > > undetected
> > > > > > > > > > issues with JNA on macOS and Windows, and if so, whether
> this
> > > > > needs to
> > > > > > > > > > be done for the current or a later release. If we don't
> fix
> > > this
> > > > > > > > > > release, obviously it needs some mention in the release
> > > notes.>
> > > > > > > > > >
> > > > > > > > > > I wouldn't characterize the issues running against
> LibreSSL
> > > as
> > > > > > > > > > "undetected". I also don't see this as an issue with
> Mac or
> > > > > Windows, but
> > > > > > > > > > with LibreSSL. Install any supported OpenSSL library on
> any
> > > > > supported
> > > > > > > > > > architecture (to include Mac and Windows) and all commons
> > > crypto
> > > > > > > > > > functionality is available. I first encountered the rand
> > > issue
> > > > > you
> > > > > > > > > > describe years ago when I was becoming familiar with
> commons
> > > > > crypto. I did
> > > > > > > > > > a little research, discovered that I was running LibreSSL
> > > (and
> > > > > an old
> > > > > > > > > > version at that), installed a supported version of
> OpenSSL
> > > and
> > > > > forgot
> > > > > > > > > > all about it until this thread. I don't think it's
> > > unreasonable
> > > > > to expect
> > > > > > > > > > users to install a supported version of OpenSSL on a
> > > supported
> > > > > architecture
> > > > > > > > > > as a precondition of using commons crypto. I think it
> could
> > > > > become
> > > > > > > > > > cumbersome to try and support every vendor default *SSL
> > > > > install. That
> > > > > > > > > > said, I don't have a problem committing this particular
> > > "fix" to
> > > > > the
> > > > > > > > > > baseline, particularly since you have already done the
> > > work. I
> > > > > just don't
> > > > > > > > > > think that the project should obligate itself to do so or
> > > > > advertise
> > > > > > > > > > LibreSSL (or any other non-OpenSSL branch) support as
> such.
> > > I'm
> > > > > advocating
> > > > > > > > > > a "use at your own risk" approach to anything but OpenSSL
> > > > > proper. I agree
> > > > > > > > > > that we should update the documentation to reflect
> whatever
> > > we
> > > > > move forward
> > > > > > > > > > with.
> > > > > > > > > >
> > > > > > > > > > Anyway, that's my $0.02 worth.
> > > > > > > > > >
> > > > > > > > > > Alex
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com>
> > > wrote:
> > > > > > > > > >
> > > > > > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <
> > > > > garydgregory@gmail.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > We cannot remove support for Windows and macOS,
> that's
> > > silly.
> > > > > > > > > > >
> > > > > > > > > > > AFAICT that means we must support the different set of
> > > > > function names
> > > > > > > > > > > in LibreSSL.
> > > > > > > > > > > Note that we only currently use a small proportion of
> them.
> > > > > > > > > > >
> > > > > > > > > > > > I have not followed all the branches and commits, so
> I'm
> > > not
> > > > > sure what
> > > > > > > > > > > the
> > > > > > > > > > > > current problems are, but I know I was able to
> release
> > > all
> > > > > OSs last go
> > > > > > > > > > > > around. I don't see why we need to rip out anything
> as
> > > > > fundamental.
> > > > > > > > > > >
> > > > > > > > > > > Well, I have tried running the Crypto and OpenSslJna
> main
> > > > > classes on
> > > > > > > > > > > macOS and Windows, and they both fail with the 1.1.0
> > > release.
> > > > > > > > > > >
> > > > > > > > > > > With current master, Crypto succeeds, but OpenSslJna
> fails
> > > to
> > > > > find
> > > > > > > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > > > > > > (The job step succeeds, because the code catches the
> > > exception)
> > > > > > > > > > >
> > > > > > > > > > > It looks like the test which would have exposed at
> least
> > > one
> > > > > of the
> > > > > > > > > > > issues was never enabled because of a failures on
> macOS;
> > > this
> > > > > hid the
> > > > > > > > > > > same problem on Windows.
> > > > > > > > > > >
> > > > > > > > > > > I am not suggesting we rip out anything at this point.
> > > > > > > > > > >
> > > > > > > > > > > The question is whether to do anything about the
> previously
> > > > > undetected
> > > > > > > > > > > issues with JNA on macOS and Windows, and if so,
> whether
> > > this
> > > > > needs to
> > > > > > > > > > > be done for the current or a later release. If we
> don't fix
> > > > > this
> > > > > > > > > > > release, obviously it needs some mention in the release
> > > notes.
> > > > > > > > > > >
> > > > > > > > > > > Sebb
> > > > > > > > > > >
> > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > > > > > > Gary
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com>
> > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> > > > > alex.remily@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I agree with Gary that we just don't support
> > > LibreSSL.
> > > > > To my
> > > > > > > > > > > knowledge
> > > > > > > > > > > > > > we've never advertised LibreSSL support, so I
> don't
> > > see
> > > > > it as an
> > > > > > > > > > > issue.
> > > > > > > > > > > > >
> > > > > > > > > > > > > In that case AFAICT we will have to drop *all*
> support
> > > for
> > > > > macOS and
> > > > > > > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Note that adding support for LibreSSL was much
> easier
> > > for
> > > > > JNI, as it
> > > > > > > > > > > > > uses far fewer methods.
> > > > > > > > > > > > > Rather than checking the version, I changed the
> code to
> > > > > try OpenSSL
> > > > > > > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > > > > > > That happens to work for 1.0.x and 1.1.x and
> LibreSSL.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If you want to try it out, compare 16345bc (old)
> with
> > > > > 3ee3f65 (new)
> > > > > > > > > > > > > macOS fails on 16345bc because it now uses LibreSSL
> > > which
> > > > > has a
> > > > > > > > > > > > > different mix of names.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think it's vital we support JNI as far as
> possible
> > > (and
> > > > > the code
> > > > > > > > > > > > > already does with commit 3ee3f65).
> > > > > > > > > > > > >
> > > > > > > > > > > > > However JNA is more of a backstop, so the fact
> that it
> > > has
> > > > > stopped
> > > > > > > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > > > > > > >
> > > > > > > > > > > > > But I don't think we can say we are not supporting
> > > > > LibreSSL at all.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <
> > > sebbaz@gmail.com>
> > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> > > > > garydgregory@gmail.com
> > > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The simple solution is to keep doing what we
> do
> > > now:
> > > > > only support
> > > > > > > > > > > > > OpenSSL
> > > > > > > > > > > > > > > > and not whatever Apple does with LibreSSL
> which
> > > may
> > > > > or may not
> > > > > > > > > > > be up
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > date.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I think this also affects Windows.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I don't have Windows box at present, but I
> have
> > > seen
> > > > > this on GH
> > > > > > > > > > > > > Actions.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > If you have a WIndows build, perhaps you could
> try
> > > > > these tests:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > > > > > >
> > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The first one should show the SSL details; on
> GH
> > > the
> > > > > output
> > > > > > > > > > > includes:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The second test crashes with:
> > > > > > > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking
> up
> > > > > function
> > > > > > > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure
> could
> > > > > not be found.
> > > > > > > > > > > > > > > isEnabled(): false
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > initialisationError():
> > > java.lang.UnsatisfiedLinkError:
> > > > > Error
> > > > > > > > > > > looking
> > > > > > > > > > > > > > > up function 'ENGINE_load_rdrand': The specified
> > > > > procedure could
> > > > > > > > > > > not be
> > > > > > > > > > > > > > > found.
> > > > > > > > > > > > > > > at
> com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > > > > > > ...
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > It would certainly be easier to ignore the
> problem
> > > for
> > > > > now.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Gary
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <
> > > sebbaz@gmail.com>
> > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > It looks like macOS 10.5+ and Windows
> (latest)
> > > use
> > > > > LibreSSL by
> > > > > > > > > > > > > default
> > > > > > > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > The LibreSSL API does not have the same
> > > functions
> > > > > as either
> > > > > > > > > > > 1.0.2
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > 1.1.1, so needs its own JNA class. In
> > > particular
> > > > > it looks like
> > > > > > > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> > > > > OpenSSL_version_num;
> > > > > > > > > > > > > 1.0.2
> > > > > > > > > > > > > > > > > has the former only, and 1.1.1 has the
> latter
> > > only.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This makes it hard to support JNA with the
> > > current
> > > > > design.
> > > > > > > > > > > > > > > > > It would require another
> > > OpenSsl<version>NativeJna
> > > > > class, and
> > > > > > > > > > > the
> > > > > > > > > > > > > > > > > parent class OpenSslNativeJna would need
> to use
> > > > > yet another
> > > > > > > > > > > > > condition
> > > > > > > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Seems to me it would be better to use
> something
> > > > > like a set of
> > > > > > > > > > > > > > > > > singleton classes that implement the
> required
> > > > > methods. The
> > > > > > > > > > > > > appropriate
> > > > > > > > > > > > > > > > > one can then be initialised and used by
> > > > > OpenSslNativeJna to
> > > > > > > > > > > field
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > actual methods. i.e. replace the
> conditional
> > > logic
> > > > > with a
> > > > > > > > > > > static
> > > > > > > > > > > > > > > > > reference to the correct API interface
> > > instance.
> > > > > This should
> > > > > > > > > > > only
> > > > > > > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Sebb
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@commons.apache.org
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@commons.apache.org
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > > > dev-help@commons.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@commons.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Fri, 1 Jul 2022 at 16:33, Alex Remily <al...@gmail.com> wrote:
>
> <BTW I'm working on code to show the actual path that was loaded; this will
> be shown in the Crypto main output.>
Now committed; seems to work on all 3 main OSes.
> I like the approach. I wrote most of the version code that you're
> touching, and your refactoring is more elegant.
Thanks, but I'm not sure I have done that much.
> I hadn't written much C
> since school, so I kind of hacked my way through it. It works, but
> definitely room for improvement.
Likewise, it has been a while since I used C (probably on this project).
> At first glance it appears that your
> approach would be easier to build on when we start on OpenSSL 3.0
> support, which is an added benefit.
If you are referring to the fallback name strategy that just seemed
like a simple way to avoid having to check versions.
But it only applies to the JNI code, and it only works where there is
an equivalent function with a different name.
That happens to be the case here currently.
We shall have to see if it works for OpenSSL 3.0.
I don't think a similar approach is possible for JNA because of the
way the binding is done.
> Alex
>
> On Fri, Jul 1, 2022 at 5:52 AM sebb <se...@gmail.com> wrote:
>
> > On Fri, 1 Jul 2022 at 04:16, Bernd Eckenfels <ec...@zusammenkunft.net>
> > wrote:
> > >
> > > The Java.library.path controls the Java side loadLibrary() but that
> > worked since it loaded the commons-crypto JNI lib.
> > >
> > > The dlopen In that shim searches various places, including PATH.
> >
> > According to dlopen(3) it does not search PATH, instead it uses
> > LD_LIBRARY_PATH, and some DYLD_* paths.
> >
> > > If you can’t put the OpenSSL variant first, then it might instead help
> > to define the path with a absolute name instead (maybe make it configurable
> > with env or system property?)
> >
> > It's not possible in general to put the required version first, so I
> > agree an (optional) override is needed.
> > I don't think it would do any harm.
> >
> > BTW I'm working on code to show the actual path that was loaded; this
> > will be shown in the Crypto main output.
> >
> > >
> > https://github.com/apache/commons-crypto/blob/8539d5f09e483d8a01635df9df062e0a4972a71d/src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c#L74
> > >
> > > It might also help to remove the global flag to avoid re-exports?
> >
> > Not sure that would make a difference here.
> >
> > > Whats the lib filename constant in above line in your case?
> > >
> > > Gruss
> > > Bernd
> > > --
> > > http://bernd.eckenfels.net
> > > ________________________________
> > > Von: Alex Remily <al...@gmail.com>
> > > Gesendet: Friday, July 1, 2022 4:26:33 AM
> > > An: Commons Developers List <de...@commons.apache.org>
> > > Betreff: Re: [CRYPTO] Problem with JNA on macOS and Windows
> > >
> > > <Did you try with "java.library.path">
> > >
> > > Yup. Still loads LibreSSL.
> > >
> > > Alex@Alexs-MacBook-Pro commons-crypto % java
> > > -Djava.library.path=/usr/local/opt/openssl/lib -cp target/classes
> > > org.apache.commons.crypto.Crypto
> > >
> > > Apache Commons Crypto 1.1.1-SNAPSHOT
> > >
> > > Native code loaded OK: 1.1.1-SNAPSHOT
> > >
> > > Native name: Apache Commons Crypto
> > >
> > > Native built: Jun 30 2022
> > >
> > > OpenSSL library loaded OK, version: 0x20000000
> > >
> > > OpenSSL library info: LibreSSL 2.2.9
> > >
> > > Random instance created OK:
> > > org.apache.commons.crypto.random.OpenSslCryptoRandom@6442b0a6
> > >
> > > Cipher AES/CTR/NoPadding instance created OK:
> > > org.apache.commons.crypto.cipher.OpenSslCipher@21bcffb5
> > >
> > > Additional OpenSSL_version(n) details:
> > >
> > > 1: not available
> > >
> > > 2: compiler: information not available
> > >
> > > 3: built on: date not available
> > >
> > > 4: platform: information not available
> > >
> > > 5: OPENSSLDIR: "/private/etc/ssl"
> > >
> > > Alex@Alexs-MacBook-Pro commons-crypto %
> > >
> > > On Thu, Jun 30, 2022 at 6:16 PM Gilles Sadowski <gi...@gmail.com>
> > > wrote:
> > >
> > > > Le jeu. 30 juin 2022 à 23:35, sebb <se...@gmail.com> a écrit :
> > > > >
> > > > > On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com>
> > wrote:
> > > > > >
> > > > > > <For example, the standalone code takes note of LD_LIBRARY_PATH
> > when
> > > > > > loading the dll, whereas Java apparently does not.>
> > > > > >
> > > > > > I experience the same behavior. What's more interesting is that
> > when
> > > > I run
> > > > > > the main function from Eclipse as a Run Configuration with the
> > > > > > LD_LIBRARY_PATH set as an Environment variable appended to the
> > native
> > > > > > environment, it runs as expected. As of yet I haven't found a way
> > to
> > > > get
> > > > > > the java CLI to recognize the LD_LIBRARY_PATH environment variable,
> > > > even
> > > > > > though it echos out correctly when queried. Strange.
> > > >
> > > > Did you try with "java.library.path" (cf. [1])?
> > > >
> > > > Gilles
> > > >
> > > > [1]
> > > >
> > https://stackoverflow.com/questions/27945268/difference-between-using-java-library-path-and-ld-library-path
> > > >
> > > > > [...]
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
<BTW I'm working on code to show the actual path that was loaded; this will
be shown in the Crypto main output.>
I like the approach. I wrote most of the version code that you're
touching, and your refactoring is more elegant. I hadn't written much C
since school, so I kind of hacked my way through it. It works, but
definitely room for improvement. At first glance it appears that your
approach would be easier to build on when we start on OpenSSL 3.0
support, which is an added benefit.
Alex
On Fri, Jul 1, 2022 at 5:52 AM sebb <se...@gmail.com> wrote:
> On Fri, 1 Jul 2022 at 04:16, Bernd Eckenfels <ec...@zusammenkunft.net>
> wrote:
> >
> > The Java.library.path controls the Java side loadLibrary() but that
> worked since it loaded the commons-crypto JNI lib.
> >
> > The dlopen In that shim searches various places, including PATH.
>
> According to dlopen(3) it does not search PATH, instead it uses
> LD_LIBRARY_PATH, and some DYLD_* paths.
>
> > If you can’t put the OpenSSL variant first, then it might instead help
> to define the path with a absolute name instead (maybe make it configurable
> with env or system property?)
>
> It's not possible in general to put the required version first, so I
> agree an (optional) override is needed.
> I don't think it would do any harm.
>
> BTW I'm working on code to show the actual path that was loaded; this
> will be shown in the Crypto main output.
>
> >
> https://github.com/apache/commons-crypto/blob/8539d5f09e483d8a01635df9df062e0a4972a71d/src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c#L74
> >
> > It might also help to remove the global flag to avoid re-exports?
>
> Not sure that would make a difference here.
>
> > Whats the lib filename constant in above line in your case?
> >
> > Gruss
> > Bernd
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Alex Remily <al...@gmail.com>
> > Gesendet: Friday, July 1, 2022 4:26:33 AM
> > An: Commons Developers List <de...@commons.apache.org>
> > Betreff: Re: [CRYPTO] Problem with JNA on macOS and Windows
> >
> > <Did you try with "java.library.path">
> >
> > Yup. Still loads LibreSSL.
> >
> > Alex@Alexs-MacBook-Pro commons-crypto % java
> > -Djava.library.path=/usr/local/opt/openssl/lib -cp target/classes
> > org.apache.commons.crypto.Crypto
> >
> > Apache Commons Crypto 1.1.1-SNAPSHOT
> >
> > Native code loaded OK: 1.1.1-SNAPSHOT
> >
> > Native name: Apache Commons Crypto
> >
> > Native built: Jun 30 2022
> >
> > OpenSSL library loaded OK, version: 0x20000000
> >
> > OpenSSL library info: LibreSSL 2.2.9
> >
> > Random instance created OK:
> > org.apache.commons.crypto.random.OpenSslCryptoRandom@6442b0a6
> >
> > Cipher AES/CTR/NoPadding instance created OK:
> > org.apache.commons.crypto.cipher.OpenSslCipher@21bcffb5
> >
> > Additional OpenSSL_version(n) details:
> >
> > 1: not available
> >
> > 2: compiler: information not available
> >
> > 3: built on: date not available
> >
> > 4: platform: information not available
> >
> > 5: OPENSSLDIR: "/private/etc/ssl"
> >
> > Alex@Alexs-MacBook-Pro commons-crypto %
> >
> > On Thu, Jun 30, 2022 at 6:16 PM Gilles Sadowski <gi...@gmail.com>
> > wrote:
> >
> > > Le jeu. 30 juin 2022 à 23:35, sebb <se...@gmail.com> a écrit :
> > > >
> > > > On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com>
> wrote:
> > > > >
> > > > > <For example, the standalone code takes note of LD_LIBRARY_PATH
> when
> > > > > loading the dll, whereas Java apparently does not.>
> > > > >
> > > > > I experience the same behavior. What's more interesting is that
> when
> > > I run
> > > > > the main function from Eclipse as a Run Configuration with the
> > > > > LD_LIBRARY_PATH set as an Environment variable appended to the
> native
> > > > > environment, it runs as expected. As of yet I haven't found a way
> to
> > > get
> > > > > the java CLI to recognize the LD_LIBRARY_PATH environment variable,
> > > even
> > > > > though it echos out correctly when queried. Strange.
> > >
> > > Did you try with "java.library.path" (cf. [1])?
> > >
> > > Gilles
> > >
> > > [1]
> > >
> https://stackoverflow.com/questions/27945268/difference-between-using-java-library-path-and-ld-library-path
> > >
> > > > [...]
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Fri, 1 Jul 2022 at 04:16, Bernd Eckenfels <ec...@zusammenkunft.net> wrote:
>
> The Java.library.path controls the Java side loadLibrary() but that worked since it loaded the commons-crypto JNI lib.
>
> The dlopen In that shim searches various places, including PATH.
According to dlopen(3) it does not search PATH, instead it uses
LD_LIBRARY_PATH, and some DYLD_* paths.
> If you can’t put the OpenSSL variant first, then it might instead help to define the path with a absolute name instead (maybe make it configurable with env or system property?)
It's not possible in general to put the required version first, so I
agree an (optional) override is needed.
I don't think it would do any harm.
BTW I'm working on code to show the actual path that was loaded; this
will be shown in the Crypto main output.
> https://github.com/apache/commons-crypto/blob/8539d5f09e483d8a01635df9df062e0a4972a71d/src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c#L74
>
> It might also help to remove the global flag to avoid re-exports?
Not sure that would make a difference here.
> Whats the lib filename constant in above line in your case?
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> ________________________________
> Von: Alex Remily <al...@gmail.com>
> Gesendet: Friday, July 1, 2022 4:26:33 AM
> An: Commons Developers List <de...@commons.apache.org>
> Betreff: Re: [CRYPTO] Problem with JNA on macOS and Windows
>
> <Did you try with "java.library.path">
>
> Yup. Still loads LibreSSL.
>
> Alex@Alexs-MacBook-Pro commons-crypto % java
> -Djava.library.path=/usr/local/opt/openssl/lib -cp target/classes
> org.apache.commons.crypto.Crypto
>
> Apache Commons Crypto 1.1.1-SNAPSHOT
>
> Native code loaded OK: 1.1.1-SNAPSHOT
>
> Native name: Apache Commons Crypto
>
> Native built: Jun 30 2022
>
> OpenSSL library loaded OK, version: 0x20000000
>
> OpenSSL library info: LibreSSL 2.2.9
>
> Random instance created OK:
> org.apache.commons.crypto.random.OpenSslCryptoRandom@6442b0a6
>
> Cipher AES/CTR/NoPadding instance created OK:
> org.apache.commons.crypto.cipher.OpenSslCipher@21bcffb5
>
> Additional OpenSSL_version(n) details:
>
> 1: not available
>
> 2: compiler: information not available
>
> 3: built on: date not available
>
> 4: platform: information not available
>
> 5: OPENSSLDIR: "/private/etc/ssl"
>
> Alex@Alexs-MacBook-Pro commons-crypto %
>
> On Thu, Jun 30, 2022 at 6:16 PM Gilles Sadowski <gi...@gmail.com>
> wrote:
>
> > Le jeu. 30 juin 2022 à 23:35, sebb <se...@gmail.com> a écrit :
> > >
> > > On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com> wrote:
> > > >
> > > > <For example, the standalone code takes note of LD_LIBRARY_PATH when
> > > > loading the dll, whereas Java apparently does not.>
> > > >
> > > > I experience the same behavior. What's more interesting is that when
> > I run
> > > > the main function from Eclipse as a Run Configuration with the
> > > > LD_LIBRARY_PATH set as an Environment variable appended to the native
> > > > environment, it runs as expected. As of yet I haven't found a way to
> > get
> > > > the java CLI to recognize the LD_LIBRARY_PATH environment variable,
> > even
> > > > though it echos out correctly when queried. Strange.
> >
> > Did you try with "java.library.path" (cf. [1])?
> >
> > Gilles
> >
> > [1]
> > https://stackoverflow.com/questions/27945268/difference-between-using-java-library-path-and-ld-library-path
> >
> > > [...]
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Bernd Eckenfels <ec...@zusammenkunft.net>.
The Java.library.path controls the Java side loadLibrary() but that worked since it loaded the commons-crypto JNI lib.
The dlopen In that shim searches various places, including PATH. If you can’t put the OpenSSL variant first, then it might instead help to define the path with a absolute name instead (maybe make it configurable with env or system property?)
https://github.com/apache/commons-crypto/blob/8539d5f09e483d8a01635df9df062e0a4972a71d/src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c#L74
It might also help to remove the global flag to avoid re-exports?
Whats the lib filename constant in above line in your case?
Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
Von: Alex Remily <al...@gmail.com>
Gesendet: Friday, July 1, 2022 4:26:33 AM
An: Commons Developers List <de...@commons.apache.org>
Betreff: Re: [CRYPTO] Problem with JNA on macOS and Windows
<Did you try with "java.library.path">
Yup. Still loads LibreSSL.
Alex@Alexs-MacBook-Pro commons-crypto % java
-Djava.library.path=/usr/local/opt/openssl/lib -cp target/classes
org.apache.commons.crypto.Crypto
Apache Commons Crypto 1.1.1-SNAPSHOT
Native code loaded OK: 1.1.1-SNAPSHOT
Native name: Apache Commons Crypto
Native built: Jun 30 2022
OpenSSL library loaded OK, version: 0x20000000
OpenSSL library info: LibreSSL 2.2.9
Random instance created OK:
org.apache.commons.crypto.random.OpenSslCryptoRandom@6442b0a6
Cipher AES/CTR/NoPadding instance created OK:
org.apache.commons.crypto.cipher.OpenSslCipher@21bcffb5
Additional OpenSSL_version(n) details:
1: not available
2: compiler: information not available
3: built on: date not available
4: platform: information not available
5: OPENSSLDIR: "/private/etc/ssl"
Alex@Alexs-MacBook-Pro commons-crypto %
On Thu, Jun 30, 2022 at 6:16 PM Gilles Sadowski <gi...@gmail.com>
wrote:
> Le jeu. 30 juin 2022 à 23:35, sebb <se...@gmail.com> a écrit :
> >
> > On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com> wrote:
> > >
> > > <For example, the standalone code takes note of LD_LIBRARY_PATH when
> > > loading the dll, whereas Java apparently does not.>
> > >
> > > I experience the same behavior. What's more interesting is that when
> I run
> > > the main function from Eclipse as a Run Configuration with the
> > > LD_LIBRARY_PATH set as an Environment variable appended to the native
> > > environment, it runs as expected. As of yet I haven't found a way to
> get
> > > the java CLI to recognize the LD_LIBRARY_PATH environment variable,
> even
> > > though it echos out correctly when queried. Strange.
>
> Did you try with "java.library.path" (cf. [1])?
>
> Gilles
>
> [1]
> https://stackoverflow.com/questions/27945268/difference-between-using-java-library-path-and-ld-library-path
>
> > [...]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
<Did you try with "java.library.path">
Yup. Still loads LibreSSL.
Alex@Alexs-MacBook-Pro commons-crypto % java
-Djava.library.path=/usr/local/opt/openssl/lib -cp target/classes
org.apache.commons.crypto.Crypto
Apache Commons Crypto 1.1.1-SNAPSHOT
Native code loaded OK: 1.1.1-SNAPSHOT
Native name: Apache Commons Crypto
Native built: Jun 30 2022
OpenSSL library loaded OK, version: 0x20000000
OpenSSL library info: LibreSSL 2.2.9
Random instance created OK:
org.apache.commons.crypto.random.OpenSslCryptoRandom@6442b0a6
Cipher AES/CTR/NoPadding instance created OK:
org.apache.commons.crypto.cipher.OpenSslCipher@21bcffb5
Additional OpenSSL_version(n) details:
1: not available
2: compiler: information not available
3: built on: date not available
4: platform: information not available
5: OPENSSLDIR: "/private/etc/ssl"
Alex@Alexs-MacBook-Pro commons-crypto %
On Thu, Jun 30, 2022 at 6:16 PM Gilles Sadowski <gi...@gmail.com>
wrote:
> Le jeu. 30 juin 2022 à 23:35, sebb <se...@gmail.com> a écrit :
> >
> > On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com> wrote:
> > >
> > > <For example, the standalone code takes note of LD_LIBRARY_PATH when
> > > loading the dll, whereas Java apparently does not.>
> > >
> > > I experience the same behavior. What's more interesting is that when
> I run
> > > the main function from Eclipse as a Run Configuration with the
> > > LD_LIBRARY_PATH set as an Environment variable appended to the native
> > > environment, it runs as expected. As of yet I haven't found a way to
> get
> > > the java CLI to recognize the LD_LIBRARY_PATH environment variable,
> even
> > > though it echos out correctly when queried. Strange.
>
> Did you try with "java.library.path" (cf. [1])?
>
> Gilles
>
> [1]
> https://stackoverflow.com/questions/27945268/difference-between-using-java-library-path-and-ld-library-path
>
> > [...]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gilles Sadowski <gi...@gmail.com>.
Le jeu. 30 juin 2022 à 23:35, sebb <se...@gmail.com> a écrit :
>
> On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com> wrote:
> >
> > <For example, the standalone code takes note of LD_LIBRARY_PATH when
> > loading the dll, whereas Java apparently does not.>
> >
> > I experience the same behavior. What's more interesting is that when I run
> > the main function from Eclipse as a Run Configuration with the
> > LD_LIBRARY_PATH set as an Environment variable appended to the native
> > environment, it runs as expected. As of yet I haven't found a way to get
> > the java CLI to recognize the LD_LIBRARY_PATH environment variable, even
> > though it echos out correctly when queried. Strange.
Did you try with "java.library.path" (cf. [1])?
Gilles
[1] https://stackoverflow.com/questions/27945268/difference-between-using-java-library-path-and-ld-library-path
> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Thu, 30 Jun 2022 at 16:21, Alex Remily <al...@gmail.com> wrote:
>
> <For example, the standalone code takes note of LD_LIBRARY_PATH when
> loading the dll, whereas Java apparently does not.>
>
> I experience the same behavior. What's more interesting is that when I run
> the main function from Eclipse as a Run Configuration with the
> LD_LIBRARY_PATH set as an Environment variable appended to the native
> environment, it runs as expected. As of yet I haven't found a way to get
> the java CLI to recognize the LD_LIBRARY_PATH environment variable, even
> though it echos out correctly when queried. Strange.
If you use getenv within the JNI C code, it does not see any of the
library-related environment variables.
It looks like DYLD_* is dropped, along with LD_LIBRARY_PATH.
There is some strange behaviour with export as well.
Try the following:
export LD_LIBRART_PATH=/usr/lib
export LD_LIBRARY_PATH=/usr/lib
env | grep LIBRAR
I get:
LD_LIBRART_PATH=/usr/lib
Similarly with DYLD_*
I think we may need to allow the user to provide the full path name to
the crypto library.
> On Thu, Jun 30, 2022 at 9:39 AM sebb <se...@gmail.com> wrote:
>
> > I've started copying bits of the C code to create a standalone tool to
> > load the dll and report some info.
> >
> > No idea why yet, but it behaves differently from basically the same
> > code when run via JNI.
> >
> > For example, the standalone code takes note of LD_LIBRARY_PATH when
> > loading the dll, whereas Java apparently does not.
> >
> > On Thu, 30 Jun 2022 at 13:22, Alex Remily <al...@gmail.com> wrote:
> > >
> > > I also get the "Error looking up function 'ENGINE_load_rdrand'" when I
> > run
> > > the Jna test class from the command line. Interesting. dlopen finds the
> > > LibreSSL when run from the CLI.
> > >
> > > On Thu, Jun 30, 2022 at 8:02 AM sebb <se...@gmail.com> wrote:
> > >
> > > > That looks fine, however I don't see the same, and nor does GH.
> > > >
> > > > What version of macOS are you using?
> > > >
> > > > On Thu, 30 Jun 2022 at 12:52, Gary Gregory <ga...@gmail.com>
> > wrote:
> > > > >
> > > > > With the 1.1.0 tagm I get:
> > > > >
> > > > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> > > > > target/classes org.apache.commons.crypto.Crypto
> > > > > Apache Commons Crypto 1.1.0
> > > > > Native code loaded OK: 1.1.0
> > > > > Native name: Apache Commons Crypto
> > > > > Native built: Jun 30 2022
> > > > > OpenSSL library loaded OK, version: 0x1010107f
> > > > > OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> > > > > Random instance created OK:
> > > > > org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> > > > > Cipher AES/CTR/NoPadding instance created OK:
> > > > > org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> > > > > Additional OpenSSL_version(n) details:
> > > > > 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> > > > > -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> > > > > -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
> > > > > -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> > > > > -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> > > > > -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> > > > > 2: built on: Tue Apr 21 13:30:00 2020 UTC
> > > > > 3: platform: darwin64-x86_64-cc
> > > > > 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> > > > > 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1
> > /1.1.1g/lib/engines-1.1"
> > > > >
> > > > > and
> > > > >
> > > > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q
> > exec:java
> > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > isEnabled(): true
> > > > > initialisationError(): null
> > > > >
> > > > > Gary
> > > > >
> > > > > On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> > > > > >
> > > > > > As previously noted, please try:
> > > > > > java -cp target/classes org.apache.commons.crypto.Crypto
> > > > > > and
> > > > > > mvn -q exec:java
> > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > >
> > > > > > See the GH run I linked yesterday:
> > > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > >
> > > > > > I just noticed also that 'openssl version' shows 1.1.1p but the
> > Crypto
> > > > > > class shows
> > > > > > OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0
> > code
> > > > crashes)
> > > > > >
> > > > > > Something strange is going on; it looks like dlopen is finding a
> > > > > > different version of the dynamic library from the default CLI.
> > > > > > I have the same issue on my mac.
> > > > > >
> > > > > > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <garydgregory@gmail.com
> > >
> > > > wrote:
> > > > > > >
> > > > > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS
> > and
> > > > ran
> > > > > > > 'mvn clean verify' and everything built just fine.
> > > > > > >
> > > > > > > The Maven ant-run output was:
> > > > > > >
> > > > > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto
> > ---
> > > > > > > [INFO] Executing tasks
> > > > > > >
> > > > > > > make:
> > > > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah"
> > -force
> > > > > > > -classpath target/classes -o
> > > > > > >
> > > >
> > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > > > > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > -I/usr/local/include
> > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > > >
> > > >
> > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > > > > > -o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah"
> > -force
> > > > > > > -classpath target/classes -o
> > > > > > >
> > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > > > > > org.apache.commons.crypto.cipher.OpenSslNative
> > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > -I/usr/local/include
> > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c
> > -o
> > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah"
> > -force
> > > > > > > -classpath target/classes -o
> > > > > > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > > > > > org.apache.commons.crypto.OpenSslInfoNative
> > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > -I/usr/local/include
> > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > > > > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > > > > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> > -I/usr/local/include
> > > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > > -I"/usr/local/Cellar/openjdk@8
> > > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > >
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > -dynamiclib
> > > > > > > -L/usr/local/lib
> > > > > > > [exec] strip -x
> > > > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > > [exec] cp
> > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > >
> > > >
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > > > [exec] cp
> > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > >
> > > >
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > > >
> > > > > > > Gary
> > > > > > >
> > > > > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <
> > garydgregory@gmail.com>
> > > > wrote:
> > > > > > > >
> > > > > > > > I agree with Alex.
> > > > > > > >
> > > > > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple,
> > and
> > > > > > > > tight. Last time I checked I had an antique version of
> > LibreSSL on
> > > > my
> > > > > > > > mac years ago, I just installed OpenSSL and never looked back.
> > > > > > > >
> > > > > > > > Gary
> > > > > > > >
> > > > > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <
> > alex.remily@gmail.com>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > <The question is whether to do anything about the previously
> > > > undetected
> > > > > > > > > issues with JNA on macOS and Windows, and if so, whether this
> > > > needs to
> > > > > > > > > be done for the current or a later release. If we don't fix
> > this
> > > > > > > > > release, obviously it needs some mention in the release
> > notes.>
> > > > > > > > >
> > > > > > > > > I wouldn't characterize the issues running against LibreSSL
> > as
> > > > > > > > > "undetected". I also don't see this as an issue with Mac or
> > > > Windows, but
> > > > > > > > > with LibreSSL. Install any supported OpenSSL library on any
> > > > supported
> > > > > > > > > architecture (to include Mac and Windows) and all commons
> > crypto
> > > > > > > > > functionality is available. I first encountered the rand
> > issue
> > > > you
> > > > > > > > > describe years ago when I was becoming familiar with commons
> > > > crypto. I did
> > > > > > > > > a little research, discovered that I was running LibreSSL
> > (and
> > > > an old
> > > > > > > > > version at that), installed a supported version of OpenSSL
> > and
> > > > forgot
> > > > > > > > > all about it until this thread. I don't think it's
> > unreasonable
> > > > to expect
> > > > > > > > > users to install a supported version of OpenSSL on a
> > supported
> > > > architecture
> > > > > > > > > as a precondition of using commons crypto. I think it could
> > > > become
> > > > > > > > > cumbersome to try and support every vendor default *SSL
> > > > install. That
> > > > > > > > > said, I don't have a problem committing this particular
> > "fix" to
> > > > the
> > > > > > > > > baseline, particularly since you have already done the
> > work. I
> > > > just don't
> > > > > > > > > think that the project should obligate itself to do so or
> > > > advertise
> > > > > > > > > LibreSSL (or any other non-OpenSSL branch) support as such.
> > I'm
> > > > advocating
> > > > > > > > > a "use at your own risk" approach to anything but OpenSSL
> > > > proper. I agree
> > > > > > > > > that we should update the documentation to reflect whatever
> > we
> > > > move forward
> > > > > > > > > with.
> > > > > > > > >
> > > > > > > > > Anyway, that's my $0.02 worth.
> > > > > > > > >
> > > > > > > > > Alex
> > > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com>
> > wrote:
> > > > > > > > >
> > > > > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <
> > > > garydgregory@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > We cannot remove support for Windows and macOS, that's
> > silly.
> > > > > > > > > >
> > > > > > > > > > AFAICT that means we must support the different set of
> > > > function names
> > > > > > > > > > in LibreSSL.
> > > > > > > > > > Note that we only currently use a small proportion of them.
> > > > > > > > > >
> > > > > > > > > > > I have not followed all the branches and commits, so I'm
> > not
> > > > sure what
> > > > > > > > > > the
> > > > > > > > > > > current problems are, but I know I was able to release
> > all
> > > > OSs last go
> > > > > > > > > > > around. I don't see why we need to rip out anything as
> > > > fundamental.
> > > > > > > > > >
> > > > > > > > > > Well, I have tried running the Crypto and OpenSslJna main
> > > > classes on
> > > > > > > > > > macOS and Windows, and they both fail with the 1.1.0
> > release.
> > > > > > > > > >
> > > > > > > > > > With current master, Crypto succeeds, but OpenSslJna fails
> > to
> > > > find
> > > > > > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > > > > > (The job step succeeds, because the code catches the
> > exception)
> > > > > > > > > >
> > > > > > > > > > It looks like the test which would have exposed at least
> > one
> > > > of the
> > > > > > > > > > issues was never enabled because of a failures on macOS;
> > this
> > > > hid the
> > > > > > > > > > same problem on Windows.
> > > > > > > > > >
> > > > > > > > > > I am not suggesting we rip out anything at this point.
> > > > > > > > > >
> > > > > > > > > > The question is whether to do anything about the previously
> > > > undetected
> > > > > > > > > > issues with JNA on macOS and Windows, and if so, whether
> > this
> > > > needs to
> > > > > > > > > > be done for the current or a later release. If we don't fix
> > > > this
> > > > > > > > > > release, obviously it needs some mention in the release
> > notes.
> > > > > > > > > >
> > > > > > > > > > Sebb
> > > > > > > > > >
> > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > > > > > Gary
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com>
> > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> > > > alex.remily@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > I agree with Gary that we just don't support
> > LibreSSL.
> > > > To my
> > > > > > > > > > knowledge
> > > > > > > > > > > > > we've never advertised LibreSSL support, so I don't
> > see
> > > > it as an
> > > > > > > > > > issue.
> > > > > > > > > > > >
> > > > > > > > > > > > In that case AFAICT we will have to drop *all* support
> > for
> > > > macOS and
> > > > > > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > > > > > >
> > > > > > > > > > > > Note that adding support for LibreSSL was much easier
> > for
> > > > JNI, as it
> > > > > > > > > > > > uses far fewer methods.
> > > > > > > > > > > > Rather than checking the version, I changed the code to
> > > > try OpenSSL
> > > > > > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > > > > > >
> > > > > > > > > > > > If you want to try it out, compare 16345bc (old) with
> > > > 3ee3f65 (new)
> > > > > > > > > > > > macOS fails on 16345bc because it now uses LibreSSL
> > which
> > > > has a
> > > > > > > > > > > > different mix of names.
> > > > > > > > > > > >
> > > > > > > > > > > > I think it's vital we support JNI as far as possible
> > (and
> > > > the code
> > > > > > > > > > > > already does with commit 3ee3f65).
> > > > > > > > > > > >
> > > > > > > > > > > > However JNA is more of a backstop, so the fact that it
> > has
> > > > stopped
> > > > > > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > > > > > >
> > > > > > > > > > > > But I don't think we can say we are not supporting
> > > > LibreSSL at all.
> > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <
> > sebbaz@gmail.com>
> > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> > > > garydgregory@gmail.com
> > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The simple solution is to keep doing what we do
> > now:
> > > > only support
> > > > > > > > > > > > OpenSSL
> > > > > > > > > > > > > > > and not whatever Apple does with LibreSSL which
> > may
> > > > or may not
> > > > > > > > > > be up
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > date.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I think this also affects Windows.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I don't have Windows box at present, but I have
> > seen
> > > > this on GH
> > > > > > > > > > > > Actions.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > If you have a WIndows build, perhaps you could try
> > > > these tests:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > > > > >
> > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The first one should show the SSL details; on GH
> > the
> > > > output
> > > > > > > > > > includes:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The second test crashes with:
> > > > > > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up
> > > > function
> > > > > > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could
> > > > not be found.
> > > > > > > > > > > > > > isEnabled(): false
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > initialisationError():
> > java.lang.UnsatisfiedLinkError:
> > > > Error
> > > > > > > > > > looking
> > > > > > > > > > > > > > up function 'ENGINE_load_rdrand': The specified
> > > > procedure could
> > > > > > > > > > not be
> > > > > > > > > > > > > > found.
> > > > > > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > > > > > ...
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > It would certainly be easier to ignore the problem
> > for
> > > > now.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Gary
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <
> > sebbaz@gmail.com>
> > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest)
> > use
> > > > LibreSSL by
> > > > > > > > > > > > default
> > > > > > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The LibreSSL API does not have the same
> > functions
> > > > as either
> > > > > > > > > > 1.0.2
> > > > > > > > > > > > or
> > > > > > > > > > > > > > > > 1.1.1, so needs its own JNA class. In
> > particular
> > > > it looks like
> > > > > > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> > > > OpenSSL_version_num;
> > > > > > > > > > > > 1.0.2
> > > > > > > > > > > > > > > > has the former only, and 1.1.1 has the latter
> > only.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This makes it hard to support JNA with the
> > current
> > > > design.
> > > > > > > > > > > > > > > > It would require another
> > OpenSsl<version>NativeJna
> > > > class, and
> > > > > > > > > > the
> > > > > > > > > > > > > > > > parent class OpenSslNativeJna would need to use
> > > > yet another
> > > > > > > > > > > > condition
> > > > > > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Seems to me it would be better to use something
> > > > like a set of
> > > > > > > > > > > > > > > > singleton classes that implement the required
> > > > methods. The
> > > > > > > > > > > > appropriate
> > > > > > > > > > > > > > > > one can then be initialised and used by
> > > > OpenSslNativeJna to
> > > > > > > > > > field
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > actual methods. i.e. replace the conditional
> > logic
> > > > with a
> > > > > > > > > > static
> > > > > > > > > > > > > > > > reference to the correct API interface
> > instance.
> > > > This should
> > > > > > > > > > only
> > > > > > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Sebb
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@commons.apache.org
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@commons.apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > > > dev-help@commons.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > For additional commands, e-mail:
> > dev-help@commons.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
<For example, the standalone code takes note of LD_LIBRARY_PATH when
loading the dll, whereas Java apparently does not.>
I experience the same behavior. What's more interesting is that when I run
the main function from Eclipse as a Run Configuration with the
LD_LIBRARY_PATH set as an Environment variable appended to the native
environment, it runs as expected. As of yet I haven't found a way to get
the java CLI to recognize the LD_LIBRARY_PATH environment variable, even
though it echos out correctly when queried. Strange.
On Thu, Jun 30, 2022 at 9:39 AM sebb <se...@gmail.com> wrote:
> I've started copying bits of the C code to create a standalone tool to
> load the dll and report some info.
>
> No idea why yet, but it behaves differently from basically the same
> code when run via JNI.
>
> For example, the standalone code takes note of LD_LIBRARY_PATH when
> loading the dll, whereas Java apparently does not.
>
> On Thu, 30 Jun 2022 at 13:22, Alex Remily <al...@gmail.com> wrote:
> >
> > I also get the "Error looking up function 'ENGINE_load_rdrand'" when I
> run
> > the Jna test class from the command line. Interesting. dlopen finds the
> > LibreSSL when run from the CLI.
> >
> > On Thu, Jun 30, 2022 at 8:02 AM sebb <se...@gmail.com> wrote:
> >
> > > That looks fine, however I don't see the same, and nor does GH.
> > >
> > > What version of macOS are you using?
> > >
> > > On Thu, 30 Jun 2022 at 12:52, Gary Gregory <ga...@gmail.com>
> wrote:
> > > >
> > > > With the 1.1.0 tagm I get:
> > > >
> > > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> > > > target/classes org.apache.commons.crypto.Crypto
> > > > Apache Commons Crypto 1.1.0
> > > > Native code loaded OK: 1.1.0
> > > > Native name: Apache Commons Crypto
> > > > Native built: Jun 30 2022
> > > > OpenSSL library loaded OK, version: 0x1010107f
> > > > OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> > > > Random instance created OK:
> > > > org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> > > > Cipher AES/CTR/NoPadding instance created OK:
> > > > org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> > > > Additional OpenSSL_version(n) details:
> > > > 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> > > > -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> > > > -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
> > > > -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> > > > -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> > > > -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> > > > 2: built on: Tue Apr 21 13:30:00 2020 UTC
> > > > 3: platform: darwin64-x86_64-cc
> > > > 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> > > > 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1
> /1.1.1g/lib/engines-1.1"
> > > >
> > > > and
> > > >
> > > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q
> exec:java
> > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > isEnabled(): true
> > > > initialisationError(): null
> > > >
> > > > Gary
> > > >
> > > > On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> > > > >
> > > > > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> > > > >
> > > > > As previously noted, please try:
> > > > > java -cp target/classes org.apache.commons.crypto.Crypto
> > > > > and
> > > > > mvn -q exec:java
> > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > >
> > > > > See the GH run I linked yesterday:
> > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > >
> > > > > I just noticed also that 'openssl version' shows 1.1.1p but the
> Crypto
> > > > > class shows
> > > > > OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0
> code
> > > crashes)
> > > > >
> > > > > Something strange is going on; it looks like dlopen is finding a
> > > > > different version of the dynamic library from the default CLI.
> > > > > I have the same issue on my mac.
> > > > >
> > > > > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <garydgregory@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS
> and
> > > ran
> > > > > > 'mvn clean verify' and everything built just fine.
> > > > > >
> > > > > > The Maven ant-run output was:
> > > > > >
> > > > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto
> ---
> > > > > > [INFO] Executing tasks
> > > > > >
> > > > > > make:
> > > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah"
> -force
> > > > > > -classpath target/classes -o
> > > > > >
> > >
> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > > > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> -I/usr/local/include
> > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > -I"/usr/local/Cellar/openjdk@8
> > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > >
> > >
> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > > > > -o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah"
> -force
> > > > > > -classpath target/classes -o
> > > > > >
> target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > > > > org.apache.commons.crypto.cipher.OpenSslNative
> > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> -I/usr/local/include
> > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > -I"/usr/local/Cellar/openjdk@8
> > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c
> -o
> > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah"
> -force
> > > > > > -classpath target/classes -o
> > > > > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > > > > org.apache.commons.crypto.OpenSslInfoNative
> > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> -I/usr/local/include
> > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > > -I"/usr/local/Cellar/openjdk@8
> > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > > > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > > > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > > > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > > -mmacosx-version-min=10.7 -fvisibility=hidden
> -I/usr/local/include
> > > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > > -I"/usr/local/Cellar/openjdk@8
> > > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > >
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > -dynamiclib
> > > > > > -L/usr/local/lib
> > > > > > [exec] strip -x
> > > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > > [exec] cp
> > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > >
> > >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > > [exec] cp
> > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > >
> > >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > >
> > > > > > Gary
> > > > > >
> > > > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <
> garydgregory@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > I agree with Alex.
> > > > > > >
> > > > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple,
> and
> > > > > > > tight. Last time I checked I had an antique version of
> LibreSSL on
> > > my
> > > > > > > mac years ago, I just installed OpenSSL and never looked back.
> > > > > > >
> > > > > > > Gary
> > > > > > >
> > > > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <
> alex.remily@gmail.com>
> > > wrote:
> > > > > > > >
> > > > > > > > <The question is whether to do anything about the previously
> > > undetected
> > > > > > > > issues with JNA on macOS and Windows, and if so, whether this
> > > needs to
> > > > > > > > be done for the current or a later release. If we don't fix
> this
> > > > > > > > release, obviously it needs some mention in the release
> notes.>
> > > > > > > >
> > > > > > > > I wouldn't characterize the issues running against LibreSSL
> as
> > > > > > > > "undetected". I also don't see this as an issue with Mac or
> > > Windows, but
> > > > > > > > with LibreSSL. Install any supported OpenSSL library on any
> > > supported
> > > > > > > > architecture (to include Mac and Windows) and all commons
> crypto
> > > > > > > > functionality is available. I first encountered the rand
> issue
> > > you
> > > > > > > > describe years ago when I was becoming familiar with commons
> > > crypto. I did
> > > > > > > > a little research, discovered that I was running LibreSSL
> (and
> > > an old
> > > > > > > > version at that), installed a supported version of OpenSSL
> and
> > > forgot
> > > > > > > > all about it until this thread. I don't think it's
> unreasonable
> > > to expect
> > > > > > > > users to install a supported version of OpenSSL on a
> supported
> > > architecture
> > > > > > > > as a precondition of using commons crypto. I think it could
> > > become
> > > > > > > > cumbersome to try and support every vendor default *SSL
> > > install. That
> > > > > > > > said, I don't have a problem committing this particular
> "fix" to
> > > the
> > > > > > > > baseline, particularly since you have already done the
> work. I
> > > just don't
> > > > > > > > think that the project should obligate itself to do so or
> > > advertise
> > > > > > > > LibreSSL (or any other non-OpenSSL branch) support as such.
> I'm
> > > advocating
> > > > > > > > a "use at your own risk" approach to anything but OpenSSL
> > > proper. I agree
> > > > > > > > that we should update the documentation to reflect whatever
> we
> > > move forward
> > > > > > > > with.
> > > > > > > >
> > > > > > > > Anyway, that's my $0.02 worth.
> > > > > > > >
> > > > > > > > Alex
> > > > > > > >
> > > > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com>
> wrote:
> > > > > > > >
> > > > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <
> > > garydgregory@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > We cannot remove support for Windows and macOS, that's
> silly.
> > > > > > > > >
> > > > > > > > > AFAICT that means we must support the different set of
> > > function names
> > > > > > > > > in LibreSSL.
> > > > > > > > > Note that we only currently use a small proportion of them.
> > > > > > > > >
> > > > > > > > > > I have not followed all the branches and commits, so I'm
> not
> > > sure what
> > > > > > > > > the
> > > > > > > > > > current problems are, but I know I was able to release
> all
> > > OSs last go
> > > > > > > > > > around. I don't see why we need to rip out anything as
> > > fundamental.
> > > > > > > > >
> > > > > > > > > Well, I have tried running the Crypto and OpenSslJna main
> > > classes on
> > > > > > > > > macOS and Windows, and they both fail with the 1.1.0
> release.
> > > > > > > > >
> > > > > > > > > With current master, Crypto succeeds, but OpenSslJna fails
> to
> > > find
> > > > > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > > > > (The job step succeeds, because the code catches the
> exception)
> > > > > > > > >
> > > > > > > > > It looks like the test which would have exposed at least
> one
> > > of the
> > > > > > > > > issues was never enabled because of a failures on macOS;
> this
> > > hid the
> > > > > > > > > same problem on Windows.
> > > > > > > > >
> > > > > > > > > I am not suggesting we rip out anything at this point.
> > > > > > > > >
> > > > > > > > > The question is whether to do anything about the previously
> > > undetected
> > > > > > > > > issues with JNA on macOS and Windows, and if so, whether
> this
> > > needs to
> > > > > > > > > be done for the current or a later release. If we don't fix
> > > this
> > > > > > > > > release, obviously it needs some mention in the release
> notes.
> > > > > > > > >
> > > > > > > > > Sebb
> > > > > > > > >
> > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > > > > Gary
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com>
> wrote:
> > > > > > > > > >
> > > > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> > > alex.remily@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > I agree with Gary that we just don't support
> LibreSSL.
> > > To my
> > > > > > > > > knowledge
> > > > > > > > > > > > we've never advertised LibreSSL support, so I don't
> see
> > > it as an
> > > > > > > > > issue.
> > > > > > > > > > >
> > > > > > > > > > > In that case AFAICT we will have to drop *all* support
> for
> > > macOS and
> > > > > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > > > > >
> > > > > > > > > > > Note that adding support for LibreSSL was much easier
> for
> > > JNI, as it
> > > > > > > > > > > uses far fewer methods.
> > > > > > > > > > > Rather than checking the version, I changed the code to
> > > try OpenSSL
> > > > > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > > > > >
> > > > > > > > > > > If you want to try it out, compare 16345bc (old) with
> > > 3ee3f65 (new)
> > > > > > > > > > > macOS fails on 16345bc because it now uses LibreSSL
> which
> > > has a
> > > > > > > > > > > different mix of names.
> > > > > > > > > > >
> > > > > > > > > > > I think it's vital we support JNI as far as possible
> (and
> > > the code
> > > > > > > > > > > already does with commit 3ee3f65).
> > > > > > > > > > >
> > > > > > > > > > > However JNA is more of a backstop, so the fact that it
> has
> > > stopped
> > > > > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > > > > >
> > > > > > > > > > > But I don't think we can say we are not supporting
> > > LibreSSL at all.
> > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <
> sebbaz@gmail.com>
> > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> > > garydgregory@gmail.com
> > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The simple solution is to keep doing what we do
> now:
> > > only support
> > > > > > > > > > > OpenSSL
> > > > > > > > > > > > > > and not whatever Apple does with LibreSSL which
> may
> > > or may not
> > > > > > > > > be up
> > > > > > > > > > > to
> > > > > > > > > > > > > > date.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think this also affects Windows.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I don't have Windows box at present, but I have
> seen
> > > this on GH
> > > > > > > > > > > Actions.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If you have a WIndows build, perhaps you could try
> > > these tests:
> > > > > > > > > > > > >
> > > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > > > >
> > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > > > > >
> > > > > > > > > > > > > The first one should show the SSL details; on GH
> the
> > > output
> > > > > > > > > includes:
> > > > > > > > > > > > >
> > > > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > > > > >
> > > > > > > > > > > > > The second test crashes with:
> > > > > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up
> > > function
> > > > > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could
> > > not be found.
> > > > > > > > > > > > > isEnabled(): false
> > > > > > > > > > > > >
> > > > > > > > > > > > > initialisationError():
> java.lang.UnsatisfiedLinkError:
> > > Error
> > > > > > > > > looking
> > > > > > > > > > > > > up function 'ENGINE_load_rdrand': The specified
> > > procedure could
> > > > > > > > > not be
> > > > > > > > > > > > > found.
> > > > > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > > > > ...
> > > > > > > > > > > > >
> > > > > > > > > > > > > It would certainly be easier to ignore the problem
> for
> > > now.
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Gary
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <
> sebbaz@gmail.com>
> > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest)
> use
> > > LibreSSL by
> > > > > > > > > > > default
> > > > > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The LibreSSL API does not have the same
> functions
> > > as either
> > > > > > > > > 1.0.2
> > > > > > > > > > > or
> > > > > > > > > > > > > > > 1.1.1, so needs its own JNA class. In
> particular
> > > it looks like
> > > > > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> > > OpenSSL_version_num;
> > > > > > > > > > > 1.0.2
> > > > > > > > > > > > > > > has the former only, and 1.1.1 has the latter
> only.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This makes it hard to support JNA with the
> current
> > > design.
> > > > > > > > > > > > > > > It would require another
> OpenSsl<version>NativeJna
> > > class, and
> > > > > > > > > the
> > > > > > > > > > > > > > > parent class OpenSslNativeJna would need to use
> > > yet another
> > > > > > > > > > > condition
> > > > > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Seems to me it would be better to use something
> > > like a set of
> > > > > > > > > > > > > > > singleton classes that implement the required
> > > methods. The
> > > > > > > > > > > appropriate
> > > > > > > > > > > > > > > one can then be initialised and used by
> > > OpenSslNativeJna to
> > > > > > > > > field
> > > > > > > > > > > the
> > > > > > > > > > > > > > > actual methods. i.e. replace the conditional
> logic
> > > with a
> > > > > > > > > static
> > > > > > > > > > > > > > > reference to the correct API interface
> instance.
> > > This should
> > > > > > > > > only
> > > > > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Sebb
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@commons.apache.org
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> > > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@commons.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > > dev-help@commons.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > For additional commands, e-mail:
> dev-help@commons.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
I've started copying bits of the C code to create a standalone tool to
load the dll and report some info.
No idea why yet, but it behaves differently from basically the same
code when run via JNI.
For example, the standalone code takes note of LD_LIBRARY_PATH when
loading the dll, whereas Java apparently does not.
On Thu, 30 Jun 2022 at 13:22, Alex Remily <al...@gmail.com> wrote:
>
> I also get the "Error looking up function 'ENGINE_load_rdrand'" when I run
> the Jna test class from the command line. Interesting. dlopen finds the
> LibreSSL when run from the CLI.
>
> On Thu, Jun 30, 2022 at 8:02 AM sebb <se...@gmail.com> wrote:
>
> > That looks fine, however I don't see the same, and nor does GH.
> >
> > What version of macOS are you using?
> >
> > On Thu, 30 Jun 2022 at 12:52, Gary Gregory <ga...@gmail.com> wrote:
> > >
> > > With the 1.1.0 tagm I get:
> > >
> > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> > > target/classes org.apache.commons.crypto.Crypto
> > > Apache Commons Crypto 1.1.0
> > > Native code loaded OK: 1.1.0
> > > Native name: Apache Commons Crypto
> > > Native built: Jun 30 2022
> > > OpenSSL library loaded OK, version: 0x1010107f
> > > OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> > > Random instance created OK:
> > > org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> > > Cipher AES/CTR/NoPadding instance created OK:
> > > org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> > > Additional OpenSSL_version(n) details:
> > > 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> > > -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> > > -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
> > > -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> > > -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> > > -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> > > 2: built on: Tue Apr 21 13:30:00 2020 UTC
> > > 3: platform: darwin64-x86_64-cc
> > > 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> > > 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1/1.1.1g/lib/engines-1.1"
> > >
> > > and
> > >
> > > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q exec:java
> > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > isEnabled(): true
> > > initialisationError(): null
> > >
> > > Gary
> > >
> > > On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> > > >
> > > > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> > > >
> > > > As previously noted, please try:
> > > > java -cp target/classes org.apache.commons.crypto.Crypto
> > > > and
> > > > mvn -q exec:java
> > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > >
> > > > See the GH run I linked yesterday:
> > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > >
> > > > I just noticed also that 'openssl version' shows 1.1.1p but the Crypto
> > > > class shows
> > > > OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0 code
> > crashes)
> > > >
> > > > Something strange is going on; it looks like dlopen is finding a
> > > > different version of the dynamic library from the default CLI.
> > > > I have the same issue on my mac.
> > > >
> > > > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <ga...@gmail.com>
> > wrote:
> > > > >
> > > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and
> > ran
> > > > > 'mvn clean verify' and everything built just fine.
> > > > >
> > > > > The Maven ant-run output was:
> > > > >
> > > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > > > > [INFO] Executing tasks
> > > > >
> > > > > make:
> > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > > -classpath target/classes -o
> > > > >
> > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > >
> > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > > > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > > -classpath target/classes -o
> > > > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > > > org.apache.commons.crypto.cipher.OpenSslNative
> > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > > -classpath target/classes -o
> > > > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > > > org.apache.commons.crypto.OpenSslInfoNative
> > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > > -I"/usr/local/Cellar/openjdk@8
> > /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > -dynamiclib
> > > > > -L/usr/local/lib
> > > > > [exec] strip -x
> > > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > > [exec] cp
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > >
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > > [exec] cp
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > >
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > >
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com>
> > wrote:
> > > > > >
> > > > > > I agree with Alex.
> > > > > >
> > > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > > > > tight. Last time I checked I had an antique version of LibreSSL on
> > my
> > > > > > mac years ago, I just installed OpenSSL and never looked back.
> > > > > >
> > > > > > Gary
> > > > > >
> > > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > <The question is whether to do anything about the previously
> > undetected
> > > > > > > issues with JNA on macOS and Windows, and if so, whether this
> > needs to
> > > > > > > be done for the current or a later release. If we don't fix this
> > > > > > > release, obviously it needs some mention in the release notes.>
> > > > > > >
> > > > > > > I wouldn't characterize the issues running against LibreSSL as
> > > > > > > "undetected". I also don't see this as an issue with Mac or
> > Windows, but
> > > > > > > with LibreSSL. Install any supported OpenSSL library on any
> > supported
> > > > > > > architecture (to include Mac and Windows) and all commons crypto
> > > > > > > functionality is available. I first encountered the rand issue
> > you
> > > > > > > describe years ago when I was becoming familiar with commons
> > crypto. I did
> > > > > > > a little research, discovered that I was running LibreSSL (and
> > an old
> > > > > > > version at that), installed a supported version of OpenSSL and
> > forgot
> > > > > > > all about it until this thread. I don't think it's unreasonable
> > to expect
> > > > > > > users to install a supported version of OpenSSL on a supported
> > architecture
> > > > > > > as a precondition of using commons crypto. I think it could
> > become
> > > > > > > cumbersome to try and support every vendor default *SSL
> > install. That
> > > > > > > said, I don't have a problem committing this particular "fix" to
> > the
> > > > > > > baseline, particularly since you have already done the work. I
> > just don't
> > > > > > > think that the project should obligate itself to do so or
> > advertise
> > > > > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm
> > advocating
> > > > > > > a "use at your own risk" approach to anything but OpenSSL
> > proper. I agree
> > > > > > > that we should update the documentation to reflect whatever we
> > move forward
> > > > > > > with.
> > > > > > >
> > > > > > > Anyway, that's my $0.02 worth.
> > > > > > >
> > > > > > > Alex
> > > > > > >
> > > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <
> > garydgregory@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > > > > >
> > > > > > > > AFAICT that means we must support the different set of
> > function names
> > > > > > > > in LibreSSL.
> > > > > > > > Note that we only currently use a small proportion of them.
> > > > > > > >
> > > > > > > > > I have not followed all the branches and commits, so I'm not
> > sure what
> > > > > > > > the
> > > > > > > > > current problems are, but I know I was able to release all
> > OSs last go
> > > > > > > > > around. I don't see why we need to rip out anything as
> > fundamental.
> > > > > > > >
> > > > > > > > Well, I have tried running the Crypto and OpenSslJna main
> > classes on
> > > > > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > > > > >
> > > > > > > > With current master, Crypto succeeds, but OpenSslJna fails to
> > find
> > > > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > > > (The job step succeeds, because the code catches the exception)
> > > > > > > >
> > > > > > > > It looks like the test which would have exposed at least one
> > of the
> > > > > > > > issues was never enabled because of a failures on macOS; this
> > hid the
> > > > > > > > same problem on Windows.
> > > > > > > >
> > > > > > > > I am not suggesting we rip out anything at this point.
> > > > > > > >
> > > > > > > > The question is whether to do anything about the previously
> > undetected
> > > > > > > > issues with JNA on macOS and Windows, and if so, whether this
> > needs to
> > > > > > > > be done for the current or a later release. If we don't fix
> > this
> > > > > > > > release, obviously it needs some mention in the release notes.
> > > > > > > >
> > > > > > > > Sebb
> > > > > > > >
> > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > > > Gary
> > > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> > alex.remily@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > I agree with Gary that we just don't support LibreSSL.
> > To my
> > > > > > > > knowledge
> > > > > > > > > > > we've never advertised LibreSSL support, so I don't see
> > it as an
> > > > > > > > issue.
> > > > > > > > > >
> > > > > > > > > > In that case AFAICT we will have to drop *all* support for
> > macOS and
> > > > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > > > >
> > > > > > > > > > Note that adding support for LibreSSL was much easier for
> > JNI, as it
> > > > > > > > > > uses far fewer methods.
> > > > > > > > > > Rather than checking the version, I changed the code to
> > try OpenSSL
> > > > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > > > >
> > > > > > > > > > If you want to try it out, compare 16345bc (old) with
> > 3ee3f65 (new)
> > > > > > > > > > macOS fails on 16345bc because it now uses LibreSSL which
> > has a
> > > > > > > > > > different mix of names.
> > > > > > > > > >
> > > > > > > > > > I think it's vital we support JNI as far as possible (and
> > the code
> > > > > > > > > > already does with commit 3ee3f65).
> > > > > > > > > >
> > > > > > > > > > However JNA is more of a backstop, so the fact that it has
> > stopped
> > > > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > > > >
> > > > > > > > > > But I don't think we can say we are not supporting
> > LibreSSL at all.
> > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com>
> > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> > garydgregory@gmail.com
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > The simple solution is to keep doing what we do now:
> > only support
> > > > > > > > > > OpenSSL
> > > > > > > > > > > > > and not whatever Apple does with LibreSSL which may
> > or may not
> > > > > > > > be up
> > > > > > > > > > to
> > > > > > > > > > > > > date.
> > > > > > > > > > > >
> > > > > > > > > > > > I think this also affects Windows.
> > > > > > > > > > > >
> > > > > > > > > > > > I don't have Windows box at present, but I have seen
> > this on GH
> > > > > > > > > > Actions.
> > > > > > > > > > > >
> > > > > > > > > > > > If you have a WIndows build, perhaps you could try
> > these tests:
> > > > > > > > > > > >
> > > > > > > > > > > > mvn -q exec:java
> > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > > >
> > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > > > >
> > > > > > > > > > > > The first one should show the SSL details; on GH the
> > output
> > > > > > > > includes:
> > > > > > > > > > > >
> > > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > > > >
> > > > > > > > > > > > The second test crashes with:
> > > > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up
> > function
> > > > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could
> > not be found.
> > > > > > > > > > > > isEnabled(): false
> > > > > > > > > > > >
> > > > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError:
> > Error
> > > > > > > > looking
> > > > > > > > > > > > up function 'ENGINE_load_rdrand': The specified
> > procedure could
> > > > > > > > not be
> > > > > > > > > > > > found.
> > > > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > > > ...
> > > > > > > > > > > >
> > > > > > > > > > > > It would certainly be easier to ignore the problem for
> > now.
> > > > > > > > > > > >
> > > > > > > > > > > > > Gary
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com>
> > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use
> > LibreSSL by
> > > > > > > > > > default
> > > > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The LibreSSL API does not have the same functions
> > as either
> > > > > > > > 1.0.2
> > > > > > > > > > or
> > > > > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular
> > it looks like
> > > > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> > OpenSSL_version_num;
> > > > > > > > > > 1.0.2
> > > > > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This makes it hard to support JNA with the current
> > design.
> > > > > > > > > > > > > > It would require another OpenSsl<version>NativeJna
> > class, and
> > > > > > > > the
> > > > > > > > > > > > > > parent class OpenSslNativeJna would need to use
> > yet another
> > > > > > > > > > condition
> > > > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Seems to me it would be better to use something
> > like a set of
> > > > > > > > > > > > > > singleton classes that implement the required
> > methods. The
> > > > > > > > > > appropriate
> > > > > > > > > > > > > > one can then be initialised and used by
> > OpenSslNativeJna to
> > > > > > > > field
> > > > > > > > > > the
> > > > > > > > > > > > > > actual methods. i.e. replace the conditional logic
> > with a
> > > > > > > > static
> > > > > > > > > > > > > > reference to the correct API interface instance.
> > This should
> > > > > > > > only
> > > > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Sebb
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@commons.apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail:
> > dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > For additional commands, e-mail:
> > dev-help@commons.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > For additional commands, e-mail:
> > dev-help@commons.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
I also get the "Error looking up function 'ENGINE_load_rdrand'" when I run
the Jna test class from the command line. Interesting. dlopen finds the
LibreSSL when run from the CLI.
On Thu, Jun 30, 2022 at 8:02 AM sebb <se...@gmail.com> wrote:
> That looks fine, however I don't see the same, and nor does GH.
>
> What version of macOS are you using?
>
> On Thu, 30 Jun 2022 at 12:52, Gary Gregory <ga...@gmail.com> wrote:
> >
> > With the 1.1.0 tagm I get:
> >
> > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> > target/classes org.apache.commons.crypto.Crypto
> > Apache Commons Crypto 1.1.0
> > Native code loaded OK: 1.1.0
> > Native name: Apache Commons Crypto
> > Native built: Jun 30 2022
> > OpenSSL library loaded OK, version: 0x1010107f
> > OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> > Random instance created OK:
> > org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> > Cipher AES/CTR/NoPadding instance created OK:
> > org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> > Additional OpenSSL_version(n) details:
> > 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> > -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> > -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
> > -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> > -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> > -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> > 2: built on: Tue Apr 21 13:30:00 2020 UTC
> > 3: platform: darwin64-x86_64-cc
> > 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> > 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1/1.1.1g/lib/engines-1.1"
> >
> > and
> >
> > garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q exec:java
> > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > isEnabled(): true
> > initialisationError(): null
> >
> > Gary
> >
> > On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> > >
> > > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> > >
> > > As previously noted, please try:
> > > java -cp target/classes org.apache.commons.crypto.Crypto
> > > and
> > > mvn -q exec:java
> -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > >
> > > See the GH run I linked yesterday:
> > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > >
> > > I just noticed also that 'openssl version' shows 1.1.1p but the Crypto
> > > class shows
> > > OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0 code
> crashes)
> > >
> > > Something strange is going on; it looks like dlopen is finding a
> > > different version of the dynamic library from the default CLI.
> > > I have the same issue on my mac.
> > >
> > > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <ga...@gmail.com>
> wrote:
> > > >
> > > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and
> ran
> > > > 'mvn clean verify' and everything built just fine.
> > > >
> > > > The Maven ant-run output was:
> > > >
> > > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > > > [INFO] Executing tasks
> > > >
> > > > make:
> > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > -classpath target/classes -o
> > > >
> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > >
> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > -classpath target/classes -o
> > > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > > org.apache.commons.crypto.cipher.OpenSslNative
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > > -classpath target/classes -o
> > > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > > org.apache.commons.crypto.OpenSslInfoNative
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"src/main/native/org/apache/commons/crypto/"
> > > > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > > -I"/usr/local/Cellar/openjdk@8
> /1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> -dynamiclib
> > > > -L/usr/local/lib
> > > > [exec] strip -x
> > > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > > [exec] cp
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > > [exec] cp
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > >
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > >
> > > > Gary
> > > >
> > > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com>
> wrote:
> > > > >
> > > > > I agree with Alex.
> > > > >
> > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > > > tight. Last time I checked I had an antique version of LibreSSL on
> my
> > > > > mac years ago, I just installed OpenSSL and never looked back.
> > > > >
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com>
> wrote:
> > > > > >
> > > > > > <The question is whether to do anything about the previously
> undetected
> > > > > > issues with JNA on macOS and Windows, and if so, whether this
> needs to
> > > > > > be done for the current or a later release. If we don't fix this
> > > > > > release, obviously it needs some mention in the release notes.>
> > > > > >
> > > > > > I wouldn't characterize the issues running against LibreSSL as
> > > > > > "undetected". I also don't see this as an issue with Mac or
> Windows, but
> > > > > > with LibreSSL. Install any supported OpenSSL library on any
> supported
> > > > > > architecture (to include Mac and Windows) and all commons crypto
> > > > > > functionality is available. I first encountered the rand issue
> you
> > > > > > describe years ago when I was becoming familiar with commons
> crypto. I did
> > > > > > a little research, discovered that I was running LibreSSL (and
> an old
> > > > > > version at that), installed a supported version of OpenSSL and
> forgot
> > > > > > all about it until this thread. I don't think it's unreasonable
> to expect
> > > > > > users to install a supported version of OpenSSL on a supported
> architecture
> > > > > > as a precondition of using commons crypto. I think it could
> become
> > > > > > cumbersome to try and support every vendor default *SSL
> install. That
> > > > > > said, I don't have a problem committing this particular "fix" to
> the
> > > > > > baseline, particularly since you have already done the work. I
> just don't
> > > > > > think that the project should obligate itself to do so or
> advertise
> > > > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm
> advocating
> > > > > > a "use at your own risk" approach to anything but OpenSSL
> proper. I agree
> > > > > > that we should update the documentation to reflect whatever we
> move forward
> > > > > > with.
> > > > > >
> > > > > > Anyway, that's my $0.02 worth.
> > > > > >
> > > > > > Alex
> > > > > >
> > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <
> garydgregory@gmail.com> wrote:
> > > > > > > >
> > > > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > > > >
> > > > > > > AFAICT that means we must support the different set of
> function names
> > > > > > > in LibreSSL.
> > > > > > > Note that we only currently use a small proportion of them.
> > > > > > >
> > > > > > > > I have not followed all the branches and commits, so I'm not
> sure what
> > > > > > > the
> > > > > > > > current problems are, but I know I was able to release all
> OSs last go
> > > > > > > > around. I don't see why we need to rip out anything as
> fundamental.
> > > > > > >
> > > > > > > Well, I have tried running the Crypto and OpenSslJna main
> classes on
> > > > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > > > >
> > > > > > > With current master, Crypto succeeds, but OpenSslJna fails to
> find
> > > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > > (The job step succeeds, because the code catches the exception)
> > > > > > >
> > > > > > > It looks like the test which would have exposed at least one
> of the
> > > > > > > issues was never enabled because of a failures on macOS; this
> hid the
> > > > > > > same problem on Windows.
> > > > > > >
> > > > > > > I am not suggesting we rip out anything at this point.
> > > > > > >
> > > > > > > The question is whether to do anything about the previously
> undetected
> > > > > > > issues with JNA on macOS and Windows, and if so, whether this
> needs to
> > > > > > > be done for the current or a later release. If we don't fix
> this
> > > > > > > release, obviously it needs some mention in the release notes.
> > > > > > >
> > > > > > > Sebb
> > > > > > >
> https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > > Gary
> > > > > > > >
> > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <
> alex.remily@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > I agree with Gary that we just don't support LibreSSL.
> To my
> > > > > > > knowledge
> > > > > > > > > > we've never advertised LibreSSL support, so I don't see
> it as an
> > > > > > > issue.
> > > > > > > > >
> > > > > > > > > In that case AFAICT we will have to drop *all* support for
> macOS and
> > > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > > >
> > > > > > > > > Note that adding support for LibreSSL was much easier for
> JNI, as it
> > > > > > > > > uses far fewer methods.
> > > > > > > > > Rather than checking the version, I changed the code to
> try OpenSSL
> > > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > > >
> > > > > > > > > If you want to try it out, compare 16345bc (old) with
> 3ee3f65 (new)
> > > > > > > > > macOS fails on 16345bc because it now uses LibreSSL which
> has a
> > > > > > > > > different mix of names.
> > > > > > > > >
> > > > > > > > > I think it's vital we support JNI as far as possible (and
> the code
> > > > > > > > > already does with commit 3ee3f65).
> > > > > > > > >
> > > > > > > > > However JNA is more of a backstop, so the fact that it has
> stopped
> > > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > > >
> > > > > > > > > But I don't think we can say we are not supporting
> LibreSSL at all.
> > > > > > > > >
> > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com>
> wrote:
> > > > > > > > > >
> > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <
> garydgregory@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > The simple solution is to keep doing what we do now:
> only support
> > > > > > > > > OpenSSL
> > > > > > > > > > > > and not whatever Apple does with LibreSSL which may
> or may not
> > > > > > > be up
> > > > > > > > > to
> > > > > > > > > > > > date.
> > > > > > > > > > >
> > > > > > > > > > > I think this also affects Windows.
> > > > > > > > > > >
> > > > > > > > > > > I don't have Windows box at present, but I have seen
> this on GH
> > > > > > > > > Actions.
> > > > > > > > > > >
> > > > > > > > > > > If you have a WIndows build, perhaps you could try
> these tests:
> > > > > > > > > > >
> > > > > > > > > > > mvn -q exec:java
> > > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > > mvn -q exec:java
> > > > > > > > > > >
> -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > > >
> > > > > > > > > > > The first one should show the SSL details; on GH the
> output
> > > > > > > includes:
> > > > > > > > > > >
> > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > > >
> > > > > > > > > > > The second test crashes with:
> > > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up
> function
> > > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could
> not be found.
> > > > > > > > > > > isEnabled(): false
> > > > > > > > > > >
> > > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError:
> Error
> > > > > > > looking
> > > > > > > > > > > up function 'ENGINE_load_rdrand': The specified
> procedure could
> > > > > > > not be
> > > > > > > > > > > found.
> > > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > > ...
> > > > > > > > > > >
> > > > > > > > > > > It would certainly be easier to ignore the problem for
> now.
> > > > > > > > > > >
> > > > > > > > > > > > Gary
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com>
> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use
> LibreSSL by
> > > > > > > > > default
> > > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The LibreSSL API does not have the same functions
> as either
> > > > > > > 1.0.2
> > > > > > > > > or
> > > > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular
> it looks like
> > > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is
> OpenSSL_version_num;
> > > > > > > > > 1.0.2
> > > > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This makes it hard to support JNA with the current
> design.
> > > > > > > > > > > > > It would require another OpenSsl<version>NativeJna
> class, and
> > > > > > > the
> > > > > > > > > > > > > parent class OpenSslNativeJna would need to use
> yet another
> > > > > > > > > condition
> > > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Seems to me it would be better to use something
> like a set of
> > > > > > > > > > > > > singleton classes that implement the required
> methods. The
> > > > > > > > > appropriate
> > > > > > > > > > > > > one can then be initialised and used by
> OpenSslNativeJna to
> > > > > > > field
> > > > > > > > > the
> > > > > > > > > > > > > actual methods. i.e. replace the conditional logic
> with a
> > > > > > > static
> > > > > > > > > > > > > reference to the correct API interface instance.
> This should
> > > > > > > only
> > > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Sebb
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > > For additional commands, e-mail:
> dev-help@commons.apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> dev-help@commons.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > For additional commands, e-mail:
> dev-help@commons.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
That looks fine, however I don't see the same, and nor does GH.
What version of macOS are you using?
On Thu, 30 Jun 2022 at 12:52, Gary Gregory <ga...@gmail.com> wrote:
>
> With the 1.1.0 tagm I get:
>
> garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
> target/classes org.apache.commons.crypto.Crypto
> Apache Commons Crypto 1.1.0
> Native code loaded OK: 1.1.0
> Native name: Apache Commons Crypto
> Native built: Jun 30 2022
> OpenSSL library loaded OK, version: 0x1010107f
> OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
> Random instance created OK:
> org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
> Cipher AES/CTR/NoPadding instance created OK:
> org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
> Additional OpenSSL_version(n) details:
> 1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
> -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
> -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
> -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
> -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
> -DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
> 2: built on: Tue Apr 21 13:30:00 2020 UTC
> 3: platform: darwin64-x86_64-cc
> 4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
> 5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1/1.1.1g/lib/engines-1.1"
>
> and
>
> garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q exec:java
> -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> isEnabled(): true
> initialisationError(): null
>
> Gary
>
> On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
> >
> > Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
> >
> > As previously noted, please try:
> > java -cp target/classes org.apache.commons.crypto.Crypto
> > and
> > mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> >
> > See the GH run I linked yesterday:
> > https://github.com/apache/commons-crypto/actions/runs/2586011129
> >
> > I just noticed also that 'openssl version' shows 1.1.1p but the Crypto
> > class shows
> > OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0 code crashes)
> >
> > Something strange is going on; it looks like dlopen is finding a
> > different version of the dynamic library from the default CLI.
> > I have the same issue on my mac.
> >
> > On Thu, 30 Jun 2022 at 01:50, Gary Gregory <ga...@gmail.com> wrote:
> > >
> > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> > > 'mvn clean verify' and everything built just fine.
> > >
> > > The Maven ant-run output was:
> > >
> > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > > [INFO] Executing tasks
> > >
> > > make:
> > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > -classpath target/classes -o
> > > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"src/main/native/org/apache/commons/crypto/"
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > -classpath target/classes -o
> > > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > > org.apache.commons.crypto.cipher.OpenSslNative
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"src/main/native/org/apache/commons/crypto/"
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > > -classpath target/classes -o
> > > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > > org.apache.commons.crypto.OpenSslInfoNative
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"src/main/native/org/apache/commons/crypto/"
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random"
> > > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > > -I"target/jni-classes/org/apache/commons/crypto" -c
> > > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> > > -L/usr/local/lib
> > > [exec] strip -x
> > > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > >
> > > Gary
> > >
> > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com> wrote:
> > > >
> > > > I agree with Alex.
> > > >
> > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > > tight. Last time I checked I had an antique version of LibreSSL on my
> > > > mac years ago, I just installed OpenSSL and never looked back.
> > > >
> > > > Gary
> > > >
> > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
> > > > >
> > > > > <The question is whether to do anything about the previously undetected
> > > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > > be done for the current or a later release. If we don't fix this
> > > > > release, obviously it needs some mention in the release notes.>
> > > > >
> > > > > I wouldn't characterize the issues running against LibreSSL as
> > > > > "undetected". I also don't see this as an issue with Mac or Windows, but
> > > > > with LibreSSL. Install any supported OpenSSL library on any supported
> > > > > architecture (to include Mac and Windows) and all commons crypto
> > > > > functionality is available. I first encountered the rand issue you
> > > > > describe years ago when I was becoming familiar with commons crypto. I did
> > > > > a little research, discovered that I was running LibreSSL (and an old
> > > > > version at that), installed a supported version of OpenSSL and forgot
> > > > > all about it until this thread. I don't think it's unreasonable to expect
> > > > > users to install a supported version of OpenSSL on a supported architecture
> > > > > as a precondition of using commons crypto. I think it could become
> > > > > cumbersome to try and support every vendor default *SSL install. That
> > > > > said, I don't have a problem committing this particular "fix" to the
> > > > > baseline, particularly since you have already done the work. I just don't
> > > > > think that the project should obligate itself to do so or advertise
> > > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> > > > > a "use at your own risk" approach to anything but OpenSSL proper. I agree
> > > > > that we should update the documentation to reflect whatever we move forward
> > > > > with.
> > > > >
> > > > > Anyway, that's my $0.02 worth.
> > > > >
> > > > > Alex
> > > > >
> > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > > > > > >
> > > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > > >
> > > > > > AFAICT that means we must support the different set of function names
> > > > > > in LibreSSL.
> > > > > > Note that we only currently use a small proportion of them.
> > > > > >
> > > > > > > I have not followed all the branches and commits, so I'm not sure what
> > > > > > the
> > > > > > > current problems are, but I know I was able to release all OSs last go
> > > > > > > around. I don't see why we need to rip out anything as fundamental.
> > > > > >
> > > > > > Well, I have tried running the Crypto and OpenSslJna main classes on
> > > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > > >
> > > > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > > (The job step succeeds, because the code catches the exception)
> > > > > >
> > > > > > It looks like the test which would have exposed at least one of the
> > > > > > issues was never enabled because of a failures on macOS; this hid the
> > > > > > same problem on Windows.
> > > > > >
> > > > > > I am not suggesting we rip out anything at this point.
> > > > > >
> > > > > > The question is whether to do anything about the previously undetected
> > > > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > > > be done for the current or a later release. If we don't fix this
> > > > > > release, obviously it needs some mention in the release notes.
> > > > > >
> > > > > > Sebb
> > > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > > Gary
> > > > > > >
> > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > > > knowledge
> > > > > > > > > we've never advertised LibreSSL support, so I don't see it as an
> > > > > > issue.
> > > > > > > >
> > > > > > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > > >
> > > > > > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > > > > > uses far fewer methods.
> > > > > > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > > > > > 1.1 names first, then a fallback.
> > > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > > >
> > > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > > > different mix of names.
> > > > > > > >
> > > > > > > > I think it's vital we support JNI as far as possible (and the code
> > > > > > > > already does with commit 3ee3f65).
> > > > > > > >
> > > > > > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > > > > > working for macOS and Windows is less of an issue.
> > > > > > > >
> > > > > > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > The simple solution is to keep doing what we do now: only support
> > > > > > > > OpenSSL
> > > > > > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > > > > > be up
> > > > > > > > to
> > > > > > > > > > > date.
> > > > > > > > > >
> > > > > > > > > > I think this also affects Windows.
> > > > > > > > > >
> > > > > > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > > > > > Actions.
> > > > > > > > > >
> > > > > > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > > > > > >
> > > > > > > > > > mvn -q exec:java
> > > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > > mvn -q exec:java
> > > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > > >
> > > > > > > > > > The first one should show the SSL details; on GH the output
> > > > > > includes:
> > > > > > > > > >
> > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > > >
> > > > > > > > > > The second test crashes with:
> > > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > > > > > isEnabled(): false
> > > > > > > > > >
> > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > > > > > looking
> > > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > > > > > not be
> > > > > > > > > > found.
> > > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > > ...
> > > > > > > > > >
> > > > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > > > >
> > > > > > > > > > > Gary
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > > > > > default
> > > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > > >
> > > > > > > > > > > > The LibreSSL API does not have the same functions as either
> > > > > > 1.0.2
> > > > > > > > or
> > > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > > > > > 1.0.2
> > > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > > >
> > > > > > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > > > > > the
> > > > > > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > > > > > condition
> > > > > > > > > > > > in each of its methods.
> > > > > > > > > > > >
> > > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > > >
> > > > > > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > > > > > singleton classes that implement the required methods. The
> > > > > > > > appropriate
> > > > > > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > > > > > field
> > > > > > > > the
> > > > > > > > > > > > actual methods. i.e. replace the conditional logic with a
> > > > > > static
> > > > > > > > > > > > reference to the correct API interface instance. This should
> > > > > > only
> > > > > > > > > > > > affect non-public classes.
> > > > > > > > > > > >
> > > > > > > > > > > > Any other suggestions?
> > > > > > > > > > > >
> > > > > > > > > > > > Sebb
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
With the 1.1.0 tagm I get:
garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b java -cp
target/classes org.apache.commons.crypto.Crypto
Apache Commons Crypto 1.1.0
Native code loaded OK: 1.1.0
Native name: Apache Commons Crypto
Native built: Jun 30 2022
OpenSSL library loaded OK, version: 0x1010107f
OpenSSL library info: OpenSSL 1.1.1g 21 Apr 2020
Random instance created OK:
org.apache.commons.crypto.random.OpenSslCryptoRandom@266474c2
Cipher AES/CTR/NoPadding instance created OK:
org.apache.commons.crypto.cipher.OpenSslCipher@66d3c617
Additional OpenSSL_version(n) details:
1: compiler: clang -fPIC -arch x86_64 -O3 -Wall -DL_ENDIAN
-DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM
-DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
-DX25519_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
2: built on: Tue Apr 21 13:30:00 2020 UTC
3: platform: darwin64-x86_64-cc
4: OPENSSLDIR: "/usr/local/etc/openssl@1.1"
5: ENGINESDIR: "/usr/local/Cellar/openssl@1.1/1.1.1g/lib/engines-1.1"
and
garydgregory@xxx ~/git/commons-crypto ➦ 3b2561b mvn -q exec:java
-D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
isEnabled(): true
initialisationError(): null
Gary
On Thu, Jun 30, 2022 at 7:18 AM sebb <se...@gmail.com> wrote:
>
> Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
>
> As previously noted, please try:
> java -cp target/classes org.apache.commons.crypto.Crypto
> and
> mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
>
> See the GH run I linked yesterday:
> https://github.com/apache/commons-crypto/actions/runs/2586011129
>
> I just noticed also that 'openssl version' shows 1.1.1p but the Crypto
> class shows
> OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0 code crashes)
>
> Something strange is going on; it looks like dlopen is finding a
> different version of the dynamic library from the default CLI.
> I have the same issue on my mac.
>
> On Thu, 30 Jun 2022 at 01:50, Gary Gregory <ga...@gmail.com> wrote:
> >
> > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> > 'mvn clean verify' and everything built just fine.
> >
> > The Maven ant-run output was:
> >
> > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> > [INFO] Executing tasks
> >
> > make:
> > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > -classpath target/classes -o
> > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> > org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"src/main/native/org/apache/commons/crypto/"
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > -classpath target/classes -o
> > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> > org.apache.commons.crypto.cipher.OpenSslNative
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"src/main/native/org/apache/commons/crypto/"
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random" -c
> > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> > -classpath target/classes -o
> > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> > org.apache.commons.crypto.OpenSslInfoNative
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"src/main/native/org/apache/commons/crypto/"
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random"
> > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> > -I"target/jni-classes/org/apache/commons/crypto" -c
> > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> > [exec] gcc -arch x86_64 -Ilib/inc_mac
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> > -I"target/jni-classes/org/apache/commons/crypto/cipher"
> > -I"target/jni-classes/org/apache/commons/crypto/random" -o
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> > -L/usr/local/lib
> > [exec] strip -x
> > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> >
> > Gary
> >
> > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com> wrote:
> > >
> > > I agree with Alex.
> > >
> > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > > tight. Last time I checked I had an antique version of LibreSSL on my
> > > mac years ago, I just installed OpenSSL and never looked back.
> > >
> > > Gary
> > >
> > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
> > > >
> > > > <The question is whether to do anything about the previously undetected
> > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > be done for the current or a later release. If we don't fix this
> > > > release, obviously it needs some mention in the release notes.>
> > > >
> > > > I wouldn't characterize the issues running against LibreSSL as
> > > > "undetected". I also don't see this as an issue with Mac or Windows, but
> > > > with LibreSSL. Install any supported OpenSSL library on any supported
> > > > architecture (to include Mac and Windows) and all commons crypto
> > > > functionality is available. I first encountered the rand issue you
> > > > describe years ago when I was becoming familiar with commons crypto. I did
> > > > a little research, discovered that I was running LibreSSL (and an old
> > > > version at that), installed a supported version of OpenSSL and forgot
> > > > all about it until this thread. I don't think it's unreasonable to expect
> > > > users to install a supported version of OpenSSL on a supported architecture
> > > > as a precondition of using commons crypto. I think it could become
> > > > cumbersome to try and support every vendor default *SSL install. That
> > > > said, I don't have a problem committing this particular "fix" to the
> > > > baseline, particularly since you have already done the work. I just don't
> > > > think that the project should obligate itself to do so or advertise
> > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> > > > a "use at your own risk" approach to anything but OpenSSL proper. I agree
> > > > that we should update the documentation to reflect whatever we move forward
> > > > with.
> > > >
> > > > Anyway, that's my $0.02 worth.
> > > >
> > > > Alex
> > > >
> > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > > >
> > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > > > > >
> > > > > > We cannot remove support for Windows and macOS, that's silly.
> > > > >
> > > > > AFAICT that means we must support the different set of function names
> > > > > in LibreSSL.
> > > > > Note that we only currently use a small proportion of them.
> > > > >
> > > > > > I have not followed all the branches and commits, so I'm not sure what
> > > > > the
> > > > > > current problems are, but I know I was able to release all OSs last go
> > > > > > around. I don't see why we need to rip out anything as fundamental.
> > > > >
> > > > > Well, I have tried running the Crypto and OpenSslJna main classes on
> > > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > > >
> > > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > > (The job step succeeds, because the code catches the exception)
> > > > >
> > > > > It looks like the test which would have exposed at least one of the
> > > > > issues was never enabled because of a failures on macOS; this hid the
> > > > > same problem on Windows.
> > > > >
> > > > > I am not suggesting we rip out anything at this point.
> > > > >
> > > > > The question is whether to do anything about the previously undetected
> > > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > > be done for the current or a later release. If we don't fix this
> > > > > release, obviously it needs some mention in the release notes.
> > > > >
> > > > > Sebb
> > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > > Gary
> > > > > >
> > > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > > knowledge
> > > > > > > > we've never advertised LibreSSL support, so I don't see it as an
> > > > > issue.
> > > > > > >
> > > > > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > > >
> > > > > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > > > > uses far fewer methods.
> > > > > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > > > > 1.1 names first, then a fallback.
> > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > > >
> > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > > different mix of names.
> > > > > > >
> > > > > > > I think it's vital we support JNI as far as possible (and the code
> > > > > > > already does with commit 3ee3f65).
> > > > > > >
> > > > > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > > > > working for macOS and Windows is less of an issue.
> > > > > > >
> > > > > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > > > > >
> > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > The simple solution is to keep doing what we do now: only support
> > > > > > > OpenSSL
> > > > > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > > > > be up
> > > > > > > to
> > > > > > > > > > date.
> > > > > > > > >
> > > > > > > > > I think this also affects Windows.
> > > > > > > > >
> > > > > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > > > > Actions.
> > > > > > > > >
> > > > > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > > > > >
> > > > > > > > > mvn -q exec:java
> > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > > mvn -q exec:java
> > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > > >
> > > > > > > > > The first one should show the SSL details; on GH the output
> > > > > includes:
> > > > > > > > >
> > > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > > >
> > > > > > > > > The second test crashes with:
> > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > > > > isEnabled(): false
> > > > > > > > >
> > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > > > > looking
> > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > > > > not be
> > > > > > > > > found.
> > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > > ...
> > > > > > > > >
> > > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > > >
> > > > > > > > > > Gary
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > > > > default
> > > > > > > > > > > rather than OpenSSL.
> > > > > > > > > > >
> > > > > > > > > > > The LibreSSL API does not have the same functions as either
> > > > > 1.0.2
> > > > > > > or
> > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > > > > 1.0.2
> > > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > > >
> > > > > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > > > > the
> > > > > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > > > > condition
> > > > > > > > > > > in each of its methods.
> > > > > > > > > > >
> > > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > > >
> > > > > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > > > > singleton classes that implement the required methods. The
> > > > > > > appropriate
> > > > > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > > > > field
> > > > > > > the
> > > > > > > > > > > actual methods. i.e. replace the conditional logic with a
> > > > > static
> > > > > > > > > > > reference to the correct API interface instance. This should
> > > > > only
> > > > > > > > > > > affect non-public classes.
> > > > > > > > > > >
> > > > > > > > > > > Any other suggestions?
> > > > > > > > > > >
> > > > > > > > > > > Sebb
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
Yes, 1.1.0 builds OK on macOS, but the sample classes fail.
As previously noted, please try:
java -cp target/classes org.apache.commons.crypto.Crypto
and
mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
See the GH run I linked yesterday:
https://github.com/apache/commons-crypto/actions/runs/2586011129
I just noticed also that 'openssl version' shows 1.1.1p but the Crypto
class shows
OpenSSL library info: LibreSSL 2.2.9 (on current code; the 1.11.0 code crashes)
Something strange is going on; it looks like dlopen is finding a
different version of the dynamic library from the default CLI.
I have the same issue on my mac.
On Thu, 30 Jun 2022 at 01:50, Gary Gregory <ga...@gmail.com> wrote:
>
> FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
> 'mvn clean verify' and everything built just fine.
>
> The Maven ant-run output was:
>
> [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
> [INFO] Executing tasks
>
> make:
> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> -classpath target/classes -o
> target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
> org.apache.commons.crypto.random.OpenSslCryptoRandomNative
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"src/main/native/org/apache/commons/crypto/"
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random" -c
> src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
> -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> -classpath target/classes -o
> target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
> org.apache.commons.crypto.cipher.OpenSslNative
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"src/main/native/org/apache/commons/crypto/"
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random" -c
> src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
> -classpath target/classes -o
> target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
> org.apache.commons.crypto.OpenSslInfoNative
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"src/main/native/org/apache/commons/crypto/"
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random"
> -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
> -I"target/jni-classes/org/apache/commons/crypto" -c
> src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
> [exec] gcc -arch x86_64 -Ilib/inc_mac
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
> -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
> -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
> -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
> -I"target/jni-classes/org/apache/commons/crypto/cipher"
> -I"target/jni-classes/org/apache/commons/crypto/random" -o
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
> target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
> -L/usr/local/lib
> [exec] strip -x
> target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
> [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
> target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
>
> Gary
>
> On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com> wrote:
> >
> > I agree with Alex.
> >
> > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> > tight. Last time I checked I had an antique version of LibreSSL on my
> > mac years ago, I just installed OpenSSL and never looked back.
> >
> > Gary
> >
> > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
> > >
> > > <The question is whether to do anything about the previously undetected
> > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > be done for the current or a later release. If we don't fix this
> > > release, obviously it needs some mention in the release notes.>
> > >
> > > I wouldn't characterize the issues running against LibreSSL as
> > > "undetected". I also don't see this as an issue with Mac or Windows, but
> > > with LibreSSL. Install any supported OpenSSL library on any supported
> > > architecture (to include Mac and Windows) and all commons crypto
> > > functionality is available. I first encountered the rand issue you
> > > describe years ago when I was becoming familiar with commons crypto. I did
> > > a little research, discovered that I was running LibreSSL (and an old
> > > version at that), installed a supported version of OpenSSL and forgot
> > > all about it until this thread. I don't think it's unreasonable to expect
> > > users to install a supported version of OpenSSL on a supported architecture
> > > as a precondition of using commons crypto. I think it could become
> > > cumbersome to try and support every vendor default *SSL install. That
> > > said, I don't have a problem committing this particular "fix" to the
> > > baseline, particularly since you have already done the work. I just don't
> > > think that the project should obligate itself to do so or advertise
> > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> > > a "use at your own risk" approach to anything but OpenSSL proper. I agree
> > > that we should update the documentation to reflect whatever we move forward
> > > with.
> > >
> > > Anyway, that's my $0.02 worth.
> > >
> > > Alex
> > >
> > > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> > >
> > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > > > >
> > > > > We cannot remove support for Windows and macOS, that's silly.
> > > >
> > > > AFAICT that means we must support the different set of function names
> > > > in LibreSSL.
> > > > Note that we only currently use a small proportion of them.
> > > >
> > > > > I have not followed all the branches and commits, so I'm not sure what
> > > > the
> > > > > current problems are, but I know I was able to release all OSs last go
> > > > > around. I don't see why we need to rip out anything as fundamental.
> > > >
> > > > Well, I have tried running the Crypto and OpenSslJna main classes on
> > > > macOS and Windows, and they both fail with the 1.1.0 release.
> > > >
> > > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > > ENGINE_load_rdrand on both macOS and Windows.
> > > > (The job step succeeds, because the code catches the exception)
> > > >
> > > > It looks like the test which would have exposed at least one of the
> > > > issues was never enabled because of a failures on macOS; this hid the
> > > > same problem on Windows.
> > > >
> > > > I am not suggesting we rip out anything at this point.
> > > >
> > > > The question is whether to do anything about the previously undetected
> > > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > > be done for the current or a later release. If we don't fix this
> > > > release, obviously it needs some mention in the release notes.
> > > >
> > > > Sebb
> > > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > > knowledge
> > > > > > > we've never advertised LibreSSL support, so I don't see it as an
> > > > issue.
> > > > > >
> > > > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > > > Windows, as they both seem to default to LibreSSL.
> > > > > >
> > > > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > > > uses far fewer methods.
> > > > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > > > 1.1 names first, then a fallback.
> > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > > >
> > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > > different mix of names.
> > > > > >
> > > > > > I think it's vital we support JNI as far as possible (and the code
> > > > > > already does with commit 3ee3f65).
> > > > > >
> > > > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > > > working for macOS and Windows is less of an issue.
> > > > > >
> > > > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > > > >
> > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > The simple solution is to keep doing what we do now: only support
> > > > > > OpenSSL
> > > > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > > > be up
> > > > > > to
> > > > > > > > > date.
> > > > > > > >
> > > > > > > > I think this also affects Windows.
> > > > > > > >
> > > > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > > > Actions.
> > > > > > > >
> > > > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > > > >
> > > > > > > > mvn -q exec:java
> > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > > mvn -q exec:java
> > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > > >
> > > > > > > > The first one should show the SSL details; on GH the output
> > > > includes:
> > > > > > > >
> > > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > > >
> > > > > > > > The second test crashes with:
> > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > > > isEnabled(): false
> > > > > > > >
> > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > > > looking
> > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > > > not be
> > > > > > > > found.
> > > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > > ...
> > > > > > > >
> > > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > > >
> > > > > > > > > Gary
> > > > > > > > >
> > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > > > default
> > > > > > > > > > rather than OpenSSL.
> > > > > > > > > >
> > > > > > > > > > The LibreSSL API does not have the same functions as either
> > > > 1.0.2
> > > > > > or
> > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > > > 1.0.2
> > > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > > >
> > > > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > > > the
> > > > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > > > condition
> > > > > > > > > > in each of its methods.
> > > > > > > > > >
> > > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > > >
> > > > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > > > singleton classes that implement the required methods. The
> > > > > > appropriate
> > > > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > > > field
> > > > > > the
> > > > > > > > > > actual methods. i.e. replace the conditional logic with a
> > > > static
> > > > > > > > > > reference to the correct API interface instance. This should
> > > > only
> > > > > > > > > > affect non-public classes.
> > > > > > > > > >
> > > > > > > > > > Any other suggestions?
> > > > > > > > > >
> > > > > > > > > > Sebb
> > > > > > > > > >
> > > > > > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran
'mvn clean verify' and everything built just fine.
The Maven ant-run output was:
[INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto ---
[INFO] Executing tasks
make:
[exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
-classpath target/classes -o
target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h
org.apache.commons.crypto.random.OpenSslCryptoRandomNative
[exec] gcc -arch x86_64 -Ilib/inc_mac
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
-mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
-I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
-I"src/main/native/org/apache/commons/crypto/"
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
-I"target/jni-classes/org/apache/commons/crypto/cipher"
-I"target/jni-classes/org/apache/commons/crypto/random" -c
src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c
-o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
[exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
-classpath target/classes -o
target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h
org.apache.commons.crypto.cipher.OpenSslNative
[exec] gcc -arch x86_64 -Ilib/inc_mac
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
-mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
-I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
-I"src/main/native/org/apache/commons/crypto/"
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
-I"target/jni-classes/org/apache/commons/crypto/cipher"
-I"target/jni-classes/org/apache/commons/crypto/random" -c
src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o
target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
[exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force
-classpath target/classes -o
target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h
org.apache.commons.crypto.OpenSslInfoNative
[exec] gcc -arch x86_64 -Ilib/inc_mac
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
-mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
-I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
-I"src/main/native/org/apache/commons/crypto/"
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
-I"target/jni-classes/org/apache/commons/crypto/cipher"
-I"target/jni-classes/org/apache/commons/crypto/random"
-DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"'
-I"target/jni-classes/org/apache/commons/crypto" -c
src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o
target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o
[exec] gcc -arch x86_64 -Ilib/inc_mac
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC
-mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include
-I/usr/local/opt/openssl/include -Ilib/include -I/usr/include
-I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin"
-I"target/jni-classes/org/apache/commons/crypto/cipher"
-I"target/jni-classes/org/apache/commons/crypto/random" -o
target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o
target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o
target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib
-L/usr/local/lib
[exec] strip -x
target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
[exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
[exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib
target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib
Gary
On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <ga...@gmail.com> wrote:
>
> I agree with Alex.
>
> Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
> tight. Last time I checked I had an antique version of LibreSSL on my
> mac years ago, I just installed OpenSSL and never looked back.
>
> Gary
>
> On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
> >
> > <The question is whether to do anything about the previously undetected
> > issues with JNA on macOS and Windows, and if so, whether this needs to
> > be done for the current or a later release. If we don't fix this
> > release, obviously it needs some mention in the release notes.>
> >
> > I wouldn't characterize the issues running against LibreSSL as
> > "undetected". I also don't see this as an issue with Mac or Windows, but
> > with LibreSSL. Install any supported OpenSSL library on any supported
> > architecture (to include Mac and Windows) and all commons crypto
> > functionality is available. I first encountered the rand issue you
> > describe years ago when I was becoming familiar with commons crypto. I did
> > a little research, discovered that I was running LibreSSL (and an old
> > version at that), installed a supported version of OpenSSL and forgot
> > all about it until this thread. I don't think it's unreasonable to expect
> > users to install a supported version of OpenSSL on a supported architecture
> > as a precondition of using commons crypto. I think it could become
> > cumbersome to try and support every vendor default *SSL install. That
> > said, I don't have a problem committing this particular "fix" to the
> > baseline, particularly since you have already done the work. I just don't
> > think that the project should obligate itself to do so or advertise
> > LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> > a "use at your own risk" approach to anything but OpenSSL proper. I agree
> > that we should update the documentation to reflect whatever we move forward
> > with.
> >
> > Anyway, that's my $0.02 worth.
> >
> > Alex
> >
> > On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> >
> > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > > >
> > > > We cannot remove support for Windows and macOS, that's silly.
> > >
> > > AFAICT that means we must support the different set of function names
> > > in LibreSSL.
> > > Note that we only currently use a small proportion of them.
> > >
> > > > I have not followed all the branches and commits, so I'm not sure what
> > > the
> > > > current problems are, but I know I was able to release all OSs last go
> > > > around. I don't see why we need to rip out anything as fundamental.
> > >
> > > Well, I have tried running the Crypto and OpenSslJna main classes on
> > > macOS and Windows, and they both fail with the 1.1.0 release.
> > >
> > > With current master, Crypto succeeds, but OpenSslJna fails to find
> > > ENGINE_load_rdrand on both macOS and Windows.
> > > (The job step succeeds, because the code catches the exception)
> > >
> > > It looks like the test which would have exposed at least one of the
> > > issues was never enabled because of a failures on macOS; this hid the
> > > same problem on Windows.
> > >
> > > I am not suggesting we rip out anything at this point.
> > >
> > > The question is whether to do anything about the previously undetected
> > > issues with JNA on macOS and Windows, and if so, whether this needs to
> > > be done for the current or a later release. If we don't fix this
> > > release, obviously it needs some mention in the release notes.
> > >
> > > Sebb
> > > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > > Gary
> > > >
> > > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > > >
> > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > I agree with Gary that we just don't support LibreSSL. To my
> > > knowledge
> > > > > > we've never advertised LibreSSL support, so I don't see it as an
> > > issue.
> > > > >
> > > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > > Windows, as they both seem to default to LibreSSL.
> > > > >
> > > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > > uses far fewer methods.
> > > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > > 1.1 names first, then a fallback.
> > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > > >
> > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > > different mix of names.
> > > > >
> > > > > I think it's vital we support JNI as far as possible (and the code
> > > > > already does with commit 3ee3f65).
> > > > >
> > > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > > working for macOS and Windows is less of an issue.
> > > > >
> > > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > > >
> > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > The simple solution is to keep doing what we do now: only support
> > > > > OpenSSL
> > > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > > be up
> > > > > to
> > > > > > > > date.
> > > > > > >
> > > > > > > I think this also affects Windows.
> > > > > > >
> > > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > > Actions.
> > > > > > >
> > > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > > >
> > > > > > > mvn -q exec:java
> > > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > > mvn -q exec:java
> > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > > >
> > > > > > > The first one should show the SSL details; on GH the output
> > > includes:
> > > > > > >
> > > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > > Additional OpenSSL_version(n) details:
> > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > > >
> > > > > > > The second test crashes with:
> > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > > isEnabled(): false
> > > > > > >
> > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > > looking
> > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > > not be
> > > > > > > found.
> > > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > > ...
> > > > > > >
> > > > > > > It would certainly be easier to ignore the problem for now.
> > > > > > >
> > > > > > > > Gary
> > > > > > > >
> > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > > default
> > > > > > > > > rather than OpenSSL.
> > > > > > > > >
> > > > > > > > > The LibreSSL API does not have the same functions as either
> > > 1.0.2
> > > > > or
> > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > > 1.0.2
> > > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > > >
> > > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > > the
> > > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > > condition
> > > > > > > > > in each of its methods.
> > > > > > > > >
> > > > > > > > > This is quite tedious and error-prone.
> > > > > > > > >
> > > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > > singleton classes that implement the required methods. The
> > > > > appropriate
> > > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > > field
> > > > > the
> > > > > > > > > actual methods. i.e. replace the conditional logic with a
> > > static
> > > > > > > > > reference to the correct API interface instance. This should
> > > only
> > > > > > > > > affect non-public classes.
> > > > > > > > >
> > > > > > > > > Any other suggestions?
> > > > > > > > >
> > > > > > > > > Sebb
> > > > > > > > >
> > > > > > > > >
> > > > > ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
I agree with Alex.
Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and
tight. Last time I checked I had an antique version of LibreSSL on my
mac years ago, I just installed OpenSSL and never looked back.
Gary
On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <al...@gmail.com> wrote:
>
> <The question is whether to do anything about the previously undetected
> issues with JNA on macOS and Windows, and if so, whether this needs to
> be done for the current or a later release. If we don't fix this
> release, obviously it needs some mention in the release notes.>
>
> I wouldn't characterize the issues running against LibreSSL as
> "undetected". I also don't see this as an issue with Mac or Windows, but
> with LibreSSL. Install any supported OpenSSL library on any supported
> architecture (to include Mac and Windows) and all commons crypto
> functionality is available. I first encountered the rand issue you
> describe years ago when I was becoming familiar with commons crypto. I did
> a little research, discovered that I was running LibreSSL (and an old
> version at that), installed a supported version of OpenSSL and forgot
> all about it until this thread. I don't think it's unreasonable to expect
> users to install a supported version of OpenSSL on a supported architecture
> as a precondition of using commons crypto. I think it could become
> cumbersome to try and support every vendor default *SSL install. That
> said, I don't have a problem committing this particular "fix" to the
> baseline, particularly since you have already done the work. I just don't
> think that the project should obligate itself to do so or advertise
> LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
> a "use at your own risk" approach to anything but OpenSSL proper. I agree
> that we should update the documentation to reflect whatever we move forward
> with.
>
> Anyway, that's my $0.02 worth.
>
> Alex
>
> On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
>
> > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> > >
> > > We cannot remove support for Windows and macOS, that's silly.
> >
> > AFAICT that means we must support the different set of function names
> > in LibreSSL.
> > Note that we only currently use a small proportion of them.
> >
> > > I have not followed all the branches and commits, so I'm not sure what
> > the
> > > current problems are, but I know I was able to release all OSs last go
> > > around. I don't see why we need to rip out anything as fundamental.
> >
> > Well, I have tried running the Crypto and OpenSslJna main classes on
> > macOS and Windows, and they both fail with the 1.1.0 release.
> >
> > With current master, Crypto succeeds, but OpenSslJna fails to find
> > ENGINE_load_rdrand on both macOS and Windows.
> > (The job step succeeds, because the code catches the exception)
> >
> > It looks like the test which would have exposed at least one of the
> > issues was never enabled because of a failures on macOS; this hid the
> > same problem on Windows.
> >
> > I am not suggesting we rip out anything at this point.
> >
> > The question is whether to do anything about the previously undetected
> > issues with JNA on macOS and Windows, and if so, whether this needs to
> > be done for the current or a later release. If we don't fix this
> > release, obviously it needs some mention in the release notes.
> >
> > Sebb
> > https://github.com/apache/commons-crypto/actions/runs/2586011129
> > > Gary
> > >
> > > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> > >
> > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> > wrote:
> > > > >
> > > > > I agree with Gary that we just don't support LibreSSL. To my
> > knowledge
> > > > > we've never advertised LibreSSL support, so I don't see it as an
> > issue.
> > > >
> > > > In that case AFAICT we will have to drop *all* support for macOS and
> > > > Windows, as they both seem to default to LibreSSL.
> > > >
> > > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > > uses far fewer methods.
> > > > Rather than checking the version, I changed the code to try OpenSSL
> > > > 1.1 names first, then a fallback.
> > > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > > >
> > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > > different mix of names.
> > > >
> > > > I think it's vital we support JNI as far as possible (and the code
> > > > already does with commit 3ee3f65).
> > > >
> > > > However JNA is more of a backstop, so the fact that it has stopped
> > > > working for macOS and Windows is less of an issue.
> > > >
> > > > But I don't think we can say we are not supporting LibreSSL at all.
> > > >
> > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> > >
> > > > wrote:
> > > > > > >
> > > > > > > The simple solution is to keep doing what we do now: only support
> > > > OpenSSL
> > > > > > > and not whatever Apple does with LibreSSL which may or may not
> > be up
> > > > to
> > > > > > > date.
> > > > > >
> > > > > > I think this also affects Windows.
> > > > > >
> > > > > > I don't have Windows box at present, but I have seen this on GH
> > > > Actions.
> > > > > >
> > > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > > >
> > > > > > mvn -q exec:java
> > -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > > mvn -q exec:java
> > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > > >
> > > > > > The first one should show the SSL details; on GH the output
> > includes:
> > > > > >
> > > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > > Additional OpenSSL_version(n) details:
> > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > > >
> > > > > > The second test crashes with:
> > > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > > isEnabled(): false
> > > > > >
> > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> > looking
> > > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> > not be
> > > > > > found.
> > > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > > ...
> > > > > >
> > > > > > It would certainly be easier to ignore the problem for now.
> > > > > >
> > > > > > > Gary
> > > > > > >
> > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > > >
> > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > > default
> > > > > > > > rather than OpenSSL.
> > > > > > > >
> > > > > > > > The LibreSSL API does not have the same functions as either
> > 1.0.2
> > > > or
> > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > > 1.0.2
> > > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > > >
> > > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > > It would require another OpenSsl<version>NativeJna class, and
> > the
> > > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > > condition
> > > > > > > > in each of its methods.
> > > > > > > >
> > > > > > > > This is quite tedious and error-prone.
> > > > > > > >
> > > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > > singleton classes that implement the required methods. The
> > > > appropriate
> > > > > > > > one can then be initialised and used by OpenSslNativeJna to
> > field
> > > > the
> > > > > > > > actual methods. i.e. replace the conditional logic with a
> > static
> > > > > > > > reference to the correct API interface instance. This should
> > only
> > > > > > > > affect non-public classes.
> > > > > > > >
> > > > > > > > Any other suggestions?
> > > > > > > >
> > > > > > > > Sebb
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
<The question is whether to do anything about the previously undetected
issues with JNA on macOS and Windows, and if so, whether this needs to
be done for the current or a later release. If we don't fix this
release, obviously it needs some mention in the release notes.>
I wouldn't characterize the issues running against LibreSSL as
"undetected". I also don't see this as an issue with Mac or Windows, but
with LibreSSL. Install any supported OpenSSL library on any supported
architecture (to include Mac and Windows) and all commons crypto
functionality is available. I first encountered the rand issue you
describe years ago when I was becoming familiar with commons crypto. I did
a little research, discovered that I was running LibreSSL (and an old
version at that), installed a supported version of OpenSSL and forgot
all about it until this thread. I don't think it's unreasonable to expect
users to install a supported version of OpenSSL on a supported architecture
as a precondition of using commons crypto. I think it could become
cumbersome to try and support every vendor default *SSL install. That
said, I don't have a problem committing this particular "fix" to the
baseline, particularly since you have already done the work. I just don't
think that the project should obligate itself to do so or advertise
LibreSSL (or any other non-OpenSSL branch) support as such. I'm advocating
a "use at your own risk" approach to anything but OpenSSL proper. I agree
that we should update the documentation to reflect whatever we move forward
with.
Anyway, that's my $0.02 worth.
Alex
On Wed, Jun 29, 2022 at 6:14 PM sebb <se...@gmail.com> wrote:
> On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
> >
> > We cannot remove support for Windows and macOS, that's silly.
>
> AFAICT that means we must support the different set of function names
> in LibreSSL.
> Note that we only currently use a small proportion of them.
>
> > I have not followed all the branches and commits, so I'm not sure what
> the
> > current problems are, but I know I was able to release all OSs last go
> > around. I don't see why we need to rip out anything as fundamental.
>
> Well, I have tried running the Crypto and OpenSslJna main classes on
> macOS and Windows, and they both fail with the 1.1.0 release.
>
> With current master, Crypto succeeds, but OpenSslJna fails to find
> ENGINE_load_rdrand on both macOS and Windows.
> (The job step succeeds, because the code catches the exception)
>
> It looks like the test which would have exposed at least one of the
> issues was never enabled because of a failures on macOS; this hid the
> same problem on Windows.
>
> I am not suggesting we rip out anything at this point.
>
> The question is whether to do anything about the previously undetected
> issues with JNA on macOS and Windows, and if so, whether this needs to
> be done for the current or a later release. If we don't fix this
> release, obviously it needs some mention in the release notes.
>
> Sebb
> https://github.com/apache/commons-crypto/actions/runs/2586011129
> > Gary
> >
> > On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> >
> > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com>
> wrote:
> > > >
> > > > I agree with Gary that we just don't support LibreSSL. To my
> knowledge
> > > > we've never advertised LibreSSL support, so I don't see it as an
> issue.
> > >
> > > In that case AFAICT we will have to drop *all* support for macOS and
> > > Windows, as they both seem to default to LibreSSL.
> > >
> > > Note that adding support for LibreSSL was much easier for JNI, as it
> > > uses far fewer methods.
> > > Rather than checking the version, I changed the code to try OpenSSL
> > > 1.1 names first, then a fallback.
> > > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> > >
> > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > > macOS fails on 16345bc because it now uses LibreSSL which has a
> > > different mix of names.
> > >
> > > I think it's vital we support JNI as far as possible (and the code
> > > already does with commit 3ee3f65).
> > >
> > > However JNA is more of a backstop, so the fact that it has stopped
> > > working for macOS and Windows is less of an issue.
> > >
> > > But I don't think we can say we are not supporting LibreSSL at all.
> > >
> > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > > >
> > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <garydgregory@gmail.com
> >
> > > wrote:
> > > > > >
> > > > > > The simple solution is to keep doing what we do now: only support
> > > OpenSSL
> > > > > > and not whatever Apple does with LibreSSL which may or may not
> be up
> > > to
> > > > > > date.
> > > > >
> > > > > I think this also affects Windows.
> > > > >
> > > > > I don't have Windows box at present, but I have seen this on GH
> > > Actions.
> > > > >
> > > > > If you have a WIndows build, perhaps you could try these tests:
> > > > >
> > > > > mvn -q exec:java
> -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > > mvn -q exec:java
> > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > > >
> > > > > The first one should show the SSL details; on GH the output
> includes:
> > > > >
> > > > > OpenSSL library loaded OK, version: 0x20000000
> > > > > OpenSSL library info: LibreSSL 3.0.2
> > > > > Additional OpenSSL_version(n) details:
> > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > > >
> > > > > The second test crashes with:
> > > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > > isEnabled(): false
> > > > >
> > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error
> looking
> > > > > up function 'ENGINE_load_rdrand': The specified procedure could
> not be
> > > > > found.
> > > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > > ...
> > > > >
> > > > > It would certainly be easier to ignore the problem for now.
> > > > >
> > > > > > Gary
> > > > > >
> > > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > > >
> > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > > default
> > > > > > > rather than OpenSSL.
> > > > > > >
> > > > > > > The LibreSSL API does not have the same functions as either
> 1.0.2
> > > or
> > > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > > 1.0.2
> > > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > > >
> > > > > > > This makes it hard to support JNA with the current design.
> > > > > > > It would require another OpenSsl<version>NativeJna class, and
> the
> > > > > > > parent class OpenSslNativeJna would need to use yet another
> > > condition
> > > > > > > in each of its methods.
> > > > > > >
> > > > > > > This is quite tedious and error-prone.
> > > > > > >
> > > > > > > Seems to me it would be better to use something like a set of
> > > > > > > singleton classes that implement the required methods. The
> > > appropriate
> > > > > > > one can then be initialised and used by OpenSslNativeJna to
> field
> > > the
> > > > > > > actual methods. i.e. replace the conditional logic with a
> static
> > > > > > > reference to the correct API interface instance. This should
> only
> > > > > > > affect non-public classes.
> > > > > > >
> > > > > > > Any other suggestions?
> > > > > > >
> > > > > > > Sebb
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Wed, 29 Jun 2022 at 18:06, Gary Gregory <ga...@gmail.com> wrote:
>
> We cannot remove support for Windows and macOS, that's silly.
AFAICT that means we must support the different set of function names
in LibreSSL.
Note that we only currently use a small proportion of them.
> I have not followed all the branches and commits, so I'm not sure what the
> current problems are, but I know I was able to release all OSs last go
> around. I don't see why we need to rip out anything as fundamental.
Well, I have tried running the Crypto and OpenSslJna main classes on
macOS and Windows, and they both fail with the 1.1.0 release.
With current master, Crypto succeeds, but OpenSslJna fails to find
ENGINE_load_rdrand on both macOS and Windows.
(The job step succeeds, because the code catches the exception)
It looks like the test which would have exposed at least one of the
issues was never enabled because of a failures on macOS; this hid the
same problem on Windows.
I am not suggesting we rip out anything at this point.
The question is whether to do anything about the previously undetected
issues with JNA on macOS and Windows, and if so, whether this needs to
be done for the current or a later release. If we don't fix this
release, obviously it needs some mention in the release notes.
Sebb
https://github.com/apache/commons-crypto/actions/runs/2586011129
> Gary
>
> On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
>
> > On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com> wrote:
> > >
> > > I agree with Gary that we just don't support LibreSSL. To my knowledge
> > > we've never advertised LibreSSL support, so I don't see it as an issue.
> >
> > In that case AFAICT we will have to drop *all* support for macOS and
> > Windows, as they both seem to default to LibreSSL.
> >
> > Note that adding support for LibreSSL was much easier for JNI, as it
> > uses far fewer methods.
> > Rather than checking the version, I changed the code to try OpenSSL
> > 1.1 names first, then a fallback.
> > That happens to work for 1.0.x and 1.1.x and LibreSSL.
> >
> > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> > macOS fails on 16345bc because it now uses LibreSSL which has a
> > different mix of names.
> >
> > I think it's vital we support JNI as far as possible (and the code
> > already does with commit 3ee3f65).
> >
> > However JNA is more of a backstop, so the fact that it has stopped
> > working for macOS and Windows is less of an issue.
> >
> > But I don't think we can say we are not supporting LibreSSL at all.
> >
> > > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> > >
> > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <ga...@gmail.com>
> > wrote:
> > > > >
> > > > > The simple solution is to keep doing what we do now: only support
> > OpenSSL
> > > > > and not whatever Apple does with LibreSSL which may or may not be up
> > to
> > > > > date.
> > > >
> > > > I think this also affects Windows.
> > > >
> > > > I don't have Windows box at present, but I have seen this on GH
> > Actions.
> > > >
> > > > If you have a WIndows build, perhaps you could try these tests:
> > > >
> > > > mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > > mvn -q exec:java
> > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > > >
> > > > The first one should show the SSL details; on GH the output includes:
> > > >
> > > > OpenSSL library loaded OK, version: 0x20000000
> > > > OpenSSL library info: LibreSSL 3.0.2
> > > > Additional OpenSSL_version(n) details:
> > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > > >
> > > > The second test crashes with:
> > > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > > isEnabled(): false
> > > >
> > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error looking
> > > > up function 'ENGINE_load_rdrand': The specified procedure could not be
> > > > found.
> > > > at com.sun.jna.Function.<init>(Function.java:252)
> > > > ...
> > > >
> > > > It would certainly be easier to ignore the problem for now.
> > > >
> > > > > Gary
> > > > >
> > > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> > default
> > > > > > rather than OpenSSL.
> > > > > >
> > > > > > The LibreSSL API does not have the same functions as either 1.0.2
> > or
> > > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> > 1.0.2
> > > > > > has the former only, and 1.1.1 has the latter only.
> > > > > >
> > > > > > This makes it hard to support JNA with the current design.
> > > > > > It would require another OpenSsl<version>NativeJna class, and the
> > > > > > parent class OpenSslNativeJna would need to use yet another
> > condition
> > > > > > in each of its methods.
> > > > > >
> > > > > > This is quite tedious and error-prone.
> > > > > >
> > > > > > Seems to me it would be better to use something like a set of
> > > > > > singleton classes that implement the required methods. The
> > appropriate
> > > > > > one can then be initialised and used by OpenSslNativeJna to field
> > the
> > > > > > actual methods. i.e. replace the conditional logic with a static
> > > > > > reference to the correct API interface instance. This should only
> > > > > > affect non-public classes.
> > > > > >
> > > > > > Any other suggestions?
> > > > > >
> > > > > > Sebb
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
We cannot remove support for Windows and macOS, that's silly.
I have not followed all the branches and commits, so I'm not sure what the
current problems are, but I know I was able to release all OSs last go
around. I don't see why we need to rip out anything as fundamental.
Gary
On Wed, Jun 29, 2022, 12:00 sebb <se...@gmail.com> wrote:
> On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com> wrote:
> >
> > I agree with Gary that we just don't support LibreSSL. To my knowledge
> > we've never advertised LibreSSL support, so I don't see it as an issue.
>
> In that case AFAICT we will have to drop *all* support for macOS and
> Windows, as they both seem to default to LibreSSL.
>
> Note that adding support for LibreSSL was much easier for JNI, as it
> uses far fewer methods.
> Rather than checking the version, I changed the code to try OpenSSL
> 1.1 names first, then a fallback.
> That happens to work for 1.0.x and 1.1.x and LibreSSL.
>
> If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
> macOS fails on 16345bc because it now uses LibreSSL which has a
> different mix of names.
>
> I think it's vital we support JNI as far as possible (and the code
> already does with commit 3ee3f65).
>
> However JNA is more of a backstop, so the fact that it has stopped
> working for macOS and Windows is less of an issue.
>
> But I don't think we can say we are not supporting LibreSSL at all.
>
> > On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> >
> > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <ga...@gmail.com>
> wrote:
> > > >
> > > > The simple solution is to keep doing what we do now: only support
> OpenSSL
> > > > and not whatever Apple does with LibreSSL which may or may not be up
> to
> > > > date.
> > >
> > > I think this also affects Windows.
> > >
> > > I don't have Windows box at present, but I have seen this on GH
> Actions.
> > >
> > > If you have a WIndows build, perhaps you could try these tests:
> > >
> > > mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > > mvn -q exec:java
> > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> > >
> > > The first one should show the SSL details; on GH the output includes:
> > >
> > > OpenSSL library loaded OK, version: 0x20000000
> > > OpenSSL library info: LibreSSL 3.0.2
> > > Additional OpenSSL_version(n) details:
> > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> > >
> > > The second test crashes with:
> > > java.lang.UnsatisfiedLinkError: Error looking up function
> > > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > > isEnabled(): false
> > >
> > > initialisationError(): java.lang.UnsatisfiedLinkError: Error looking
> > > up function 'ENGINE_load_rdrand': The specified procedure could not be
> > > found.
> > > at com.sun.jna.Function.<init>(Function.java:252)
> > > ...
> > >
> > > It would certainly be easier to ignore the problem for now.
> > >
> > > > Gary
> > > >
> > > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > > >
> > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by
> default
> > > > > rather than OpenSSL.
> > > > >
> > > > > The LibreSSL API does not have the same functions as either 1.0.2
> or
> > > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num;
> 1.0.2
> > > > > has the former only, and 1.1.1 has the latter only.
> > > > >
> > > > > This makes it hard to support JNA with the current design.
> > > > > It would require another OpenSsl<version>NativeJna class, and the
> > > > > parent class OpenSslNativeJna would need to use yet another
> condition
> > > > > in each of its methods.
> > > > >
> > > > > This is quite tedious and error-prone.
> > > > >
> > > > > Seems to me it would be better to use something like a set of
> > > > > singleton classes that implement the required methods. The
> appropriate
> > > > > one can then be initialised and used by OpenSslNativeJna to field
> the
> > > > > actual methods. i.e. replace the conditional logic with a static
> > > > > reference to the correct API interface instance. This should only
> > > > > affect non-public classes.
> > > > >
> > > > > Any other suggestions?
> > > > >
> > > > > Sebb
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Wed, 29 Jun 2022 at 16:11, Alex Remily <al...@gmail.com> wrote:
>
> I agree with Gary that we just don't support LibreSSL. To my knowledge
> we've never advertised LibreSSL support, so I don't see it as an issue.
In that case AFAICT we will have to drop *all* support for macOS and
Windows, as they both seem to default to LibreSSL.
Note that adding support for LibreSSL was much easier for JNI, as it
uses far fewer methods.
Rather than checking the version, I changed the code to try OpenSSL
1.1 names first, then a fallback.
That happens to work for 1.0.x and 1.1.x and LibreSSL.
If you want to try it out, compare 16345bc (old) with 3ee3f65 (new)
macOS fails on 16345bc because it now uses LibreSSL which has a
different mix of names.
I think it's vital we support JNI as far as possible (and the code
already does with commit 3ee3f65).
However JNA is more of a backstop, so the fact that it has stopped
working for macOS and Windows is less of an issue.
But I don't think we can say we are not supporting LibreSSL at all.
> On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
>
> > On Wed, 29 Jun 2022 at 14:17, Gary Gregory <ga...@gmail.com> wrote:
> > >
> > > The simple solution is to keep doing what we do now: only support OpenSSL
> > > and not whatever Apple does with LibreSSL which may or may not be up to
> > > date.
> >
> > I think this also affects Windows.
> >
> > I don't have Windows box at present, but I have seen this on GH Actions.
> >
> > If you have a WIndows build, perhaps you could try these tests:
> >
> > mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> > mvn -q exec:java
> > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
> >
> > The first one should show the SSL details; on GH the output includes:
> >
> > OpenSSL library loaded OK, version: 0x20000000
> > OpenSSL library info: LibreSSL 3.0.2
> > Additional OpenSSL_version(n) details:
> > 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
> >
> > The second test crashes with:
> > java.lang.UnsatisfiedLinkError: Error looking up function
> > 'ENGINE_load_rdrand': The specified procedure could not be found.
> > isEnabled(): false
> >
> > initialisationError(): java.lang.UnsatisfiedLinkError: Error looking
> > up function 'ENGINE_load_rdrand': The specified procedure could not be
> > found.
> > at com.sun.jna.Function.<init>(Function.java:252)
> > ...
> >
> > It would certainly be easier to ignore the problem for now.
> >
> > > Gary
> > >
> > > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> > >
> > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by default
> > > > rather than OpenSSL.
> > > >
> > > > The LibreSSL API does not have the same functions as either 1.0.2 or
> > > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num; 1.0.2
> > > > has the former only, and 1.1.1 has the latter only.
> > > >
> > > > This makes it hard to support JNA with the current design.
> > > > It would require another OpenSsl<version>NativeJna class, and the
> > > > parent class OpenSslNativeJna would need to use yet another condition
> > > > in each of its methods.
> > > >
> > > > This is quite tedious and error-prone.
> > > >
> > > > Seems to me it would be better to use something like a set of
> > > > singleton classes that implement the required methods. The appropriate
> > > > one can then be initialised and used by OpenSslNativeJna to field the
> > > > actual methods. i.e. replace the conditional logic with a static
> > > > reference to the correct API interface instance. This should only
> > > > affect non-public classes.
> > > >
> > > > Any other suggestions?
> > > >
> > > > Sebb
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: dev-help@commons.apache.org
> > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Alex Remily <al...@gmail.com>.
I agree with Gary that we just don't support LibreSSL. To my knowledge
we've never advertised LibreSSL support, so I don't see it as an issue.
On Wed, Jun 29, 2022 at 10:21 AM sebb <se...@gmail.com> wrote:
> On Wed, 29 Jun 2022 at 14:17, Gary Gregory <ga...@gmail.com> wrote:
> >
> > The simple solution is to keep doing what we do now: only support OpenSSL
> > and not whatever Apple does with LibreSSL which may or may not be up to
> > date.
>
> I think this also affects Windows.
>
> I don't have Windows box at present, but I have seen this on GH Actions.
>
> If you have a WIndows build, perhaps you could try these tests:
>
> mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.Crypto"
> mvn -q exec:java
> -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
>
> The first one should show the SSL details; on GH the output includes:
>
> OpenSSL library loaded OK, version: 0x20000000
> OpenSSL library info: LibreSSL 3.0.2
> Additional OpenSSL_version(n) details:
> 4: OPENSSLDIR: "C:/Windows/libressl/ssl"
>
> The second test crashes with:
> java.lang.UnsatisfiedLinkError: Error looking up function
> 'ENGINE_load_rdrand': The specified procedure could not be found.
> isEnabled(): false
>
> initialisationError(): java.lang.UnsatisfiedLinkError: Error looking
> up function 'ENGINE_load_rdrand': The specified procedure could not be
> found.
> at com.sun.jna.Function.<init>(Function.java:252)
> ...
>
> It would certainly be easier to ignore the problem for now.
>
> > Gary
> >
> > On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> >
> > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by default
> > > rather than OpenSSL.
> > >
> > > The LibreSSL API does not have the same functions as either 1.0.2 or
> > > 1.1.1, so needs its own JNA class. In particular it looks like
> > > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num; 1.0.2
> > > has the former only, and 1.1.1 has the latter only.
> > >
> > > This makes it hard to support JNA with the current design.
> > > It would require another OpenSsl<version>NativeJna class, and the
> > > parent class OpenSslNativeJna would need to use yet another condition
> > > in each of its methods.
> > >
> > > This is quite tedious and error-prone.
> > >
> > > Seems to me it would be better to use something like a set of
> > > singleton classes that implement the required methods. The appropriate
> > > one can then be initialised and used by OpenSslNativeJna to field the
> > > actual methods. i.e. replace the conditional logic with a static
> > > reference to the correct API interface instance. This should only
> > > affect non-public classes.
> > >
> > > Any other suggestions?
> > >
> > > Sebb
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by sebb <se...@gmail.com>.
On Wed, 29 Jun 2022 at 14:17, Gary Gregory <ga...@gmail.com> wrote:
>
> The simple solution is to keep doing what we do now: only support OpenSSL
> and not whatever Apple does with LibreSSL which may or may not be up to
> date.
I think this also affects Windows.
I don't have Windows box at present, but I have seen this on GH Actions.
If you have a WIndows build, perhaps you could try these tests:
mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.Crypto"
mvn -q exec:java -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna"
The first one should show the SSL details; on GH the output includes:
OpenSSL library loaded OK, version: 0x20000000
OpenSSL library info: LibreSSL 3.0.2
Additional OpenSSL_version(n) details:
4: OPENSSLDIR: "C:/Windows/libressl/ssl"
The second test crashes with:
java.lang.UnsatisfiedLinkError: Error looking up function
'ENGINE_load_rdrand': The specified procedure could not be found.
isEnabled(): false
initialisationError(): java.lang.UnsatisfiedLinkError: Error looking
up function 'ENGINE_load_rdrand': The specified procedure could not be
found.
at com.sun.jna.Function.<init>(Function.java:252)
...
It would certainly be easier to ignore the problem for now.
> Gary
>
> On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
>
> > It looks like macOS 10.5+ and Windows (latest) use LibreSSL by default
> > rather than OpenSSL.
> >
> > The LibreSSL API does not have the same functions as either 1.0.2 or
> > 1.1.1, so needs its own JNA class. In particular it looks like
> > ENGINE_load_rdrand is not present, nor is OpenSSL_version_num; 1.0.2
> > has the former only, and 1.1.1 has the latter only.
> >
> > This makes it hard to support JNA with the current design.
> > It would require another OpenSsl<version>NativeJna class, and the
> > parent class OpenSslNativeJna would need to use yet another condition
> > in each of its methods.
> >
> > This is quite tedious and error-prone.
> >
> > Seems to me it would be better to use something like a set of
> > singleton classes that implement the required methods. The appropriate
> > one can then be initialised and used by OpenSslNativeJna to field the
> > actual methods. i.e. replace the conditional logic with a static
> > reference to the correct API interface instance. This should only
> > affect non-public classes.
> >
> > Any other suggestions?
> >
> > Sebb
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [CRYPTO] Problem with JNA on macOS and Windows
Posted by Gary Gregory <ga...@gmail.com>.
The simple solution is to keep doing what we do now: only support OpenSSL
and not whatever Apple does with LibreSSL which may or may not be up to
date.
Gary
On Wed, Jun 29, 2022, 06:59 sebb <se...@gmail.com> wrote:
> It looks like macOS 10.5+ and Windows (latest) use LibreSSL by default
> rather than OpenSSL.
>
> The LibreSSL API does not have the same functions as either 1.0.2 or
> 1.1.1, so needs its own JNA class. In particular it looks like
> ENGINE_load_rdrand is not present, nor is OpenSSL_version_num; 1.0.2
> has the former only, and 1.1.1 has the latter only.
>
> This makes it hard to support JNA with the current design.
> It would require another OpenSsl<version>NativeJna class, and the
> parent class OpenSslNativeJna would need to use yet another condition
> in each of its methods.
>
> This is quite tedious and error-prone.
>
> Seems to me it would be better to use something like a set of
> singleton classes that implement the required methods. The appropriate
> one can then be initialised and used by OpenSslNativeJna to field the
> actual methods. i.e. replace the conditional logic with a static
> reference to the correct API interface instance. This should only
> affect non-public classes.
>
> Any other suggestions?
>
> Sebb
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>