You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by James Duong <ja...@bitquilltech.com> on 2020/10/02 08:21:52 UTC
Re: [FlightRPC][Java][Python/c_lib] Encryption without validation in
Flight clients
Hi David,
I've created a WIP PR illustrating what I'm planning to do. However I'm
still trying to figure out a few things:
- In my environment, gRPC is coming from miniconda3 for C++, and seems
to be an older version. In more recent versions, gRPC has an experimental
API TlsCredentials, which has the option to disable cert validation. Is it
normal that this comes from miniconda, or is it just my setup?
- See:
https://github.com/grpc/grpc/blob/a6a5cbfee2ff2709a8bc66c3d4b428db48ae46af/src/cpp/common/tls_credentials_options.cc#L284
. This shows up starting with 1.30.x
- Where do we specify which version of gRPC we depend on? In java
it's in maven and looks to be 1.30.2.
- The Java implementation looks fairly straightforward.
- Python implementation is pending.
- Tests are pending.
On Wed, Sep 30, 2020 at 8:12 AM James Duong <ja...@bitquilltech.com> wrote:
> Will do David.
>
> I've created a JIRA for the TLS_SNI extension as well:
> https://issues.apache.org/jira/browse/ARROW-10144
>
> On Wed, Sep 30, 2020 at 7:52 AM David Li <li...@gmail.com> wrote:
>
>> Hi James,
>>
>> Please feel free to tag me as a reviewer if needed.
>>
>> Best,
>> David
>>
>> On 9/28/20, James Duong <ja...@bitquilltech.com> wrote:
>> > Hi Arrow-Dev,
>> >
>> > I've created a JIRA for supporting using Flight clients to connect to
>> > Flight servers without verifying the server certificate.
>> >
>> > This is something I've seen done frequently for other connectivity tools
>> > such as JDBC drivers.
>> >
>> > I'll be looking to implement this one for the and hopefully get it in
>> for
>> > the next release:
>> > https://issues.apache.org/jira/plugins/servlet/mobile#issue/ARROW-10105
>> >
>> > Related, I'm also planning to add support for using the TLS_SNI
>> extension.
>> >
>>
>
>
> --
>
> *James Duong*
> Lead Software Developer
> Bit Quill Technologies Inc.
> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> https://www.bitquilltech.com
>
> This email message is for the sole use of the intended recipient(s) and
> may contain confidential and privileged information. Any unauthorized
> review, use, disclosure, or distribution is prohibited. If you are not the
> intended recipient, please contact the sender by reply email and destroy
> all copies of the original message. Thank you.
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.
Re: [FlightRPC][Java][Python/c_lib] Encryption without validation in
Flight clients
Posted by James Duong <ja...@bitquilltech.com>.
Regarding #2, what I've found now is that gRPC basically removed what
-DGPR_MANYLINUX1=1
does to try to collapse the case into the normal Linux case during the
upgrade to 1.32:
https://github.com/grpc/grpc/commit/23e1d30327923ac93beaeb6cdcfb0eef9be759c4
I'm currently trying out turning off platform autodetection when building
gRPC and manually
redefining the macro definitions that were removed in the above commit.
On Mon, Oct 5, 2020 at 5:07 PM David Li <li...@gmail.com> wrote:
> For #1, can we embed a dummy root to make gRPC happy, since we're only
> using it in the case that we're disabling validation? (Also, is there
> a bug report for gRPC/would you like to file one?)
>
> For #2, I think you can get cmake to pass -D flags, perhaps you can
> get it to define a symbol you can check (though that is maybe not a
> preferable solution).
>
> David
>
> On 10/5/20, James Duong <ja...@bitquilltech.com> wrote:
> > An update about this issue (also on JIRA at
> > https://issues.apache.org/jira/browse/ARROW-10105)
> > Hoping for some suggestions on how to deal with C++ issues with gRPC
> > versioning.
> >
> > I have the Java version of this feature working and tested.
> >
> > The C++ client has some challenges:
> > The experimental TlsCredentials interface has an option to specify that
> you
> > want to disable server certificate checks. You also must supply
> > a callback that is supposed to represent how you the client should verify
> > the server instead. There are two problems with this interface
> > though:
> > 1. The caller needs to supply a CA roots PEM file to this. This would be
> > unexpected since the point of using this is to skip over
> > passing in certificate information. Furthermore, the SslCredentials
> > interface that's currently being used for SSL automatically
> > chooses a default CA roots PEM file if none is supplied. gRPC ships one
> as
> > part of its installation process that it uses.
> > There does not appear to be a programmatic way to get to the path to the
> > shipped with gRPC. We may need to supply
> > a default/dummy PEM file for the purpose of skipping server cert
> > verification. I've verified that if I do supply
> > a PEM file, everything works correctly and the server verification gets
> > skipped. The Python client changes associated
> > with this also work.
> >
> > 2. The TlsCredentials class and related Options, etc classes are in a
> > different namespace depending on gRPC version.
> > It first shows up in gRPC 1.25 in the grpc_impl::experimental namespace.
> It
> > got moved to grpc::experimental namespace
> > starting in gRPC 1.32. What's problematic is that we can't get the gRPC
> > version at compile-time, so we can't choose
> > the appropriate namespace for looking these up. My preference is to just
> > switch entirely to 1.32, however I'm
> > seeing issues in some CI jobs that don't seem to correctly use 1.32 such
> > as:
> > https://ci.ursalabs.org/#/builders/66/builds/11440
> > The CentOS 5.11 build also can't seem to build gRPC 1.32. It fails
> > compiling some code in linux/tcp.h, which
> > may be related to using an older Linux kernel:
> > https://github.com/apache/arrow/pull/8325/checks?check_run_id=1208358439
> ):
> >
> >
> CMakeFiles/grpc_unsecure.dir/src/core/lib/iomgr/socket_utils_common_posix.cc.o
> > -c src/core/lib/iomgr/socket_utils_common_posix.cc
> > In file included from /usr/include/asm-x86_64/byteorder.h:30:0,
> > from /usr/include/asm/byteorder.h:5,
> > from /usr/include/linux/tcp.h:21,
> > from src/core/lib/iomgr/socket_utils_common_posix.cc:34:
> > /usr/include/linux/byteorder/little_endian.h:43:19: error: ‘__le64’
> > does not name a type
> > static __inline__ __le64 __cpu_to_le64p(const __u64 *p)
> > ^
> > /usr/include/linux/byteorder/little_endian.h:47:46: error: ‘__le64’
> > does not name a type
> > static __inline__ __u64 __le64_to_cpup(const __le64 *p)
> > ^
> > /usr/include/linux/byteorder/little_endian.h:67:19: error: ‘__be64’
> > does not name a type
> > static __inline__ __be64 __cpu_to_be64p(const __u64 *p)
> > ^
> > /usr/include/linux/byteorder/little_endian.h:71:46: error: ‘__be64’
> > does not name a type
> > static __inline__ __u64 __be64_to_cpup(const __be64 *p)
> >
> >
> >
> > On Fri, Oct 2, 2020 at 5:36 AM David Li <li...@gmail.com> wrote:
> >
> >> Hey James,
> >>
> >> Where gRPC comes from depends on your setup; the C++ development
> >> guide[1] describes it all, but I personally use Conda with conda-forge
> >> which gets me gRPC 1.32.
> >>
> >> The dependency version again depends on where you're looking - for
> >> bundled builds it's set in thirdparty/versions.txt[2], for the other
> >> CI builds it's set in various config files.
> >>
> >> Best,
> >> David
> >>
> >> [1]: https://arrow.apache.org/docs/developers/cpp/building.html
> >> [2]:
> >>
> https://arrow.apache.org/docs/developers/cpp/building.html#bundled-dependency-versions
> >>
> >> On 10/2/20, James Duong <ja...@bitquilltech.com> wrote:
> >> > Hi David,
> >> >
> >> > I've created a WIP PR illustrating what I'm planning to do. However
> I'm
> >> > still trying to figure out a few things:
> >> >
> >> > - In my environment, gRPC is coming from miniconda3 for C++, and
> >> > seems
> >> > to be an older version. In more recent versions, gRPC has an
> >> > experimental
> >> > API TlsCredentials, which has the option to disable cert
> validation.
> >> Is
> >> > it
> >> > normal that this comes from miniconda, or is it just my setup?
> >> > - See:
> >> >
> >> >
> >>
> https://github.com/grpc/grpc/blob/a6a5cbfee2ff2709a8bc66c3d4b428db48ae46af/src/cpp/common/tls_credentials_options.cc#L284
> >> > . This shows up starting with 1.30.x
> >> > - Where do we specify which version of gRPC we depend on? In
> java
> >> > it's in maven and looks to be 1.30.2.
> >> > - The Java implementation looks fairly straightforward.
> >> > - Python implementation is pending.
> >> > - Tests are pending.
> >> >
> >> >
> >> > On Wed, Sep 30, 2020 at 8:12 AM James Duong <ja...@bitquilltech.com>
> >> > wrote:
> >> >
> >> >> Will do David.
> >> >>
> >> >> I've created a JIRA for the TLS_SNI extension as well:
> >> >> https://issues.apache.org/jira/browse/ARROW-10144
> >> >>
> >> >> On Wed, Sep 30, 2020 at 7:52 AM David Li <li...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi James,
> >> >>>
> >> >>> Please feel free to tag me as a reviewer if needed.
> >> >>>
> >> >>> Best,
> >> >>> David
> >> >>>
> >> >>> On 9/28/20, James Duong <ja...@bitquilltech.com> wrote:
> >> >>> > Hi Arrow-Dev,
> >> >>> >
> >> >>> > I've created a JIRA for supporting using Flight clients to connect
> >> >>> > to
> >> >>> > Flight servers without verifying the server certificate.
> >> >>> >
> >> >>> > This is something I've seen done frequently for other connectivity
> >> >>> > tools
> >> >>> > such as JDBC drivers.
> >> >>> >
> >> >>> > I'll be looking to implement this one for the and hopefully get it
> >> >>> > in
> >> >>> for
> >> >>> > the next release:
> >> >>> >
> >> https://issues.apache.org/jira/plugins/servlet/mobile#issue/ARROW-10105
> >> >>> >
> >> >>> > Related, I'm also planning to add support for using the TLS_SNI
> >> >>> extension.
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >> *James Duong*
> >> >> Lead Software Developer
> >> >> Bit Quill Technologies Inc.
> >> >> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> >> >> https://www.bitquilltech.com
> >> >>
> >> >> This email message is for the sole use of the intended recipient(s)
> >> >> and
> >> >> may contain confidential and privileged information. Any
> unauthorized
> >> >> review, use, disclosure, or distribution is prohibited. If you are
> >> >> not
> >> >> the
> >> >> intended recipient, please contact the sender by reply email and
> >> >> destroy
> >> >> all copies of the original message. Thank you.
> >> >>
> >> >
> >> >
> >> > --
> >> >
> >> > *James Duong*
> >> > Lead Software Developer
> >> > Bit Quill Technologies Inc.
> >> > Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> >> > https://www.bitquilltech.com
> >> >
> >> > This email message is for the sole use of the intended recipient(s)
> and
> >> may
> >> > contain confidential and privileged information. Any unauthorized
> >> review,
> >> > use, disclosure, or distribution is prohibited. If you are not the
> >> > intended recipient, please contact the sender by reply email and
> >> > destroy
> >> > all copies of the original message. Thank you.
> >> >
> >>
> >
> >
> > --
> >
> > *James Duong*
> > Lead Software Developer
> > Bit Quill Technologies Inc.
> > Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> > https://www.bitquilltech.com
> >
> > This email message is for the sole use of the intended recipient(s) and
> may
> > contain confidential and privileged information. Any unauthorized
> review,
> > use, disclosure, or distribution is prohibited. If you are not the
> > intended recipient, please contact the sender by reply email and destroy
> > all copies of the original message. Thank you.
> >
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.
Re: [FlightRPC][Java][Python/c_lib] Encryption without validation in
Flight clients
Posted by David Li <li...@gmail.com>.
For #1, can we embed a dummy root to make gRPC happy, since we're only
using it in the case that we're disabling validation? (Also, is there
a bug report for gRPC/would you like to file one?)
For #2, I think you can get cmake to pass -D flags, perhaps you can
get it to define a symbol you can check (though that is maybe not a
preferable solution).
David
On 10/5/20, James Duong <ja...@bitquilltech.com> wrote:
> An update about this issue (also on JIRA at
> https://issues.apache.org/jira/browse/ARROW-10105)
> Hoping for some suggestions on how to deal with C++ issues with gRPC
> versioning.
>
> I have the Java version of this feature working and tested.
>
> The C++ client has some challenges:
> The experimental TlsCredentials interface has an option to specify that you
> want to disable server certificate checks. You also must supply
> a callback that is supposed to represent how you the client should verify
> the server instead. There are two problems with this interface
> though:
> 1. The caller needs to supply a CA roots PEM file to this. This would be
> unexpected since the point of using this is to skip over
> passing in certificate information. Furthermore, the SslCredentials
> interface that's currently being used for SSL automatically
> chooses a default CA roots PEM file if none is supplied. gRPC ships one as
> part of its installation process that it uses.
> There does not appear to be a programmatic way to get to the path to the
> shipped with gRPC. We may need to supply
> a default/dummy PEM file for the purpose of skipping server cert
> verification. I've verified that if I do supply
> a PEM file, everything works correctly and the server verification gets
> skipped. The Python client changes associated
> with this also work.
>
> 2. The TlsCredentials class and related Options, etc classes are in a
> different namespace depending on gRPC version.
> It first shows up in gRPC 1.25 in the grpc_impl::experimental namespace. It
> got moved to grpc::experimental namespace
> starting in gRPC 1.32. What's problematic is that we can't get the gRPC
> version at compile-time, so we can't choose
> the appropriate namespace for looking these up. My preference is to just
> switch entirely to 1.32, however I'm
> seeing issues in some CI jobs that don't seem to correctly use 1.32 such
> as:
> https://ci.ursalabs.org/#/builders/66/builds/11440
> The CentOS 5.11 build also can't seem to build gRPC 1.32. It fails
> compiling some code in linux/tcp.h, which
> may be related to using an older Linux kernel:
> https://github.com/apache/arrow/pull/8325/checks?check_run_id=1208358439):
>
> CMakeFiles/grpc_unsecure.dir/src/core/lib/iomgr/socket_utils_common_posix.cc.o
> -c src/core/lib/iomgr/socket_utils_common_posix.cc
> In file included from /usr/include/asm-x86_64/byteorder.h:30:0,
> from /usr/include/asm/byteorder.h:5,
> from /usr/include/linux/tcp.h:21,
> from src/core/lib/iomgr/socket_utils_common_posix.cc:34:
> /usr/include/linux/byteorder/little_endian.h:43:19: error: ‘__le64’
> does not name a type
> static __inline__ __le64 __cpu_to_le64p(const __u64 *p)
> ^
> /usr/include/linux/byteorder/little_endian.h:47:46: error: ‘__le64’
> does not name a type
> static __inline__ __u64 __le64_to_cpup(const __le64 *p)
> ^
> /usr/include/linux/byteorder/little_endian.h:67:19: error: ‘__be64’
> does not name a type
> static __inline__ __be64 __cpu_to_be64p(const __u64 *p)
> ^
> /usr/include/linux/byteorder/little_endian.h:71:46: error: ‘__be64’
> does not name a type
> static __inline__ __u64 __be64_to_cpup(const __be64 *p)
>
>
>
> On Fri, Oct 2, 2020 at 5:36 AM David Li <li...@gmail.com> wrote:
>
>> Hey James,
>>
>> Where gRPC comes from depends on your setup; the C++ development
>> guide[1] describes it all, but I personally use Conda with conda-forge
>> which gets me gRPC 1.32.
>>
>> The dependency version again depends on where you're looking - for
>> bundled builds it's set in thirdparty/versions.txt[2], for the other
>> CI builds it's set in various config files.
>>
>> Best,
>> David
>>
>> [1]: https://arrow.apache.org/docs/developers/cpp/building.html
>> [2]:
>> https://arrow.apache.org/docs/developers/cpp/building.html#bundled-dependency-versions
>>
>> On 10/2/20, James Duong <ja...@bitquilltech.com> wrote:
>> > Hi David,
>> >
>> > I've created a WIP PR illustrating what I'm planning to do. However I'm
>> > still trying to figure out a few things:
>> >
>> > - In my environment, gRPC is coming from miniconda3 for C++, and
>> > seems
>> > to be an older version. In more recent versions, gRPC has an
>> > experimental
>> > API TlsCredentials, which has the option to disable cert validation.
>> Is
>> > it
>> > normal that this comes from miniconda, or is it just my setup?
>> > - See:
>> >
>> >
>> https://github.com/grpc/grpc/blob/a6a5cbfee2ff2709a8bc66c3d4b428db48ae46af/src/cpp/common/tls_credentials_options.cc#L284
>> > . This shows up starting with 1.30.x
>> > - Where do we specify which version of gRPC we depend on? In java
>> > it's in maven and looks to be 1.30.2.
>> > - The Java implementation looks fairly straightforward.
>> > - Python implementation is pending.
>> > - Tests are pending.
>> >
>> >
>> > On Wed, Sep 30, 2020 at 8:12 AM James Duong <ja...@bitquilltech.com>
>> > wrote:
>> >
>> >> Will do David.
>> >>
>> >> I've created a JIRA for the TLS_SNI extension as well:
>> >> https://issues.apache.org/jira/browse/ARROW-10144
>> >>
>> >> On Wed, Sep 30, 2020 at 7:52 AM David Li <li...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi James,
>> >>>
>> >>> Please feel free to tag me as a reviewer if needed.
>> >>>
>> >>> Best,
>> >>> David
>> >>>
>> >>> On 9/28/20, James Duong <ja...@bitquilltech.com> wrote:
>> >>> > Hi Arrow-Dev,
>> >>> >
>> >>> > I've created a JIRA for supporting using Flight clients to connect
>> >>> > to
>> >>> > Flight servers without verifying the server certificate.
>> >>> >
>> >>> > This is something I've seen done frequently for other connectivity
>> >>> > tools
>> >>> > such as JDBC drivers.
>> >>> >
>> >>> > I'll be looking to implement this one for the and hopefully get it
>> >>> > in
>> >>> for
>> >>> > the next release:
>> >>> >
>> https://issues.apache.org/jira/plugins/servlet/mobile#issue/ARROW-10105
>> >>> >
>> >>> > Related, I'm also planning to add support for using the TLS_SNI
>> >>> extension.
>> >>> >
>> >>>
>> >>
>> >>
>> >> --
>> >>
>> >> *James Duong*
>> >> Lead Software Developer
>> >> Bit Quill Technologies Inc.
>> >> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
>> >> https://www.bitquilltech.com
>> >>
>> >> This email message is for the sole use of the intended recipient(s)
>> >> and
>> >> may contain confidential and privileged information. Any unauthorized
>> >> review, use, disclosure, or distribution is prohibited. If you are
>> >> not
>> >> the
>> >> intended recipient, please contact the sender by reply email and
>> >> destroy
>> >> all copies of the original message. Thank you.
>> >>
>> >
>> >
>> > --
>> >
>> > *James Duong*
>> > Lead Software Developer
>> > Bit Quill Technologies Inc.
>> > Direct: +1.604.562.6082 | jamesd@bitquilltech.com
>> > https://www.bitquilltech.com
>> >
>> > This email message is for the sole use of the intended recipient(s) and
>> may
>> > contain confidential and privileged information. Any unauthorized
>> review,
>> > use, disclosure, or distribution is prohibited. If you are not the
>> > intended recipient, please contact the sender by reply email and
>> > destroy
>> > all copies of the original message. Thank you.
>> >
>>
>
>
> --
>
> *James Duong*
> Lead Software Developer
> Bit Quill Technologies Inc.
> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> https://www.bitquilltech.com
>
> This email message is for the sole use of the intended recipient(s) and may
> contain confidential and privileged information. Any unauthorized review,
> use, disclosure, or distribution is prohibited. If you are not the
> intended recipient, please contact the sender by reply email and destroy
> all copies of the original message. Thank you.
>
Re: [FlightRPC][Java][Python/c_lib] Encryption without validation in
Flight clients
Posted by James Duong <ja...@bitquilltech.com>.
An update about this issue (also on JIRA at
https://issues.apache.org/jira/browse/ARROW-10105)
Hoping for some suggestions on how to deal with C++ issues with gRPC
versioning.
I have the Java version of this feature working and tested.
The C++ client has some challenges:
The experimental TlsCredentials interface has an option to specify that you
want to disable server certificate checks. You also must supply
a callback that is supposed to represent how you the client should verify
the server instead. There are two problems with this interface
though:
1. The caller needs to supply a CA roots PEM file to this. This would be
unexpected since the point of using this is to skip over
passing in certificate information. Furthermore, the SslCredentials
interface that's currently being used for SSL automatically
chooses a default CA roots PEM file if none is supplied. gRPC ships one as
part of its installation process that it uses.
There does not appear to be a programmatic way to get to the path to the
shipped with gRPC. We may need to supply
a default/dummy PEM file for the purpose of skipping server cert
verification. I've verified that if I do supply
a PEM file, everything works correctly and the server verification gets
skipped. The Python client changes associated
with this also work.
2. The TlsCredentials class and related Options, etc classes are in a
different namespace depending on gRPC version.
It first shows up in gRPC 1.25 in the grpc_impl::experimental namespace. It
got moved to grpc::experimental namespace
starting in gRPC 1.32. What's problematic is that we can't get the gRPC
version at compile-time, so we can't choose
the appropriate namespace for looking these up. My preference is to just
switch entirely to 1.32, however I'm
seeing issues in some CI jobs that don't seem to correctly use 1.32 such as:
https://ci.ursalabs.org/#/builders/66/builds/11440
The CentOS 5.11 build also can't seem to build gRPC 1.32. It fails
compiling some code in linux/tcp.h, which
may be related to using an older Linux kernel:
https://github.com/apache/arrow/pull/8325/checks?check_run_id=1208358439):
CMakeFiles/grpc_unsecure.dir/src/core/lib/iomgr/socket_utils_common_posix.cc.o
-c src/core/lib/iomgr/socket_utils_common_posix.cc
In file included from /usr/include/asm-x86_64/byteorder.h:30:0,
from /usr/include/asm/byteorder.h:5,
from /usr/include/linux/tcp.h:21,
from src/core/lib/iomgr/socket_utils_common_posix.cc:34:
/usr/include/linux/byteorder/little_endian.h:43:19: error: ‘__le64’
does not name a type
static __inline__ __le64 __cpu_to_le64p(const __u64 *p)
^
/usr/include/linux/byteorder/little_endian.h:47:46: error: ‘__le64’
does not name a type
static __inline__ __u64 __le64_to_cpup(const __le64 *p)
^
/usr/include/linux/byteorder/little_endian.h:67:19: error: ‘__be64’
does not name a type
static __inline__ __be64 __cpu_to_be64p(const __u64 *p)
^
/usr/include/linux/byteorder/little_endian.h:71:46: error: ‘__be64’
does not name a type
static __inline__ __u64 __be64_to_cpup(const __be64 *p)
On Fri, Oct 2, 2020 at 5:36 AM David Li <li...@gmail.com> wrote:
> Hey James,
>
> Where gRPC comes from depends on your setup; the C++ development
> guide[1] describes it all, but I personally use Conda with conda-forge
> which gets me gRPC 1.32.
>
> The dependency version again depends on where you're looking - for
> bundled builds it's set in thirdparty/versions.txt[2], for the other
> CI builds it's set in various config files.
>
> Best,
> David
>
> [1]: https://arrow.apache.org/docs/developers/cpp/building.html
> [2]:
> https://arrow.apache.org/docs/developers/cpp/building.html#bundled-dependency-versions
>
> On 10/2/20, James Duong <ja...@bitquilltech.com> wrote:
> > Hi David,
> >
> > I've created a WIP PR illustrating what I'm planning to do. However I'm
> > still trying to figure out a few things:
> >
> > - In my environment, gRPC is coming from miniconda3 for C++, and seems
> > to be an older version. In more recent versions, gRPC has an
> > experimental
> > API TlsCredentials, which has the option to disable cert validation.
> Is
> > it
> > normal that this comes from miniconda, or is it just my setup?
> > - See:
> >
> >
> https://github.com/grpc/grpc/blob/a6a5cbfee2ff2709a8bc66c3d4b428db48ae46af/src/cpp/common/tls_credentials_options.cc#L284
> > . This shows up starting with 1.30.x
> > - Where do we specify which version of gRPC we depend on? In java
> > it's in maven and looks to be 1.30.2.
> > - The Java implementation looks fairly straightforward.
> > - Python implementation is pending.
> > - Tests are pending.
> >
> >
> > On Wed, Sep 30, 2020 at 8:12 AM James Duong <ja...@bitquilltech.com>
> > wrote:
> >
> >> Will do David.
> >>
> >> I've created a JIRA for the TLS_SNI extension as well:
> >> https://issues.apache.org/jira/browse/ARROW-10144
> >>
> >> On Wed, Sep 30, 2020 at 7:52 AM David Li <li...@gmail.com> wrote:
> >>
> >>> Hi James,
> >>>
> >>> Please feel free to tag me as a reviewer if needed.
> >>>
> >>> Best,
> >>> David
> >>>
> >>> On 9/28/20, James Duong <ja...@bitquilltech.com> wrote:
> >>> > Hi Arrow-Dev,
> >>> >
> >>> > I've created a JIRA for supporting using Flight clients to connect to
> >>> > Flight servers without verifying the server certificate.
> >>> >
> >>> > This is something I've seen done frequently for other connectivity
> >>> > tools
> >>> > such as JDBC drivers.
> >>> >
> >>> > I'll be looking to implement this one for the and hopefully get it in
> >>> for
> >>> > the next release:
> >>> >
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/ARROW-10105
> >>> >
> >>> > Related, I'm also planning to add support for using the TLS_SNI
> >>> extension.
> >>> >
> >>>
> >>
> >>
> >> --
> >>
> >> *James Duong*
> >> Lead Software Developer
> >> Bit Quill Technologies Inc.
> >> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> >> https://www.bitquilltech.com
> >>
> >> This email message is for the sole use of the intended recipient(s) and
> >> may contain confidential and privileged information. Any unauthorized
> >> review, use, disclosure, or distribution is prohibited. If you are not
> >> the
> >> intended recipient, please contact the sender by reply email and destroy
> >> all copies of the original message. Thank you.
> >>
> >
> >
> > --
> >
> > *James Duong*
> > Lead Software Developer
> > Bit Quill Technologies Inc.
> > Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> > https://www.bitquilltech.com
> >
> > This email message is for the sole use of the intended recipient(s) and
> may
> > contain confidential and privileged information. Any unauthorized
> review,
> > use, disclosure, or distribution is prohibited. If you are not the
> > intended recipient, please contact the sender by reply email and destroy
> > all copies of the original message. Thank you.
> >
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.
Re: [FlightRPC][Java][Python/c_lib] Encryption without validation in
Flight clients
Posted by David Li <li...@gmail.com>.
Hey James,
Where gRPC comes from depends on your setup; the C++ development
guide[1] describes it all, but I personally use Conda with conda-forge
which gets me gRPC 1.32.
The dependency version again depends on where you're looking - for
bundled builds it's set in thirdparty/versions.txt[2], for the other
CI builds it's set in various config files.
Best,
David
[1]: https://arrow.apache.org/docs/developers/cpp/building.html
[2]: https://arrow.apache.org/docs/developers/cpp/building.html#bundled-dependency-versions
On 10/2/20, James Duong <ja...@bitquilltech.com> wrote:
> Hi David,
>
> I've created a WIP PR illustrating what I'm planning to do. However I'm
> still trying to figure out a few things:
>
> - In my environment, gRPC is coming from miniconda3 for C++, and seems
> to be an older version. In more recent versions, gRPC has an
> experimental
> API TlsCredentials, which has the option to disable cert validation. Is
> it
> normal that this comes from miniconda, or is it just my setup?
> - See:
>
> https://github.com/grpc/grpc/blob/a6a5cbfee2ff2709a8bc66c3d4b428db48ae46af/src/cpp/common/tls_credentials_options.cc#L284
> . This shows up starting with 1.30.x
> - Where do we specify which version of gRPC we depend on? In java
> it's in maven and looks to be 1.30.2.
> - The Java implementation looks fairly straightforward.
> - Python implementation is pending.
> - Tests are pending.
>
>
> On Wed, Sep 30, 2020 at 8:12 AM James Duong <ja...@bitquilltech.com>
> wrote:
>
>> Will do David.
>>
>> I've created a JIRA for the TLS_SNI extension as well:
>> https://issues.apache.org/jira/browse/ARROW-10144
>>
>> On Wed, Sep 30, 2020 at 7:52 AM David Li <li...@gmail.com> wrote:
>>
>>> Hi James,
>>>
>>> Please feel free to tag me as a reviewer if needed.
>>>
>>> Best,
>>> David
>>>
>>> On 9/28/20, James Duong <ja...@bitquilltech.com> wrote:
>>> > Hi Arrow-Dev,
>>> >
>>> > I've created a JIRA for supporting using Flight clients to connect to
>>> > Flight servers without verifying the server certificate.
>>> >
>>> > This is something I've seen done frequently for other connectivity
>>> > tools
>>> > such as JDBC drivers.
>>> >
>>> > I'll be looking to implement this one for the and hopefully get it in
>>> for
>>> > the next release:
>>> > https://issues.apache.org/jira/plugins/servlet/mobile#issue/ARROW-10105
>>> >
>>> > Related, I'm also planning to add support for using the TLS_SNI
>>> extension.
>>> >
>>>
>>
>>
>> --
>>
>> *James Duong*
>> Lead Software Developer
>> Bit Quill Technologies Inc.
>> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
>> https://www.bitquilltech.com
>>
>> This email message is for the sole use of the intended recipient(s) and
>> may contain confidential and privileged information. Any unauthorized
>> review, use, disclosure, or distribution is prohibited. If you are not
>> the
>> intended recipient, please contact the sender by reply email and destroy
>> all copies of the original message. Thank you.
>>
>
>
> --
>
> *James Duong*
> Lead Software Developer
> Bit Quill Technologies Inc.
> Direct: +1.604.562.6082 | jamesd@bitquilltech.com
> https://www.bitquilltech.com
>
> This email message is for the sole use of the intended recipient(s) and may
> contain confidential and privileged information. Any unauthorized review,
> use, disclosure, or distribution is prohibited. If you are not the
> intended recipient, please contact the sender by reply email and destroy
> all copies of the original message. Thank you.
>