You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Antoine Pitrou <so...@pitrou.net> on 2019/02/27 16:06:04 UTC
URI library for C++
Hello,
As part of ARROW-4651, we would need to have a URI parsing library in
the C++ project.
One such library is https://github.com/cpp-netlib/uri, it's based on a
previous proposal for the standard C++ library. It has no dependencies
except boost::algorithm.
One problem is that the library ships its own backports of
`std::string_view` and `std::optional`. We already have a backport of
`std::string_view` in our source tree (it seems more complete). So we
would need to patch the uri library to use our backport. Maintaining
the patch will be a bit annoying.
Another possibility is to use the C-only, no-dependencies uriparser
library (and write a small C++ wrapper around it):
https://uriparser.github.io/
Regards
Antoine.
Re: URI library for C++
Posted by Antoine Pitrou <so...@pitrou.net>.
Hi Uwe,
I am working on integrating uriparser into Arrow and I am using the
CMake support indeed:
https://github.com/apache/arrow/compare/master...pitrou:ARROW-4697-cpp-uri-parsing
Regards
Antoine.
On Mon, 18 Mar 2019 05:58:26 -0400
"Uwe L. Korn" <uw...@xhochy.com> wrote:
> I have looked into packaging uriparser for conda-forge to have it available in our build system and it feels to me that we rather should wait for their next release where they switch the build system over to CMake. This will make packaging in conda and the ExternalProject much easier. Not sure if anyone has already engaged with them but that would save us a bit of work.
>
> Uwe
>
> On Wed, Feb 27, 2019, at 5:38 PM, Francois Saint-Jacques wrote:
> > I agree that vendoring curl is crazyness, my main point is that curl now
> > has a url api. If we can find a way to avoid pulling another dependency.
> >
> > On Wed, Feb 27, 2019 at 11:30 AM Antoine Pitrou <so...@pitrou.net> wrote:
> >
> > >
> > > Vendoring curl sounds a bit crazy IMHO. We'll end up having to vendor a
> > > TLS library and who knows what else...
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > > On Wed, 27 Feb 2019 11:16:49 -0500
> > > Francois Saint-Jacques <fs...@gmail.com> wrote:
> > > > There's a good chance we end up using curl for the dataset project. Curl
> > > > has a new url API https://github.com/curl/curl/wiki/URL-API , but it
> > > > requires a recent version (7.62.0 october 2018) which means vendoring.
> > > >
> > > > François
> > > >
> > > > On Wed, Feb 27, 2019 at 11:06 AM Antoine Pitrou <so...@pitrou.net>
> > > wrote:
> > > >
> > > > >
> > > > > Hello,
> > > > >
> > > > > As part of ARROW-4651, we would need to have a URI parsing library in
> > > > > the C++ project.
> > > > >
> > > > > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > > > > previous proposal for the standard C++ library. It has no dependencies
> > > > > except boost::algorithm.
> > > > >
> > > > > One problem is that the library ships its own backports of
> > > > > `std::string_view` and `std::optional`. We already have a backport of
> > > > > `std::string_view` in our source tree (it seems more complete). So we
> > > > > would need to patch the uri library to use our backport. Maintaining
> > > > > the patch will be a bit annoying.
> > > > >
> > > > > Another possibility is to use the C-only, no-dependencies uriparser
> > > > > library (and write a small C++ wrapper around it):
> > > > > https://uriparser.github.io/
> > > > >
> > > > > Regards
> > > > >
> > > > > Antoine.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > >
> >
>
Re: URI library for C++
Posted by "Uwe L. Korn" <uw...@xhochy.com>.
I have looked into packaging uriparser for conda-forge to have it available in our build system and it feels to me that we rather should wait for their next release where they switch the build system over to CMake. This will make packaging in conda and the ExternalProject much easier. Not sure if anyone has already engaged with them but that would save us a bit of work.
Uwe
On Wed, Feb 27, 2019, at 5:38 PM, Francois Saint-Jacques wrote:
> I agree that vendoring curl is crazyness, my main point is that curl now
> has a url api. If we can find a way to avoid pulling another dependency.
>
> On Wed, Feb 27, 2019 at 11:30 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
> >
> > Vendoring curl sounds a bit crazy IMHO. We'll end up having to vendor a
> > TLS library and who knows what else...
> >
> > Regards
> >
> > Antoine.
> >
> >
> > On Wed, 27 Feb 2019 11:16:49 -0500
> > Francois Saint-Jacques <fs...@gmail.com> wrote:
> > > There's a good chance we end up using curl for the dataset project. Curl
> > > has a new url API https://github.com/curl/curl/wiki/URL-API , but it
> > > requires a recent version (7.62.0 october 2018) which means vendoring.
> > >
> > > François
> > >
> > > On Wed, Feb 27, 2019 at 11:06 AM Antoine Pitrou <so...@pitrou.net>
> > wrote:
> > >
> > > >
> > > > Hello,
> > > >
> > > > As part of ARROW-4651, we would need to have a URI parsing library in
> > > > the C++ project.
> > > >
> > > > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > > > previous proposal for the standard C++ library. It has no dependencies
> > > > except boost::algorithm.
> > > >
> > > > One problem is that the library ships its own backports of
> > > > `std::string_view` and `std::optional`. We already have a backport of
> > > > `std::string_view` in our source tree (it seems more complete). So we
> > > > would need to patch the uri library to use our backport. Maintaining
> > > > the patch will be a bit annoying.
> > > >
> > > > Another possibility is to use the C-only, no-dependencies uriparser
> > > > library (and write a small C++ wrapper around it):
> > > > https://uriparser.github.io/
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
>
Re: URI library for C++
Posted by Francois Saint-Jacques <fs...@gmail.com>.
I agree that vendoring curl is crazyness, my main point is that curl now
has a url api. If we can find a way to avoid pulling another dependency.
On Wed, Feb 27, 2019 at 11:30 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
> Vendoring curl sounds a bit crazy IMHO. We'll end up having to vendor a
> TLS library and who knows what else...
>
> Regards
>
> Antoine.
>
>
> On Wed, 27 Feb 2019 11:16:49 -0500
> Francois Saint-Jacques <fs...@gmail.com> wrote:
> > There's a good chance we end up using curl for the dataset project. Curl
> > has a new url API https://github.com/curl/curl/wiki/URL-API , but it
> > requires a recent version (7.62.0 october 2018) which means vendoring.
> >
> > François
> >
> > On Wed, Feb 27, 2019 at 11:06 AM Antoine Pitrou <so...@pitrou.net>
> wrote:
> >
> > >
> > > Hello,
> > >
> > > As part of ARROW-4651, we would need to have a URI parsing library in
> > > the C++ project.
> > >
> > > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > > previous proposal for the standard C++ library. It has no dependencies
> > > except boost::algorithm.
> > >
> > > One problem is that the library ships its own backports of
> > > `std::string_view` and `std::optional`. We already have a backport of
> > > `std::string_view` in our source tree (it seems more complete). So we
> > > would need to patch the uri library to use our backport. Maintaining
> > > the patch will be a bit annoying.
> > >
> > > Another possibility is to use the C-only, no-dependencies uriparser
> > > library (and write a small C++ wrapper around it):
> > > https://uriparser.github.io/
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > >
> >
>
>
>
>
Re: URI library for C++
Posted by Wes McKinney <we...@gmail.com>.
Yes I do not think vendoring curl is a good idea
On Wed, Feb 27, 2019 at 10:30 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
>
> Vendoring curl sounds a bit crazy IMHO. We'll end up having to vendor a
> TLS library and who knows what else...
>
> Regards
>
> Antoine.
>
>
> On Wed, 27 Feb 2019 11:16:49 -0500
> Francois Saint-Jacques <fs...@gmail.com> wrote:
> > There's a good chance we end up using curl for the dataset project. Curl
> > has a new url API https://github.com/curl/curl/wiki/URL-API , but it
> > requires a recent version (7.62.0 october 2018) which means vendoring.
> >
> > François
> >
> > On Wed, Feb 27, 2019 at 11:06 AM Antoine Pitrou <so...@pitrou.net> wrote:
> >
> > >
> > > Hello,
> > >
> > > As part of ARROW-4651, we would need to have a URI parsing library in
> > > the C++ project.
> > >
> > > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > > previous proposal for the standard C++ library. It has no dependencies
> > > except boost::algorithm.
> > >
> > > One problem is that the library ships its own backports of
> > > `std::string_view` and `std::optional`. We already have a backport of
> > > `std::string_view` in our source tree (it seems more complete). So we
> > > would need to patch the uri library to use our backport. Maintaining
> > > the patch will be a bit annoying.
> > >
> > > Another possibility is to use the C-only, no-dependencies uriparser
> > > library (and write a small C++ wrapper around it):
> > > https://uriparser.github.io/
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > >
> >
>
>
>
Re: URI library for C++
Posted by Antoine Pitrou <so...@pitrou.net>.
Vendoring curl sounds a bit crazy IMHO. We'll end up having to vendor a
TLS library and who knows what else...
Regards
Antoine.
On Wed, 27 Feb 2019 11:16:49 -0500
Francois Saint-Jacques <fs...@gmail.com> wrote:
> There's a good chance we end up using curl for the dataset project. Curl
> has a new url API https://github.com/curl/curl/wiki/URL-API , but it
> requires a recent version (7.62.0 october 2018) which means vendoring.
>
> François
>
> On Wed, Feb 27, 2019 at 11:06 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
> >
> > Hello,
> >
> > As part of ARROW-4651, we would need to have a URI parsing library in
> > the C++ project.
> >
> > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > previous proposal for the standard C++ library. It has no dependencies
> > except boost::algorithm.
> >
> > One problem is that the library ships its own backports of
> > `std::string_view` and `std::optional`. We already have a backport of
> > `std::string_view` in our source tree (it seems more complete). So we
> > would need to patch the uri library to use our backport. Maintaining
> > the patch will be a bit annoying.
> >
> > Another possibility is to use the C-only, no-dependencies uriparser
> > library (and write a small C++ wrapper around it):
> > https://uriparser.github.io/
> >
> > Regards
> >
> > Antoine.
> >
> >
> >
>
Re: URI library for C++
Posted by Francois Saint-Jacques <fs...@gmail.com>.
There's a good chance we end up using curl for the dataset project. Curl
has a new url API https://github.com/curl/curl/wiki/URL-API , but it
requires a recent version (7.62.0 october 2018) which means vendoring.
François
On Wed, Feb 27, 2019 at 11:06 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
> Hello,
>
> As part of ARROW-4651, we would need to have a URI parsing library in
> the C++ project.
>
> One such library is https://github.com/cpp-netlib/uri, it's based on a
> previous proposal for the standard C++ library. It has no dependencies
> except boost::algorithm.
>
> One problem is that the library ships its own backports of
> `std::string_view` and `std::optional`. We already have a backport of
> `std::string_view` in our source tree (it seems more complete). So we
> would need to patch the uri library to use our backport. Maintaining
> the patch will be a bit annoying.
>
> Another possibility is to use the C-only, no-dependencies uriparser
> library (and write a small C++ wrapper around it):
> https://uriparser.github.io/
>
> Regards
>
> Antoine.
>
>
>
Re: URI library for C++
Posted by Wes McKinney <we...@gmail.com>.
> There's a good chance we end up using curl for the dataset project
We'll need curl in our toolchain at some point for interacting e.g.
with Google Cloud Storage [1]
[1]: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/cloud/gcs_file_system.cc
On Wed, Feb 27, 2019 at 10:17 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
>
> I've opened a couple issues for cpp-netlib. Let's how the maintainer
> responds.
>
> Otherwise I agree uriparser sounds better.
>
> Regards
>
> Antoine.
>
>
> On Wed, 27 Feb 2019 10:12:24 -0600
> Wes McKinney <we...@gmail.com> wrote:
> > Seems like uriparser might be a better choice, but I haven't looked
> > into the C++ uri library to see how annoying maintaining a patched
> > version would be
> >
> > On Wed, Feb 27, 2019 at 10:06 AM Antoine Pitrou <so...@pitrou.net> wrote:
> > >
> > >
> > > Hello,
> > >
> > > As part of ARROW-4651, we would need to have a URI parsing library in
> > > the C++ project.
> > >
> > > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > > previous proposal for the standard C++ library. It has no dependencies
> > > except boost::algorithm.
> > >
> > > One problem is that the library ships its own backports of
> > > `std::string_view` and `std::optional`. We already have a backport of
> > > `std::string_view` in our source tree (it seems more complete). So we
> > > would need to patch the uri library to use our backport. Maintaining
> > > the patch will be a bit annoying.
> > >
> > > Another possibility is to use the C-only, no-dependencies uriparser
> > > library (and write a small C++ wrapper around it):
> > > https://uriparser.github.io/
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> >
>
>
>
Re: URI library for C++
Posted by Antoine Pitrou <so...@pitrou.net>.
I've opened a couple issues for cpp-netlib. Let's how the maintainer
responds.
Otherwise I agree uriparser sounds better.
Regards
Antoine.
On Wed, 27 Feb 2019 10:12:24 -0600
Wes McKinney <we...@gmail.com> wrote:
> Seems like uriparser might be a better choice, but I haven't looked
> into the C++ uri library to see how annoying maintaining a patched
> version would be
>
> On Wed, Feb 27, 2019 at 10:06 AM Antoine Pitrou <so...@pitrou.net> wrote:
> >
> >
> > Hello,
> >
> > As part of ARROW-4651, we would need to have a URI parsing library in
> > the C++ project.
> >
> > One such library is https://github.com/cpp-netlib/uri, it's based on a
> > previous proposal for the standard C++ library. It has no dependencies
> > except boost::algorithm.
> >
> > One problem is that the library ships its own backports of
> > `std::string_view` and `std::optional`. We already have a backport of
> > `std::string_view` in our source tree (it seems more complete). So we
> > would need to patch the uri library to use our backport. Maintaining
> > the patch will be a bit annoying.
> >
> > Another possibility is to use the C-only, no-dependencies uriparser
> > library (and write a small C++ wrapper around it):
> > https://uriparser.github.io/
> >
> > Regards
> >
> > Antoine.
> >
> >
>
Re: URI library for C++
Posted by Wes McKinney <we...@gmail.com>.
Seems like uriparser might be a better choice, but I haven't looked
into the C++ uri library to see how annoying maintaining a patched
version would be
On Wed, Feb 27, 2019 at 10:06 AM Antoine Pitrou <so...@pitrou.net> wrote:
>
>
> Hello,
>
> As part of ARROW-4651, we would need to have a URI parsing library in
> the C++ project.
>
> One such library is https://github.com/cpp-netlib/uri, it's based on a
> previous proposal for the standard C++ library. It has no dependencies
> except boost::algorithm.
>
> One problem is that the library ships its own backports of
> `std::string_view` and `std::optional`. We already have a backport of
> `std::string_view` in our source tree (it seems more complete). So we
> would need to patch the uri library to use our backport. Maintaining
> the patch will be a bit annoying.
>
> Another possibility is to use the C-only, no-dependencies uriparser
> library (and write a small C++ wrapper around it):
> https://uriparser.github.io/
>
> Regards
>
> Antoine.
>
>