You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Gildas Quéméner <gi...@gmail.com> on 2018/12/18 06:26:31 UTC
Modern PHP support
Hello,
We've been working for a few months on using Avro encoded messages between
our services in my company.
Our backend heavily relies on PHP, whereas some orbiting services are
written in Java/Kotlin/Python.
At first glance, the lake of official support for PHP looked like a big
no-go for us.
I know that there is an official library, embedded in the avro library at
https://github.com/apache/avro/tree/master/lang/php/lib.
However this library suffers from blocking issues:
- It is not available through the de facto standard Composer dependency
manager
- It is not tested
- It does not rely on autoloading PSR
- It uses global namespace
- It uses unknown php functions
- It lacks some spec features (support for logicalType attribute for
instance)
- It is buggy
Despite these issues, latest commit is 1 year old (about a typo), and the
one before that is almost 3 years old. Thus, I think it is safe to say that
this library is not maintained anymore.
Many organizations have realized such issue and have created their own fork
of the library, none of them being thoroughly maintained and having the
same level of bug fixing.
Here are a few examples:
- https://github.com/wikimedia/avro-php
- https://github.com/flix-tech/avro-php
- https://github.com/researchgate/avro-php
- https://gitlab.com/Jaumo/avro-php (I am the maintainer of this one)
I have tried to contribute to Wikimedia's fork, but they also seem to have
dropped support (see activity in
https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The awesome
thing is that they have brought a test suite (that we are reusing in our
fork)!
Flix-tech is doing an awesome job at providing a Schema Registry API client
as well as an Avro Serializer/Deserializer using their Schema Registry
client.
There are great initiatives out there to modernize this library, but none
of them trully gather the focus of the many maintainers, scattered through
private organizations.
I think it is time to give avro-php some love and I would like to help in
this matter.
My feeling is that the first step should be to open a separate git
repository under the hood of the ASF organization.
This could either be done through an automatic subtree of
https://github.com/apache/avro/tree/master/lang/php/lib to a read-only
repository (as Symfony is doing), in order to register it on packagist.org
or through the opening of a separate read/write repository (and the removal
of the lib from the avro lib repository).
Second step is to communicate, directly by contacting fork maintainers
annoucing the good news, and through ASF official communication vectors
(thinking about ML, Twitter, ...). The goal being to bring back all the PHP
avro user to an official implementation of the library.
Thank you very much for your answer,
And I wish you all fantastic time within friends and family for the end of
the year.
Best Regards.
*Gildas Quéméner*
Re: Modern PHP support
Posted by Gildas Quéméner <gi...@gmail.com>.
Hello Dan,
I wish you a happy new year!
We've been quite busy this month sorting out implementation details of the
avro php library rewrite (which is available at
https://gitlab.com/Jaumo/avro-php).
I believe we're already covering all the features that the official library
is providing, as well as a few others (validation of the default record
field value, support for logical type, ...).
Furthermore, the library is extensively tested (we have written more than
450 test cases), supports PSR-4 autoloading, follow a strict set of code
standard rules, as well as the highest quality level of phpstan.
Now is the time to start thinking about the release and how we can
efficiently provide the library to the world.
As a first step, I believe it would be really useful, and would avoid
wasting everyone's time, if someone at the Avro organization could spend
some time reviewing the library.
That way you could tell us if you're interested, or not, in having it moved
under the Avro organization.
As separating the official repository for each implementations is not an
option for you (at least until 1.9 is released), I suggest that we, at
Jaumo, keep "ownership" of the git repository for now.
That brings me to the point you have raised on your previous message, about
trademark.
It appears that many subtree projects of the official library has re-used
"avro" in their package name (
https://github.com/wikimedia/avro-php/blob/master/composer.json#L2,
https://github.com/researchgate/avro-php/blob/master/composer.json#L2 and
https://github.com/flix-tech/avro-php/blob/master/composer.json#L2, for
instance).
I would like to know if this is an issue as well, and if not, how is it
different to our case?
We are, naturally, willing to comply to any trademark obligation, and as
such would like to ask if you have any suggestions?
However, we find it difficult to release an avro php library, without
mentioning "avro" in its name.
Thanks in advance for your time,
Best Regards.
*Gildas Quéméner*
On Sat, Dec 22, 2018 at 7:05 PM Daniel Kulp <dk...@apache.org> wrote:
>
> Keep in mind, if this is done outside of Apache, you cannot call it
> avro-php. You will need to find a name that does not use Apache trademarks
> and would not cause confusion with people trying to find the official Avro
> PHP builds and such that are supplied by Apache.
>
> Dan
>
>
>
> > On Dec 21, 2018, at 11:16 PM, Gildas Quéméner <gi...@gmail.com>
> wrote:
> >
> > Hello,
> >
> > Thanks for your answer.
> >
> > After giving a little bit more time to think about this situation, we've
> > decided to rewrite the php implementation of the avro library (keeping it
> > open sourced).
> > We're doing the development on this repository:
> > https://gitlab.com/Jaumo/avro-php/merge_requests/1.
> >
> > Let's see what we decide to do when we reach a correct level of
> > specification coverage (The rewrite relis on the 1.8.2 version).
> >
> > The current plan is, besides taking care of the few issues pointed in my
> > first mail, to have the schema (de)serialization (from/to json), data
> > (de)serialization (from/to avro) and protocol implementation at the end
> of
> > January 2019.
> >
> > Any contribution/review is very welcome!
> >
> > I'm off now until January 7th, enjoy the rest of the year!
> >
> > Best regards.
> >
> > *Gildas Quéméner*
> >
> >
> > On Thu, Dec 20, 2018 at 6:37 PM Driesprong, Fokko <fo...@driesprong.frl>
> > wrote:
> >
> >> Hi Gildas,
> >>
> >> This would be very welcome. Currently, I'm working on getting the CI to
> >> work with a newer base image <https://github.com/apache/avro/pull/390>.
> >> But
> >> the tests only pass against a specific version of PHP 5.6. Let me know
> if
> >> there is anything that I can do.
> >>
> >> You could use git blame
> >> <https://github.com/apache/avro/blame/master/lang/php/lib/avro.php> to
> >> check who worked on the PHP stuff, but most of it has been migrated from
> >> SVN so a lot of history is lost. If you feel confident in taking
> ownership
> >> of the PHP, please do! :-)
> >>
> >> Cheers, Fokko
> >>
> >> Op di 18 dec. 2018 om 17:26 schreef Gildas Quéméner <
> >> gildas.quemener@gmail.com>:
> >>
> >>> Hello Daniel,
> >>>
> >>> Thanks for your answer.
> >>>
> >>> I will prepare a pull request in the next days (basically more or less
> >> with
> >>> the content of https://gitlab.com/Jaumo/avro-php/).
> >>>
> >>> The most important things to do in order to make this package usable by
> >> any
> >>> php projects out there is:
> >>>
> >>> 1) To synchronize a separate read-only repository with the content of
> >>> https://github.com/apache/avro/tree/master/lang/php/lib (I'm looking
> at
> >>> various solution to do so) => required by 3).
> >>> 2) To setup a serious CI/CD pipeline which jobs will be to run the
> tests
> >>> (against several php environments), check the code consistency and
> >>> synchronize the read only repository.
> >>> 3) To register the php package on packagist.org
> >>>
> >>> I would gladly take responsibility for these actions as I have
> experience
> >>> in this area and will directly benefit from them.
> >>>
> >>> Problem is that this would require some administrative permission on
> the
> >>> org/repository and I would probably need a referent at the ASF to know
> >> how
> >>> things are done in your organization.
> >>> Is there already someone in charge of the PHP implementation I could
> >>> contact directly?
> >>> If you have some apply process to become an official maintainer as
> well,
> >>> please let me know.
> >>>
> >>> Cheers.
> >>>
> >>> *Gildas Quéméner*
> >>>
> >>>
> >>> On Tue, Dec 18, 2018 at 8:11 PM Daniel Kulp <dk...@apache.org> wrote:
> >>>
> >>>> We would definitely welcome any php contributions. If you have any
> >>>> specific fixes or updates or anything, PR’s would be more than
> welcome.
> >>>> It would be great if someone with some PHP expertise could take some
> >>>> ownership in collecting various fixes/enhancements from various
> >> locations
> >>>> and getting pull requests submitted for review. That would be
> >> awesome.
> >>>>
> >>>> At this point, we won’t be splitting out the various languages to
> >>> separate
> >>>> repositories. We have discussed it in the past and we MAY do it in
> >> the
> >>>> future, but right now we’re trying to concentrate on getting 1.9 out
> >> (and
> >>>> then likely a patch or two follow up) and splitting the repo right now
> >>>> would be too disruptive to that process.
> >>>>
> >>>>
> >>>> Dan
> >>>>
> >>>>
> >>>>> On Dec 18, 2018, at 1:26 AM, Gildas Quéméner <
> >>> gildas.quemener@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Hello,
> >>>>>
> >>>>> We've been working for a few months on using Avro encoded messages
> >>>> between
> >>>>> our services in my company.
> >>>>>
> >>>>> Our backend heavily relies on PHP, whereas some orbiting services are
> >>>>> written in Java/Kotlin/Python.
> >>>>> At first glance, the lake of official support for PHP looked like a
> >> big
> >>>>> no-go for us.
> >>>>>
> >>>>> I know that there is an official library, embedded in the avro
> >> library
> >>> at
> >>>>> https://github.com/apache/avro/tree/master/lang/php/lib.
> >>>>>
> >>>>> However this library suffers from blocking issues:
> >>>>> - It is not available through the de facto standard Composer
> >> dependency
> >>>>> manager
> >>>>> - It is not tested
> >>>>> - It does not rely on autoloading PSR
> >>>>> - It uses global namespace
> >>>>> - It uses unknown php functions
> >>>>> - It lacks some spec features (support for logicalType attribute for
> >>>>> instance)
> >>>>> - It is buggy
> >>>>>
> >>>>> Despite these issues, latest commit is 1 year old (about a typo), and
> >>> the
> >>>>> one before that is almost 3 years old. Thus, I think it is safe to
> >> say
> >>>> that
> >>>>> this library is not maintained anymore.
> >>>>>
> >>>>> Many organizations have realized such issue and have created their
> >> own
> >>>> fork
> >>>>> of the library, none of them being thoroughly maintained and having
> >> the
> >>>>> same level of bug fixing.
> >>>>>
> >>>>> Here are a few examples:
> >>>>> - https://github.com/wikimedia/avro-php
> >>>>> - https://github.com/flix-tech/avro-php
> >>>>> - https://github.com/researchgate/avro-php
> >>>>> - https://gitlab.com/Jaumo/avro-php (I am the maintainer of this
> >> one)
> >>>>>
> >>>>> I have tried to contribute to Wikimedia's fork, but they also seem to
> >>>> have
> >>>>> dropped support (see activity in
> >>>>> https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The
> >>>> awesome
> >>>>> thing is that they have brought a test suite (that we are reusing in
> >>> our
> >>>>> fork)!
> >>>>>
> >>>>> Flix-tech is doing an awesome job at providing a Schema Registry API
> >>>> client
> >>>>> as well as an Avro Serializer/Deserializer using their Schema
> >> Registry
> >>>>> client.
> >>>>>
> >>>>> There are great initiatives out there to modernize this library, but
> >>> none
> >>>>> of them trully gather the focus of the many maintainers, scattered
> >>>> through
> >>>>> private organizations.
> >>>>>
> >>>>> I think it is time to give avro-php some love and I would like to
> >> help
> >>> in
> >>>>> this matter.
> >>>>>
> >>>>> My feeling is that the first step should be to open a separate git
> >>>>> repository under the hood of the ASF organization.
> >>>>> This could either be done through an automatic subtree of
> >>>>> https://github.com/apache/avro/tree/master/lang/php/lib to a
> >> read-only
> >>>>> repository (as Symfony is doing), in order to register it on
> >>>> packagist.org
> >>>>> or through the opening of a separate read/write repository (and the
> >>>> removal
> >>>>> of the lib from the avro lib repository).
> >>>>>
> >>>>> Second step is to communicate, directly by contacting fork
> >> maintainers
> >>>>> annoucing the good news, and through ASF official communication
> >> vectors
> >>>>> (thinking about ML, Twitter, ...). The goal being to bring back all
> >> the
> >>>> PHP
> >>>>> avro user to an official implementation of the library.
> >>>>>
> >>>>> Thank you very much for your answer,
> >>>>> And I wish you all fantastic time within friends and family for the
> >> end
> >>>> of
> >>>>> the year.
> >>>>>
> >>>>> Best Regards.
> >>>>>
> >>>>> *Gildas Quéméner*
> >>>>
> >>>> --
> >>>> Daniel Kulp
> >>>> dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog
> <
> >>>> http://dankulp.com/blog>
> >>>> Talend Community Coder - http://talend.com <http://coders.talend.com/
> >
> >>>>
> >>>
> >>
>
> --
> Daniel Kulp
> dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <
> http://dankulp.com/blog>
> Talend Community Coder - http://talend.com <http://coders.talend.com/>
>
Scala code generation
Posted by Michael Heuer <he...@gmail.com>.
All,
We generate Java objects for use in Spark RDDs from an .avdl schema using the Avro Maven Plugin
https://github.com/bigdatagenomics/bdg-formats <https://github.com/bigdatagenomics/bdg-formats>
We also separately generate Scala products and projection field enums for use in Spark SQL Datasets, a bit more hacked together, declared in
https://github.com/bigdatagenomics/adam/tree/master/adam-codegen <https://github.com/bigdatagenomics/adam/tree/master/adam-codegen>
and used via exec-maven-plugin in
https://github.com/bigdatagenomics/adam/blob/master/adam-core/pom.xml#L91 <https://github.com/bigdatagenomics/adam/blob/master/adam-core/pom.xml#L91>
If of interest to the Avro project, could someone familiar with codegen and the Avro Maven Plugin help me refactor or otherwise adapt this for contribution?
Thank you in advance,
michael
Re: Modern PHP support
Posted by Daniel Kulp <dk...@apache.org>.
Keep in mind, if this is done outside of Apache, you cannot call it avro-php. You will need to find a name that does not use Apache trademarks and would not cause confusion with people trying to find the official Avro PHP builds and such that are supplied by Apache.
Dan
> On Dec 21, 2018, at 11:16 PM, Gildas Quéméner <gi...@gmail.com> wrote:
>
> Hello,
>
> Thanks for your answer.
>
> After giving a little bit more time to think about this situation, we've
> decided to rewrite the php implementation of the avro library (keeping it
> open sourced).
> We're doing the development on this repository:
> https://gitlab.com/Jaumo/avro-php/merge_requests/1.
>
> Let's see what we decide to do when we reach a correct level of
> specification coverage (The rewrite relis on the 1.8.2 version).
>
> The current plan is, besides taking care of the few issues pointed in my
> first mail, to have the schema (de)serialization (from/to json), data
> (de)serialization (from/to avro) and protocol implementation at the end of
> January 2019.
>
> Any contribution/review is very welcome!
>
> I'm off now until January 7th, enjoy the rest of the year!
>
> Best regards.
>
> *Gildas Quéméner*
>
>
> On Thu, Dec 20, 2018 at 6:37 PM Driesprong, Fokko <fo...@driesprong.frl>
> wrote:
>
>> Hi Gildas,
>>
>> This would be very welcome. Currently, I'm working on getting the CI to
>> work with a newer base image <https://github.com/apache/avro/pull/390>.
>> But
>> the tests only pass against a specific version of PHP 5.6. Let me know if
>> there is anything that I can do.
>>
>> You could use git blame
>> <https://github.com/apache/avro/blame/master/lang/php/lib/avro.php> to
>> check who worked on the PHP stuff, but most of it has been migrated from
>> SVN so a lot of history is lost. If you feel confident in taking ownership
>> of the PHP, please do! :-)
>>
>> Cheers, Fokko
>>
>> Op di 18 dec. 2018 om 17:26 schreef Gildas Quéméner <
>> gildas.quemener@gmail.com>:
>>
>>> Hello Daniel,
>>>
>>> Thanks for your answer.
>>>
>>> I will prepare a pull request in the next days (basically more or less
>> with
>>> the content of https://gitlab.com/Jaumo/avro-php/).
>>>
>>> The most important things to do in order to make this package usable by
>> any
>>> php projects out there is:
>>>
>>> 1) To synchronize a separate read-only repository with the content of
>>> https://github.com/apache/avro/tree/master/lang/php/lib (I'm looking at
>>> various solution to do so) => required by 3).
>>> 2) To setup a serious CI/CD pipeline which jobs will be to run the tests
>>> (against several php environments), check the code consistency and
>>> synchronize the read only repository.
>>> 3) To register the php package on packagist.org
>>>
>>> I would gladly take responsibility for these actions as I have experience
>>> in this area and will directly benefit from them.
>>>
>>> Problem is that this would require some administrative permission on the
>>> org/repository and I would probably need a referent at the ASF to know
>> how
>>> things are done in your organization.
>>> Is there already someone in charge of the PHP implementation I could
>>> contact directly?
>>> If you have some apply process to become an official maintainer as well,
>>> please let me know.
>>>
>>> Cheers.
>>>
>>> *Gildas Quéméner*
>>>
>>>
>>> On Tue, Dec 18, 2018 at 8:11 PM Daniel Kulp <dk...@apache.org> wrote:
>>>
>>>> We would definitely welcome any php contributions. If you have any
>>>> specific fixes or updates or anything, PR’s would be more than welcome.
>>>> It would be great if someone with some PHP expertise could take some
>>>> ownership in collecting various fixes/enhancements from various
>> locations
>>>> and getting pull requests submitted for review. That would be
>> awesome.
>>>>
>>>> At this point, we won’t be splitting out the various languages to
>>> separate
>>>> repositories. We have discussed it in the past and we MAY do it in
>> the
>>>> future, but right now we’re trying to concentrate on getting 1.9 out
>> (and
>>>> then likely a patch or two follow up) and splitting the repo right now
>>>> would be too disruptive to that process.
>>>>
>>>>
>>>> Dan
>>>>
>>>>
>>>>> On Dec 18, 2018, at 1:26 AM, Gildas Quéméner <
>>> gildas.quemener@gmail.com>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> We've been working for a few months on using Avro encoded messages
>>>> between
>>>>> our services in my company.
>>>>>
>>>>> Our backend heavily relies on PHP, whereas some orbiting services are
>>>>> written in Java/Kotlin/Python.
>>>>> At first glance, the lake of official support for PHP looked like a
>> big
>>>>> no-go for us.
>>>>>
>>>>> I know that there is an official library, embedded in the avro
>> library
>>> at
>>>>> https://github.com/apache/avro/tree/master/lang/php/lib.
>>>>>
>>>>> However this library suffers from blocking issues:
>>>>> - It is not available through the de facto standard Composer
>> dependency
>>>>> manager
>>>>> - It is not tested
>>>>> - It does not rely on autoloading PSR
>>>>> - It uses global namespace
>>>>> - It uses unknown php functions
>>>>> - It lacks some spec features (support for logicalType attribute for
>>>>> instance)
>>>>> - It is buggy
>>>>>
>>>>> Despite these issues, latest commit is 1 year old (about a typo), and
>>> the
>>>>> one before that is almost 3 years old. Thus, I think it is safe to
>> say
>>>> that
>>>>> this library is not maintained anymore.
>>>>>
>>>>> Many organizations have realized such issue and have created their
>> own
>>>> fork
>>>>> of the library, none of them being thoroughly maintained and having
>> the
>>>>> same level of bug fixing.
>>>>>
>>>>> Here are a few examples:
>>>>> - https://github.com/wikimedia/avro-php
>>>>> - https://github.com/flix-tech/avro-php
>>>>> - https://github.com/researchgate/avro-php
>>>>> - https://gitlab.com/Jaumo/avro-php (I am the maintainer of this
>> one)
>>>>>
>>>>> I have tried to contribute to Wikimedia's fork, but they also seem to
>>>> have
>>>>> dropped support (see activity in
>>>>> https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The
>>>> awesome
>>>>> thing is that they have brought a test suite (that we are reusing in
>>> our
>>>>> fork)!
>>>>>
>>>>> Flix-tech is doing an awesome job at providing a Schema Registry API
>>>> client
>>>>> as well as an Avro Serializer/Deserializer using their Schema
>> Registry
>>>>> client.
>>>>>
>>>>> There are great initiatives out there to modernize this library, but
>>> none
>>>>> of them trully gather the focus of the many maintainers, scattered
>>>> through
>>>>> private organizations.
>>>>>
>>>>> I think it is time to give avro-php some love and I would like to
>> help
>>> in
>>>>> this matter.
>>>>>
>>>>> My feeling is that the first step should be to open a separate git
>>>>> repository under the hood of the ASF organization.
>>>>> This could either be done through an automatic subtree of
>>>>> https://github.com/apache/avro/tree/master/lang/php/lib to a
>> read-only
>>>>> repository (as Symfony is doing), in order to register it on
>>>> packagist.org
>>>>> or through the opening of a separate read/write repository (and the
>>>> removal
>>>>> of the lib from the avro lib repository).
>>>>>
>>>>> Second step is to communicate, directly by contacting fork
>> maintainers
>>>>> annoucing the good news, and through ASF official communication
>> vectors
>>>>> (thinking about ML, Twitter, ...). The goal being to bring back all
>> the
>>>> PHP
>>>>> avro user to an official implementation of the library.
>>>>>
>>>>> Thank you very much for your answer,
>>>>> And I wish you all fantastic time within friends and family for the
>> end
>>>> of
>>>>> the year.
>>>>>
>>>>> Best Regards.
>>>>>
>>>>> *Gildas Quéméner*
>>>>
>>>> --
>>>> Daniel Kulp
>>>> dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <
>>>> http://dankulp.com/blog>
>>>> Talend Community Coder - http://talend.com <http://coders.talend.com/>
>>>>
>>>
>>
--
Daniel Kulp
dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <http://dankulp.com/blog>
Talend Community Coder - http://talend.com <http://coders.talend.com/>
Re: Modern PHP support
Posted by Gildas Quéméner <gi...@gmail.com>.
Hello,
Thanks for your answer.
After giving a little bit more time to think about this situation, we've
decided to rewrite the php implementation of the avro library (keeping it
open sourced).
We're doing the development on this repository:
https://gitlab.com/Jaumo/avro-php/merge_requests/1.
Let's see what we decide to do when we reach a correct level of
specification coverage (The rewrite relis on the 1.8.2 version).
The current plan is, besides taking care of the few issues pointed in my
first mail, to have the schema (de)serialization (from/to json), data
(de)serialization (from/to avro) and protocol implementation at the end of
January 2019.
Any contribution/review is very welcome!
I'm off now until January 7th, enjoy the rest of the year!
Best regards.
*Gildas Quéméner*
On Thu, Dec 20, 2018 at 6:37 PM Driesprong, Fokko <fo...@driesprong.frl>
wrote:
> Hi Gildas,
>
> This would be very welcome. Currently, I'm working on getting the CI to
> work with a newer base image <https://github.com/apache/avro/pull/390>.
> But
> the tests only pass against a specific version of PHP 5.6. Let me know if
> there is anything that I can do.
>
> You could use git blame
> <https://github.com/apache/avro/blame/master/lang/php/lib/avro.php> to
> check who worked on the PHP stuff, but most of it has been migrated from
> SVN so a lot of history is lost. If you feel confident in taking ownership
> of the PHP, please do! :-)
>
> Cheers, Fokko
>
> Op di 18 dec. 2018 om 17:26 schreef Gildas Quéméner <
> gildas.quemener@gmail.com>:
>
> > Hello Daniel,
> >
> > Thanks for your answer.
> >
> > I will prepare a pull request in the next days (basically more or less
> with
> > the content of https://gitlab.com/Jaumo/avro-php/).
> >
> > The most important things to do in order to make this package usable by
> any
> > php projects out there is:
> >
> > 1) To synchronize a separate read-only repository with the content of
> > https://github.com/apache/avro/tree/master/lang/php/lib (I'm looking at
> > various solution to do so) => required by 3).
> > 2) To setup a serious CI/CD pipeline which jobs will be to run the tests
> > (against several php environments), check the code consistency and
> > synchronize the read only repository.
> > 3) To register the php package on packagist.org
> >
> > I would gladly take responsibility for these actions as I have experience
> > in this area and will directly benefit from them.
> >
> > Problem is that this would require some administrative permission on the
> > org/repository and I would probably need a referent at the ASF to know
> how
> > things are done in your organization.
> > Is there already someone in charge of the PHP implementation I could
> > contact directly?
> > If you have some apply process to become an official maintainer as well,
> > please let me know.
> >
> > Cheers.
> >
> > *Gildas Quéméner*
> >
> >
> > On Tue, Dec 18, 2018 at 8:11 PM Daniel Kulp <dk...@apache.org> wrote:
> >
> > > We would definitely welcome any php contributions. If you have any
> > > specific fixes or updates or anything, PR’s would be more than welcome.
> > > It would be great if someone with some PHP expertise could take some
> > > ownership in collecting various fixes/enhancements from various
> locations
> > > and getting pull requests submitted for review. That would be
> awesome.
> > >
> > > At this point, we won’t be splitting out the various languages to
> > separate
> > > repositories. We have discussed it in the past and we MAY do it in
> the
> > > future, but right now we’re trying to concentrate on getting 1.9 out
> (and
> > > then likely a patch or two follow up) and splitting the repo right now
> > > would be too disruptive to that process.
> > >
> > >
> > > Dan
> > >
> > >
> > > > On Dec 18, 2018, at 1:26 AM, Gildas Quéméner <
> > gildas.quemener@gmail.com>
> > > wrote:
> > > >
> > > > Hello,
> > > >
> > > > We've been working for a few months on using Avro encoded messages
> > > between
> > > > our services in my company.
> > > >
> > > > Our backend heavily relies on PHP, whereas some orbiting services are
> > > > written in Java/Kotlin/Python.
> > > > At first glance, the lake of official support for PHP looked like a
> big
> > > > no-go for us.
> > > >
> > > > I know that there is an official library, embedded in the avro
> library
> > at
> > > > https://github.com/apache/avro/tree/master/lang/php/lib.
> > > >
> > > > However this library suffers from blocking issues:
> > > > - It is not available through the de facto standard Composer
> dependency
> > > > manager
> > > > - It is not tested
> > > > - It does not rely on autoloading PSR
> > > > - It uses global namespace
> > > > - It uses unknown php functions
> > > > - It lacks some spec features (support for logicalType attribute for
> > > > instance)
> > > > - It is buggy
> > > >
> > > > Despite these issues, latest commit is 1 year old (about a typo), and
> > the
> > > > one before that is almost 3 years old. Thus, I think it is safe to
> say
> > > that
> > > > this library is not maintained anymore.
> > > >
> > > > Many organizations have realized such issue and have created their
> own
> > > fork
> > > > of the library, none of them being thoroughly maintained and having
> the
> > > > same level of bug fixing.
> > > >
> > > > Here are a few examples:
> > > > - https://github.com/wikimedia/avro-php
> > > > - https://github.com/flix-tech/avro-php
> > > > - https://github.com/researchgate/avro-php
> > > > - https://gitlab.com/Jaumo/avro-php (I am the maintainer of this
> one)
> > > >
> > > > I have tried to contribute to Wikimedia's fork, but they also seem to
> > > have
> > > > dropped support (see activity in
> > > > https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The
> > > awesome
> > > > thing is that they have brought a test suite (that we are reusing in
> > our
> > > > fork)!
> > > >
> > > > Flix-tech is doing an awesome job at providing a Schema Registry API
> > > client
> > > > as well as an Avro Serializer/Deserializer using their Schema
> Registry
> > > > client.
> > > >
> > > > There are great initiatives out there to modernize this library, but
> > none
> > > > of them trully gather the focus of the many maintainers, scattered
> > > through
> > > > private organizations.
> > > >
> > > > I think it is time to give avro-php some love and I would like to
> help
> > in
> > > > this matter.
> > > >
> > > > My feeling is that the first step should be to open a separate git
> > > > repository under the hood of the ASF organization.
> > > > This could either be done through an automatic subtree of
> > > > https://github.com/apache/avro/tree/master/lang/php/lib to a
> read-only
> > > > repository (as Symfony is doing), in order to register it on
> > > packagist.org
> > > > or through the opening of a separate read/write repository (and the
> > > removal
> > > > of the lib from the avro lib repository).
> > > >
> > > > Second step is to communicate, directly by contacting fork
> maintainers
> > > > annoucing the good news, and through ASF official communication
> vectors
> > > > (thinking about ML, Twitter, ...). The goal being to bring back all
> the
> > > PHP
> > > > avro user to an official implementation of the library.
> > > >
> > > > Thank you very much for your answer,
> > > > And I wish you all fantastic time within friends and family for the
> end
> > > of
> > > > the year.
> > > >
> > > > Best Regards.
> > > >
> > > > *Gildas Quéméner*
> > >
> > > --
> > > Daniel Kulp
> > > dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <
> > > http://dankulp.com/blog>
> > > Talend Community Coder - http://talend.com <http://coders.talend.com/>
> > >
> >
>
Re: Modern PHP support
Posted by "Driesprong, Fokko" <fo...@driesprong.frl>.
Hi Gildas,
This would be very welcome. Currently, I'm working on getting the CI to
work with a newer base image <https://github.com/apache/avro/pull/390>. But
the tests only pass against a specific version of PHP 5.6. Let me know if
there is anything that I can do.
You could use git blame
<https://github.com/apache/avro/blame/master/lang/php/lib/avro.php> to
check who worked on the PHP stuff, but most of it has been migrated from
SVN so a lot of history is lost. If you feel confident in taking ownership
of the PHP, please do! :-)
Cheers, Fokko
Op di 18 dec. 2018 om 17:26 schreef Gildas Quéméner <
gildas.quemener@gmail.com>:
> Hello Daniel,
>
> Thanks for your answer.
>
> I will prepare a pull request in the next days (basically more or less with
> the content of https://gitlab.com/Jaumo/avro-php/).
>
> The most important things to do in order to make this package usable by any
> php projects out there is:
>
> 1) To synchronize a separate read-only repository with the content of
> https://github.com/apache/avro/tree/master/lang/php/lib (I'm looking at
> various solution to do so) => required by 3).
> 2) To setup a serious CI/CD pipeline which jobs will be to run the tests
> (against several php environments), check the code consistency and
> synchronize the read only repository.
> 3) To register the php package on packagist.org
>
> I would gladly take responsibility for these actions as I have experience
> in this area and will directly benefit from them.
>
> Problem is that this would require some administrative permission on the
> org/repository and I would probably need a referent at the ASF to know how
> things are done in your organization.
> Is there already someone in charge of the PHP implementation I could
> contact directly?
> If you have some apply process to become an official maintainer as well,
> please let me know.
>
> Cheers.
>
> *Gildas Quéméner*
>
>
> On Tue, Dec 18, 2018 at 8:11 PM Daniel Kulp <dk...@apache.org> wrote:
>
> > We would definitely welcome any php contributions. If you have any
> > specific fixes or updates or anything, PR’s would be more than welcome.
> > It would be great if someone with some PHP expertise could take some
> > ownership in collecting various fixes/enhancements from various locations
> > and getting pull requests submitted for review. That would be awesome.
> >
> > At this point, we won’t be splitting out the various languages to
> separate
> > repositories. We have discussed it in the past and we MAY do it in the
> > future, but right now we’re trying to concentrate on getting 1.9 out (and
> > then likely a patch or two follow up) and splitting the repo right now
> > would be too disruptive to that process.
> >
> >
> > Dan
> >
> >
> > > On Dec 18, 2018, at 1:26 AM, Gildas Quéméner <
> gildas.quemener@gmail.com>
> > wrote:
> > >
> > > Hello,
> > >
> > > We've been working for a few months on using Avro encoded messages
> > between
> > > our services in my company.
> > >
> > > Our backend heavily relies on PHP, whereas some orbiting services are
> > > written in Java/Kotlin/Python.
> > > At first glance, the lake of official support for PHP looked like a big
> > > no-go for us.
> > >
> > > I know that there is an official library, embedded in the avro library
> at
> > > https://github.com/apache/avro/tree/master/lang/php/lib.
> > >
> > > However this library suffers from blocking issues:
> > > - It is not available through the de facto standard Composer dependency
> > > manager
> > > - It is not tested
> > > - It does not rely on autoloading PSR
> > > - It uses global namespace
> > > - It uses unknown php functions
> > > - It lacks some spec features (support for logicalType attribute for
> > > instance)
> > > - It is buggy
> > >
> > > Despite these issues, latest commit is 1 year old (about a typo), and
> the
> > > one before that is almost 3 years old. Thus, I think it is safe to say
> > that
> > > this library is not maintained anymore.
> > >
> > > Many organizations have realized such issue and have created their own
> > fork
> > > of the library, none of them being thoroughly maintained and having the
> > > same level of bug fixing.
> > >
> > > Here are a few examples:
> > > - https://github.com/wikimedia/avro-php
> > > - https://github.com/flix-tech/avro-php
> > > - https://github.com/researchgate/avro-php
> > > - https://gitlab.com/Jaumo/avro-php (I am the maintainer of this one)
> > >
> > > I have tried to contribute to Wikimedia's fork, but they also seem to
> > have
> > > dropped support (see activity in
> > > https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The
> > awesome
> > > thing is that they have brought a test suite (that we are reusing in
> our
> > > fork)!
> > >
> > > Flix-tech is doing an awesome job at providing a Schema Registry API
> > client
> > > as well as an Avro Serializer/Deserializer using their Schema Registry
> > > client.
> > >
> > > There are great initiatives out there to modernize this library, but
> none
> > > of them trully gather the focus of the many maintainers, scattered
> > through
> > > private organizations.
> > >
> > > I think it is time to give avro-php some love and I would like to help
> in
> > > this matter.
> > >
> > > My feeling is that the first step should be to open a separate git
> > > repository under the hood of the ASF organization.
> > > This could either be done through an automatic subtree of
> > > https://github.com/apache/avro/tree/master/lang/php/lib to a read-only
> > > repository (as Symfony is doing), in order to register it on
> > packagist.org
> > > or through the opening of a separate read/write repository (and the
> > removal
> > > of the lib from the avro lib repository).
> > >
> > > Second step is to communicate, directly by contacting fork maintainers
> > > annoucing the good news, and through ASF official communication vectors
> > > (thinking about ML, Twitter, ...). The goal being to bring back all the
> > PHP
> > > avro user to an official implementation of the library.
> > >
> > > Thank you very much for your answer,
> > > And I wish you all fantastic time within friends and family for the end
> > of
> > > the year.
> > >
> > > Best Regards.
> > >
> > > *Gildas Quéméner*
> >
> > --
> > Daniel Kulp
> > dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <
> > http://dankulp.com/blog>
> > Talend Community Coder - http://talend.com <http://coders.talend.com/>
> >
>
Re: Modern PHP support
Posted by Gildas Quéméner <gi...@gmail.com>.
Hello Daniel,
Thanks for your answer.
I will prepare a pull request in the next days (basically more or less with
the content of https://gitlab.com/Jaumo/avro-php/).
The most important things to do in order to make this package usable by any
php projects out there is:
1) To synchronize a separate read-only repository with the content of
https://github.com/apache/avro/tree/master/lang/php/lib (I'm looking at
various solution to do so) => required by 3).
2) To setup a serious CI/CD pipeline which jobs will be to run the tests
(against several php environments), check the code consistency and
synchronize the read only repository.
3) To register the php package on packagist.org
I would gladly take responsibility for these actions as I have experience
in this area and will directly benefit from them.
Problem is that this would require some administrative permission on the
org/repository and I would probably need a referent at the ASF to know how
things are done in your organization.
Is there already someone in charge of the PHP implementation I could
contact directly?
If you have some apply process to become an official maintainer as well,
please let me know.
Cheers.
*Gildas Quéméner*
On Tue, Dec 18, 2018 at 8:11 PM Daniel Kulp <dk...@apache.org> wrote:
> We would definitely welcome any php contributions. If you have any
> specific fixes or updates or anything, PR’s would be more than welcome.
> It would be great if someone with some PHP expertise could take some
> ownership in collecting various fixes/enhancements from various locations
> and getting pull requests submitted for review. That would be awesome.
>
> At this point, we won’t be splitting out the various languages to separate
> repositories. We have discussed it in the past and we MAY do it in the
> future, but right now we’re trying to concentrate on getting 1.9 out (and
> then likely a patch or two follow up) and splitting the repo right now
> would be too disruptive to that process.
>
>
> Dan
>
>
> > On Dec 18, 2018, at 1:26 AM, Gildas Quéméner <gi...@gmail.com>
> wrote:
> >
> > Hello,
> >
> > We've been working for a few months on using Avro encoded messages
> between
> > our services in my company.
> >
> > Our backend heavily relies on PHP, whereas some orbiting services are
> > written in Java/Kotlin/Python.
> > At first glance, the lake of official support for PHP looked like a big
> > no-go for us.
> >
> > I know that there is an official library, embedded in the avro library at
> > https://github.com/apache/avro/tree/master/lang/php/lib.
> >
> > However this library suffers from blocking issues:
> > - It is not available through the de facto standard Composer dependency
> > manager
> > - It is not tested
> > - It does not rely on autoloading PSR
> > - It uses global namespace
> > - It uses unknown php functions
> > - It lacks some spec features (support for logicalType attribute for
> > instance)
> > - It is buggy
> >
> > Despite these issues, latest commit is 1 year old (about a typo), and the
> > one before that is almost 3 years old. Thus, I think it is safe to say
> that
> > this library is not maintained anymore.
> >
> > Many organizations have realized such issue and have created their own
> fork
> > of the library, none of them being thoroughly maintained and having the
> > same level of bug fixing.
> >
> > Here are a few examples:
> > - https://github.com/wikimedia/avro-php
> > - https://github.com/flix-tech/avro-php
> > - https://github.com/researchgate/avro-php
> > - https://gitlab.com/Jaumo/avro-php (I am the maintainer of this one)
> >
> > I have tried to contribute to Wikimedia's fork, but they also seem to
> have
> > dropped support (see activity in
> > https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The
> awesome
> > thing is that they have brought a test suite (that we are reusing in our
> > fork)!
> >
> > Flix-tech is doing an awesome job at providing a Schema Registry API
> client
> > as well as an Avro Serializer/Deserializer using their Schema Registry
> > client.
> >
> > There are great initiatives out there to modernize this library, but none
> > of them trully gather the focus of the many maintainers, scattered
> through
> > private organizations.
> >
> > I think it is time to give avro-php some love and I would like to help in
> > this matter.
> >
> > My feeling is that the first step should be to open a separate git
> > repository under the hood of the ASF organization.
> > This could either be done through an automatic subtree of
> > https://github.com/apache/avro/tree/master/lang/php/lib to a read-only
> > repository (as Symfony is doing), in order to register it on
> packagist.org
> > or through the opening of a separate read/write repository (and the
> removal
> > of the lib from the avro lib repository).
> >
> > Second step is to communicate, directly by contacting fork maintainers
> > annoucing the good news, and through ASF official communication vectors
> > (thinking about ML, Twitter, ...). The goal being to bring back all the
> PHP
> > avro user to an official implementation of the library.
> >
> > Thank you very much for your answer,
> > And I wish you all fantastic time within friends and family for the end
> of
> > the year.
> >
> > Best Regards.
> >
> > *Gildas Quéméner*
>
> --
> Daniel Kulp
> dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <
> http://dankulp.com/blog>
> Talend Community Coder - http://talend.com <http://coders.talend.com/>
>
Re: Modern PHP support
Posted by Daniel Kulp <dk...@apache.org>.
We would definitely welcome any php contributions. If you have any specific fixes or updates or anything, PR’s would be more than welcome. It would be great if someone with some PHP expertise could take some ownership in collecting various fixes/enhancements from various locations and getting pull requests submitted for review. That would be awesome.
At this point, we won’t be splitting out the various languages to separate repositories. We have discussed it in the past and we MAY do it in the future, but right now we’re trying to concentrate on getting 1.9 out (and then likely a patch or two follow up) and splitting the repo right now would be too disruptive to that process.
Dan
> On Dec 18, 2018, at 1:26 AM, Gildas Quéméner <gi...@gmail.com> wrote:
>
> Hello,
>
> We've been working for a few months on using Avro encoded messages between
> our services in my company.
>
> Our backend heavily relies on PHP, whereas some orbiting services are
> written in Java/Kotlin/Python.
> At first glance, the lake of official support for PHP looked like a big
> no-go for us.
>
> I know that there is an official library, embedded in the avro library at
> https://github.com/apache/avro/tree/master/lang/php/lib.
>
> However this library suffers from blocking issues:
> - It is not available through the de facto standard Composer dependency
> manager
> - It is not tested
> - It does not rely on autoloading PSR
> - It uses global namespace
> - It uses unknown php functions
> - It lacks some spec features (support for logicalType attribute for
> instance)
> - It is buggy
>
> Despite these issues, latest commit is 1 year old (about a typo), and the
> one before that is almost 3 years old. Thus, I think it is safe to say that
> this library is not maintained anymore.
>
> Many organizations have realized such issue and have created their own fork
> of the library, none of them being thoroughly maintained and having the
> same level of bug fixing.
>
> Here are a few examples:
> - https://github.com/wikimedia/avro-php
> - https://github.com/flix-tech/avro-php
> - https://github.com/researchgate/avro-php
> - https://gitlab.com/Jaumo/avro-php (I am the maintainer of this one)
>
> I have tried to contribute to Wikimedia's fork, but they also seem to have
> dropped support (see activity in
> https://gerrit.wikimedia.org/r/q/status:open+project:avro-php). The awesome
> thing is that they have brought a test suite (that we are reusing in our
> fork)!
>
> Flix-tech is doing an awesome job at providing a Schema Registry API client
> as well as an Avro Serializer/Deserializer using their Schema Registry
> client.
>
> There are great initiatives out there to modernize this library, but none
> of them trully gather the focus of the many maintainers, scattered through
> private organizations.
>
> I think it is time to give avro-php some love and I would like to help in
> this matter.
>
> My feeling is that the first step should be to open a separate git
> repository under the hood of the ASF organization.
> This could either be done through an automatic subtree of
> https://github.com/apache/avro/tree/master/lang/php/lib to a read-only
> repository (as Symfony is doing), in order to register it on packagist.org
> or through the opening of a separate read/write repository (and the removal
> of the lib from the avro lib repository).
>
> Second step is to communicate, directly by contacting fork maintainers
> annoucing the good news, and through ASF official communication vectors
> (thinking about ML, Twitter, ...). The goal being to bring back all the PHP
> avro user to an official implementation of the library.
>
> Thank you very much for your answer,
> And I wish you all fantastic time within friends and family for the end of
> the year.
>
> Best Regards.
>
> *Gildas Quéméner*
--
Daniel Kulp
dkulp@apache.org <ma...@apache.org> - http://dankulp.com/blog <http://dankulp.com/blog>
Talend Community Coder - http://talend.com <http://coders.talend.com/>