You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/01/17 20:28:00 UTC
[GitHub] [druid] a2l007 opened a new issue #9211: Reanimate Pig specific
handling in the Avro parser
a2l007 opened a new issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211
### Description
Apache Pig dependencies as well as related functionality were removed from the Avro parser in #7810
We internally use Pig extensively and the Pig specific transformation within the Avro parser extension is useful for us especially for parsing data bags. I would like to add the `fromPigAvroStorage` flag based processing back to the Avro Parser. In terms of tests, I can find a way to test this without adding the Pig dependency back.
I apologize for missing out on raising this concern on the original PR.
Any objections @gianm @Fokko ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm edited a comment on issue #9211: Reanimate Pig
specific handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
gianm edited a comment on issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211#issuecomment-575812774
Hmm, I originally thought it made sense to drop Pig support since (a) it was blocking updating dependencies, and (b) it seems like Pig is no longer actively being maintained (the last release was 2.5 years ago).
But I guess I think that if Pig support can be re-introduced without causing problems with getting stuck on old versions of dependencies, then that is fine by me. That was what @Fokko was trying to unblock originally in #7810.
@Fokko what do you think? (Also @ccaominh, any opinion? — I know you also did some dependency-management related work recently.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm commented on issue #9211: Reanimate Pig specific
handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
gianm commented on issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211#issuecomment-575812774
Hmm, I think if Pig support can be re-introduced without causing problems with getting stuck on old versions of dependencies, then that is fine by me. That was what @Fokko was trying to unblock originally in #7810.
@Fokko what do you think? (Also @ccaominh, any opinion? — I know you also did some dependency-management related work recently.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] Fokko commented on issue #9211: Reanimate Pig specific
handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
Fokko commented on issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211#issuecomment-578637398
I'm fine with re-introducing pig support in a lightweight manner. It would be great if we can keep the pig dependency out of it since it hasn't been updated for a very long time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] a2l007 commented on issue #9211: Reanimate Pig specific
handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
a2l007 commented on issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211#issuecomment-578527081
Thanks for being supportive of the usecase. I've raised a PR for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson closed issue #9211: Reanimate Pig specific
handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
jihoonson closed issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] gianm edited a comment on issue #9211: Reanimate Pig
specific handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
gianm edited a comment on issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211#issuecomment-575812774
Hmm, I originally thought it made sense to drop Pig support since (a) it was blocking updating dependencies, and (b) it seems like Pig is no longer actively being maintained (the last release was 2.5 years ago).
But I guess I think that if Pig support can be re-introduced with good tests and without causing problems with getting stuck on old versions of dependencies, then that is fine by me. That was what @Fokko was trying to unblock originally in #7810.
@Fokko what do you think? (Also @ccaominh, any opinion? — I know you also did some dependency-management related work recently.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ccaominh commented on issue #9211: Reanimate Pig specific
handling in the Avro parser
Posted by GitBox <gi...@apache.org>.
ccaominh commented on issue #9211: Reanimate Pig specific handling in the Avro parser
URL: https://github.com/apache/druid/issues/9211#issuecomment-575850615
As long as we can still use newer versions of dependencies I don't see any problems. (For example, avro needs to be on the latest 1.9.1 version as it addresses security vulnerabilities present in older versions.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org