You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/02/06 06:49:36 UTC

[GitHub] [druid] mitchlloyd opened a new pull request #9319: Make Transform an ExtensionPoint

mitchlloyd opened a new pull request #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319

Fixes #9311.

### Description

`Transform` is a useful place for 3rd party developers to add their own extensions. Informally this has been an available extension point since 2017. This PR proposes adding the compatibility guarantees of @ExtensionPoint.

This PR has:
- [ ] been self-reviewed.
- [ ] added documentation for new or modified features or behaviors.

ToDo:
- [ ] Add documentation changes to https://druid.apache.org/docs/latest/development/modules.html#writing-your-own-extensions

It makes sense to me to add a new item in the list of 15 items under "Writing your own extensions" for completeness sake. I'd be happy to write a dedicated section of these docs, although I think this page seems destined to be split into many pages: one describing the approach for each kind of extension.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] suneet-s commented on a change in pull request #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

suneet-s commented on a change in pull request #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#discussion_r375671116

##########
File path: docs/development/modules.md
##########
@@ -46,6 +46,7 @@ Druid's extensions leverage Guice in order to add things at runtime. Basically,
1. Add new Jersey resources by calling `Jerseys.addResource(binder, clazz)`.
1. Add new Jetty filters by extending `org.apache.druid.server.initialization.jetty.ServletFilterHolder`.
1. Add new secret providers by extending `org.apache.druid.metadata.PasswordProvider`.
+1. Add new ingest transform by implementing the `org.apache.druid.segment.transform.Transform` interface from the `druid-processing` package.

Review comment:
Can you add a section in here describing how to register the custom transformation similar to what's documented for adding support for an Input Source (https://github.com/apache/druid/blob/118e89c1e2ae9ea0e1c923c355bff6f1ccacf756/docs/development/modules.md#adding-support-for-a-new-input-source) or a Password Provider (https://github.com/apache/druid/blob/118e89c1e2ae9ea0e1c923c355bff6f1ccacf756/docs/development/modules.md#adding-a-new-password-provider-implementation)

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] jihoonson commented on a change in pull request #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

jihoonson commented on a change in pull request #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#discussion_r377360313

Review comment:
> It makes sense to me to add a new item in the list of 15 items under "Writing your own extensions" for completeness sake. I'd be happy to write a dedicated section of these docs, although I think this page seems destined to be split into many pages: one describing the approach for each kind of extension.

Hi @mitchlloyd, I assume you are interested in writing docs what @suneet-s mentioned based on your comment above. Do you want to add them in this PR? I think that could be done in a follow-up PR if you want.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] mitchlloyd commented on issue #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

mitchlloyd commented on issue #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#issuecomment-588043844
 
 
   @suneet-s, @jihoonson I added an example implementation for a Transform extension. I realize this is a little more verbose than the other sections, but the full implementation with imports would have helped me out. Happy to trim it down by removing imports, class declarations or anything else. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] suneet-s commented on issue #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

suneet-s commented on issue #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#issuecomment-594793267
 
 
   > @suneet-s do you have more comments?
   
   lgtm !

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] jihoonson commented on issue #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

jihoonson commented on issue #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#issuecomment-584487646
 
 
   @mitchlloyd great, thanks!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] jihoonson merged pull request #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

jihoonson merged pull request #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] jihoonson commented on issue #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

jihoonson commented on issue #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#issuecomment-594787279
 
 
   @suneet-s do you have more comments?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

[GitHub] [druid] mitchlloyd commented on issue #9319: Make Transform an ExtensionPoint

Posted by GitBox <gi...@apache.org>.

mitchlloyd commented on issue #9319: Make Transform an ExtensionPoint
URL: https://github.com/apache/druid/pull/9319#issuecomment-584468534
 
 
   I’m happy to take at shot at more documentation in this PR. I should have a
   chance to revisit this week.
   
   On Mon, Feb 10, 2020 at 2:40 PM Jihoon Son <no...@github.com> wrote:
   
   > *@jihoonson* commented on this pull request.
   > ------------------------------
   >
   > In docs/development/modules.md
   > <https://github.com/apache/druid/pull/9319#discussion_r377360313>:
   >
   > > @@ -46,6 +46,7 @@ Druid's extensions leverage Guice in order to add things at runtime.  Basically,
   >  1. Add new Jersey resources by calling `Jerseys.addResource(binder, clazz)`.
   >  1. Add new Jetty filters by extending `org.apache.druid.server.initialization.jetty.ServletFilterHolder`.
   >  1. Add new secret providers by extending `org.apache.druid.metadata.PasswordProvider`.
   > +1. Add new ingest transform by implementing the `org.apache.druid.segment.transform.Transform` interface from the `druid-processing` package.
   >
   > It makes sense to me to add a new item in the list of 15 items under
   > "Writing your own extensions" for completeness sake. I'd be happy to write
   > a dedicated section of these docs, although I think this page seems
   > destined to be split into many pages: one describing the approach for each
   > kind of extension.
   >
   > Hi @mitchlloyd <https://github.com/mitchlloyd>, I assume you are
   > interested in writing docs what @suneet-s <https://github.com/suneet-s>
   > mentioned based on your comment above. Do you want to add them in this PR?
   > I think that could be done in a follow-up PR if you want.
   >
   > —
   > You are receiving this because you were mentioned.
   >
   >
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/druid/pull/9319?email_source=notifications&email_token=AAADWQPAJMXRT47DZ4NL4XLRCHJUZA5CNFSM4KQXQN52YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCU6PYFQ#discussion_r377360313>,
   > or unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/AAADWQIRNDWSRSEOEWZEB7TRCHJUZANCNFSM4KQXQN5Q>
   > .
   >
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org