You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/09/13 21:33:49 UTC

[GitHub] [pinot] dongxiaoman opened a new pull request, #9391: broker filter

dongxiaoman opened a new pull request, #9391:
URL: https://github.com/apache/pinot/pull/9391

   ## Add Authentication Filter to Broker Endpoints
   Right now the Broker does not have Authentication Filter installed.
   The `ClientRequest` (a.k.a. the `/sql/` endpoints) are taking AccessControlFactory instance and doing Auth in code;
   However the other endpoints like `/appconfigs` and `/debug/` , `/routing/` endpoints are not authenticated.
   
   In this PR we add AuthenticationFilter like what we do in Controller.
   
   ## Simplified Model
   Right now we only manage by Identity, it is not implemented for the heuristic control of Table yet.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] sajjad-moradi commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
sajjad-moradi commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972286109


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only
+   *
+   * @return {@code true} if annotated methods are protected only, {@code false} otherwise
+   */
+  default boolean protectAnnotatedOnly() {
+    return true;
+  }

Review Comment:
   You should be using @ManualAuthorization annotation for those endpoints, so AuthFilter will simply return with no actual authentication done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972167421


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only

Review Comment:
   Broker endpoints like `client/sql` (the one taking SQL queries) are doing access controls inside the endpoint. The endpoints takes one AccessControlFactory, and then calls AccessControl interface actively to verify.
   
   There is no need for those endpoints to be verified twice since they have access control baked in it already.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] jackjlli commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
jackjlli commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r971342975


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/HttpRequesterIdentity.java:
##########
@@ -30,6 +31,17 @@ public class HttpRequesterIdentity extends RequesterIdentity {
   private Multimap<String, String> _httpHeaders;
   private String _endpointUrl;
 
+  public static HttpRequesterIdentity fromRequest(org.glassfish.grizzly.http.server.Request request) {

Review Comment:
   nit: import `org.glassfish.grizzly.http.server.Request` instead of specifying the full path here?



##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only

Review Comment:
   Could you elaborate on this? Why do we need to mention controller endpoints in the pinot-broker module?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on PR #9391:
URL: https://github.com/apache/pinot/pull/9391#issuecomment-1256819236

   Having error `Error: [ERROR] error: error reading /home/runner/.m2/repository/com/uber/h3/4.0.0/h3-4.0.0.jar; error in opening zip file`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972415292


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   I was thinking that I have to keep those interfaces in pinot-controller. If it is fine to relocate them into different packages (downstream may fail to build), refactor will be much smaller.
   Let me come up with something



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on PR #9391:
URL: https://github.com/apache/pinot/pull/9391#issuecomment-1246099862

   @apucher can you help having a look? Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r975671806


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   Thanks, all fixed now. Sorry for the slow update but it should be working now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972168144


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only

Review Comment:
   My mistake, copied the method from controller, forgot to update comment. Will change it accordingly



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] sajjad-moradi commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
sajjad-moradi commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972284881


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   This is not an interface change. It's just moving the same interface to another module to prevent duplicate code.
   I assume the huge refactoring you're referring to is for just package name change, right? If that's the only change, we should go for it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r975671806


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   Thanks, all fixed now. Sorry for the slow update but it should be working now, please have another look @sajjad-moradi 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] apucher commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
apucher commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r983854279


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotClientRequest.java:
##########
@@ -195,6 +199,7 @@ public String cancelQuery(
   @ApiResponses(value = {
       @ApiResponse(code = 200, message = "Success"), @ApiResponse(code = 500, message = "Internal server error")
   })
+  @ManualAuthorization
   public Map<Long, String> getRunningQueries() {

Review Comment:
   is there any access control built into these?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972165082


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   I tried that in another attempt, such change will trigger a huge refactor.
   Am I allowed to make breaking changes like removing public interfaces in controller module? @apucher 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on PR #9391:
URL: https://github.com/apache/pinot/pull/9391#issuecomment-1262465981

   @apucher @sajjad-moradi can you have another look at the changes? Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r984799574


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/resources/PinotClientRequest.java:
##########
@@ -195,6 +199,7 @@ public String cancelQuery(
   @ApiResponses(value = {
       @ApiResponse(code = 200, message = "Success"), @ApiResponse(code = 500, message = "Internal server error")
   })
+  @ManualAuthorization
   public Map<Long, String> getRunningQueries() {

Review Comment:
   Good point. After reading code, there's no auth in there.  I have removed the one on `cancelQuery` too



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] sajjad-moradi commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
sajjad-moradi commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r971405384


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   This is already defined in controller module. Can you move it to a common place, so it's not duplicated in two places? Maybe in `pinot-common > src > main > java > org > apache > pinot > common > auth`?



##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only
+   *
+   * @return {@code true} if annotated methods are protected only, {@code false} otherwise
+   */
+  default boolean protectAnnotatedOnly() {
+    return true;
+  }

Review Comment:
   +1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] dongxiaoman commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972169945


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only
+   *
+   * @return {@code true} if annotated methods are protected only, {@code false} otherwise
+   */
+  default boolean protectAnnotatedOnly() {
+    return true;
+  }

Review Comment:
   I thought we can avoid calling Auth endpoints twice. `AuthenticationFilter` verifies once, the `client/sql` endpoints have its own AccessControl that calls again.
   Can we confirm that it is fine to just authenticate everything?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] apucher commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
apucher commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r972407174


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/ManualAuthorization.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation don't perform default
+ * authorization via AuthenticationFilter. This is useful when performing authorization manually via calls to
+ * {@code AuthenticationFiler.validatePermissions()}
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ManualAuthorization {
+}

Review Comment:
   the `@ManualAuthorization` annotation was just introduced recently. It should be fine to move it around.
   
   In terms of preventing a `huge refactor` - what issues are you running into there?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] apucher commented on a diff in pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
apucher commented on code in PR #9391:
URL: https://github.com/apache/pinot/pull/9391#discussion_r971344881


##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/Authenticate.java:
##########
@@ -0,0 +1,37 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.broker.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+import org.apache.pinot.spi.config.user.AccessType;
+
+
+/**
+ * Annotation to be used on top of REST endpoints. Methods annotated with this annotation automatically get
+ * authenticated in {@link AuthenticationFilter} and if validation passes, then the methods get executed.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface Authenticate {
+  AccessType value();
+}

Review Comment:
   same here. no need for the annotation. IMO broker auth filter should by default protect everything



##########
pinot-broker/src/main/java/org/apache/pinot/broker/api/AccessControl.java:
##########
@@ -47,4 +47,13 @@ default boolean hasAccess(RequesterIdentity requesterIdentity) {
    * @return {@code true} if authorized, {@code false} otherwise
    */
   boolean hasAccess(RequesterIdentity requesterIdentity, BrokerRequest brokerRequest);
+
+  /**
+   * Determine whether authentication is required for annotated (controller) endpoints only
+   *
+   * @return {@code true} if annotated methods are protected only, {@code false} otherwise
+   */
+  default boolean protectAnnotatedOnly() {
+    return true;
+  }

Review Comment:
   no need for this one here. it was introduced for backwards-compatibility in controller only. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] codecov-commenter commented on pull request #9391: broker filter

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9391:
URL: https://github.com/apache/pinot/pull/9391#issuecomment-1246040248

   # [Codecov](https://codecov.io/gh/apache/pinot/pull/9391?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#9391](https://codecov.io/gh/apache/pinot/pull/9391?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (de42452) into [master](https://codecov.io/gh/apache/pinot/commit/45cc5dd0e155578e9c3acd3efeb61cec239838cf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (45cc5dd) will **decrease** coverage by `54.40%`.
   > The diff coverage is `24.13%`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #9391       +/-   ##
   =============================================
   - Coverage     69.75%   15.35%   -54.41%     
   + Complexity     5095      170     -4925     
   =============================================
     Files          1885     1833       -52     
     Lines        100406    98173     -2233     
     Branches      15277    15024      -253     
   =============================================
   - Hits          70042    15070    -54972     
   - Misses        25414    81958    +56544     
   + Partials       4950     1145     -3805     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `15.35% <24.13%> (-0.02%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9391?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...ava/org/apache/pinot/broker/api/AccessControl.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL0FjY2Vzc0NvbnRyb2wuamF2YQ==) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [...apache/pinot/broker/api/HttpRequesterIdentity.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL0h0dHBSZXF1ZXN0ZXJJZGVudGl0eS5qYXZh) | `14.81% <0.00%> (-13.76%)` | :arrow_down: |
   | [...pinot/broker/api/resources/PinotClientRequest.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdENsaWVudFJlcXVlc3QuamF2YQ==) | `0.00% <ø> (-43.29%)` | :arrow_down: |
   | [...ache/pinot/broker/broker/AuthenticationFilter.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0F1dGhlbnRpY2F0aW9uRmlsdGVyLmphdmE=) | `16.66% <16.66%> (ø)` | |
   | [...pinot/broker/broker/BrokerAdminApiApplication.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0Jyb2tlckFkbWluQXBpQXBwbGljYXRpb24uamF2YQ==) | `82.25% <100.00%> (-2.75%)` | :arrow_down: |
   | [...e/pinot/broker/broker/helix/BaseBrokerStarter.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jhc2VCcm9rZXJTdGFydGVyLmphdmE=) | `72.09% <100.00%> (-6.79%)` | :arrow_down: |
   | [...src/main/java/org/apache/pinot/sql/FilterKind.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvRmlsdGVyS2luZC5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...ain/java/org/apache/pinot/core/data/table/Key.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL0tleS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...in/java/org/apache/pinot/spi/utils/BytesUtils.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvQnl0ZXNVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...n/java/org/apache/pinot/core/data/table/Table.java](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL1RhYmxlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | ... and [1435 more](https://codecov.io/gh/apache/pinot/pull/9391/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org