You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by GitBox <gi...@apache.org> on 2021/08/27 09:56:31 UTC
[GitHub] [knox] zeroflag opened a new pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
zeroflag opened a new pull request #486:
URL: https://github.com/apache/knox/pull/486
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
1. Creating 2900 tokens
2. Loding the token management page
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] zeroflag commented on a change in pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
zeroflag commented on a change in pull request #486:
URL: https://github.com/apache/knox/pull/486#discussion_r698306001
##########
File path: gateway-server/src/test/java/org/apache/knox/gateway/services/token/impl/JDBCTokenStateServiceTest.java
##########
@@ -49,6 +36,25 @@
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.Locale;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
Review comment:
I reconfigured it so that static comes first, then java.* then org.*. This order matches the original order of the affected files. But I see exceptions, for example in GatewayServer.java, org.* imports are followed by java.* imports.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] zeroflag closed pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
zeroflag closed pull request #486:
URL: https://github.com/apache/knox/pull/486
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] zeroflag commented on pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
zeroflag commented on pull request #486:
URL: https://github.com/apache/knox/pull/486#issuecomment-907174484
cc: @pzampino @moresandeep @smolnar82 @lmccay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 commented on a change in pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #486:
URL: https://github.com/apache/knox/pull/486#discussion_r698049332
##########
File path: gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/TokenStateDatabase.java
##########
@@ -192,24 +191,34 @@ TokenMetadata getTokenMetadata(String tokenId) throws SQLException {
final Map<String, String> metadataMap = new HashMap<>();
while (rs.next()) {
String metadataName = rs.getString(1);
- metadataMap.put(metadataName, metadataName.equals(TokenMetadata.PASSCODE) ? new String(Base64.decodeBase64(rs.getString(2).getBytes(UTF_8)), UTF_8) : rs.getString(2));
+ metadataMap.put(metadataName, decodeMetadata(metadataName, rs.getString(2)));
}
return metadataMap.isEmpty() ? null : new TokenMetadata(metadataMap);
}
}
}
+ private static String decodeMetadata(String metadataName, String metadataValue) {
+ return metadataName.equals(TokenMetadata.PASSCODE) ? new String(Base64.decodeBase64(metadataValue.getBytes(UTF_8)), UTF_8) : metadataValue;
+ }
+
Collection<KnoxToken> getTokens(String userName) throws SQLException {
- final Collection<KnoxToken> tokens = new TreeSet<>();
+ Map<String, KnoxToken> tokenMap = new LinkedHashMap<>();
try (Connection connection = dataSource.getConnection(); PreparedStatement getTokenIdsStatement = connection.prepareStatement(GET_TOKENS_BY_USER_NAME_SQL)) {
getTokenIdsStatement.setString(1, userName);
try (ResultSet rs = getTokenIdsStatement.executeQuery()) {
while(rs.next()) {
- tokens.add(new KnoxToken(rs.getString(1), rs.getLong(2), rs.getLong(3), rs.getLong(4)));
+ String tokenId = rs.getString(1);
+ long issueTime = rs.getLong(2);
+ long expiration = rs.getLong(3);
+ long maxLifeTime = rs.getLong(4);
+ String metaName = rs.getString(5);
+ String metaValue = rs.getString(6);
+ KnoxToken token = tokenMap.computeIfAbsent(tokenId, id -> new KnoxToken(tokenId, issueTime, expiration, maxLifeTime));
Review comment:
There is a KnoxToken constructor that takes a TokenMetadata; I'd use that one and remove the constructor being used here (in fact I added the one w/o metadata to support the 'poor performance' getTokens method which you are fixing now).
In that case, a simple TressEt is enough instead of the LinkedHashMap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 commented on a change in pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #486:
URL: https://github.com/apache/knox/pull/486#discussion_r698049332
##########
File path: gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/TokenStateDatabase.java
##########
@@ -192,24 +191,34 @@ TokenMetadata getTokenMetadata(String tokenId) throws SQLException {
final Map<String, String> metadataMap = new HashMap<>();
while (rs.next()) {
String metadataName = rs.getString(1);
- metadataMap.put(metadataName, metadataName.equals(TokenMetadata.PASSCODE) ? new String(Base64.decodeBase64(rs.getString(2).getBytes(UTF_8)), UTF_8) : rs.getString(2));
+ metadataMap.put(metadataName, decodeMetadata(metadataName, rs.getString(2)));
}
return metadataMap.isEmpty() ? null : new TokenMetadata(metadataMap);
}
}
}
+ private static String decodeMetadata(String metadataName, String metadataValue) {
+ return metadataName.equals(TokenMetadata.PASSCODE) ? new String(Base64.decodeBase64(metadataValue.getBytes(UTF_8)), UTF_8) : metadataValue;
+ }
+
Collection<KnoxToken> getTokens(String userName) throws SQLException {
- final Collection<KnoxToken> tokens = new TreeSet<>();
+ Map<String, KnoxToken> tokenMap = new LinkedHashMap<>();
try (Connection connection = dataSource.getConnection(); PreparedStatement getTokenIdsStatement = connection.prepareStatement(GET_TOKENS_BY_USER_NAME_SQL)) {
getTokenIdsStatement.setString(1, userName);
try (ResultSet rs = getTokenIdsStatement.executeQuery()) {
while(rs.next()) {
- tokens.add(new KnoxToken(rs.getString(1), rs.getLong(2), rs.getLong(3), rs.getLong(4)));
+ String tokenId = rs.getString(1);
+ long issueTime = rs.getLong(2);
+ long expiration = rs.getLong(3);
+ long maxLifeTime = rs.getLong(4);
+ String metaName = rs.getString(5);
+ String metaValue = rs.getString(6);
+ KnoxToken token = tokenMap.computeIfAbsent(tokenId, id -> new KnoxToken(tokenId, issueTime, expiration, maxLifeTime));
Review comment:
There is a KnoxToken constructor that takes a TokenMetadata; I'd use that one and remove the constructor being used here (in fact I added the one w/o metadata to support the 'poor performance' getTokens method which you are fixing now).
In that case, a simple TressEt is enough instead of the LinkedHashMap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 commented on a change in pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #486:
URL: https://github.com/apache/knox/pull/486#discussion_r698048414
##########
File path: gateway-server/src/test/java/org/apache/knox/gateway/services/token/impl/JDBCTokenStateServiceTest.java
##########
@@ -49,6 +36,25 @@
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.Locale;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
Review comment:
We need to make sure your IDE is configured properly in terms of imports (i.e. static imports must come first)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 merged pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
smolnar82 merged pull request #486:
URL: https://github.com/apache/knox/pull/486
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 commented on a change in pull request #486: KNOX-2650 - Loading token management page can be slow if there are lots of tokens
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #486:
URL: https://github.com/apache/knox/pull/486#discussion_r698049332
##########
File path: gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/TokenStateDatabase.java
##########
@@ -192,24 +191,34 @@ TokenMetadata getTokenMetadata(String tokenId) throws SQLException {
final Map<String, String> metadataMap = new HashMap<>();
while (rs.next()) {
String metadataName = rs.getString(1);
- metadataMap.put(metadataName, metadataName.equals(TokenMetadata.PASSCODE) ? new String(Base64.decodeBase64(rs.getString(2).getBytes(UTF_8)), UTF_8) : rs.getString(2));
+ metadataMap.put(metadataName, decodeMetadata(metadataName, rs.getString(2)));
}
return metadataMap.isEmpty() ? null : new TokenMetadata(metadataMap);
}
}
}
+ private static String decodeMetadata(String metadataName, String metadataValue) {
+ return metadataName.equals(TokenMetadata.PASSCODE) ? new String(Base64.decodeBase64(metadataValue.getBytes(UTF_8)), UTF_8) : metadataValue;
+ }
+
Collection<KnoxToken> getTokens(String userName) throws SQLException {
- final Collection<KnoxToken> tokens = new TreeSet<>();
+ Map<String, KnoxToken> tokenMap = new LinkedHashMap<>();
try (Connection connection = dataSource.getConnection(); PreparedStatement getTokenIdsStatement = connection.prepareStatement(GET_TOKENS_BY_USER_NAME_SQL)) {
getTokenIdsStatement.setString(1, userName);
try (ResultSet rs = getTokenIdsStatement.executeQuery()) {
while(rs.next()) {
- tokens.add(new KnoxToken(rs.getString(1), rs.getLong(2), rs.getLong(3), rs.getLong(4)));
+ String tokenId = rs.getString(1);
+ long issueTime = rs.getLong(2);
+ long expiration = rs.getLong(3);
+ long maxLifeTime = rs.getLong(4);
+ String metaName = rs.getString(5);
+ String metaValue = rs.getString(6);
+ KnoxToken token = tokenMap.computeIfAbsent(tokenId, id -> new KnoxToken(tokenId, issueTime, expiration, maxLifeTime));
Review comment:
There is a KnoxToken constructor that takes a TokenMetadata; I'd use that one and remove the constructor being used here (in fact I added the one w/o metadata to support the 'poor performance' getTokens method which you are fixing now).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org