[GitHub] [gobblin] umustafi commented on a diff in pull request #3640: [GOBBLIN-1783] Initialize scheduler with batch gets instead of individual get per flow

["umustafi (via GitHub)" <gi...@apache.org>]

umustafi commented on code in PR #3640:
URL: https://github.com/apache/gobblin/pull/3640#discussion_r1103408096


##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/spec_store/MysqlBaseSpecStore.java:
##########
@@ -84,6 +84,7 @@ public class MysqlBaseSpecStore extends InstrumentedSpecStore {
   private static final String GET_ALL_STATEMENT = "SELECT spec_uri, spec FROM %s";
   private static final String GET_ALL_URIS_STATEMENT = "SELECT spec_uri FROM %s";
   private static final String GET_ALL_URIS_WITH_TAG_STATEMENT = "SELECT spec_uri FROM %s WHERE tag = ?";
+  private static final String GET_SPECS_BATCH_STATEMENT = "SELECT spec_uri, spec FROM %s ORDER BY spec_uri ASC LIMIT ? OFFSET ?";

Review Comment:
   the `spec_uri` is the primary key [here](https://jarvis.corp.linkedin.com/codesearch/result/?name=MysqlBaseSpecStore.java&path=gobblin-elr%2Fgobblin-runtime%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fruntime%2Fspec_store&reponame=linkedin%2Fgobblin-elr#91) in `MysqlBaseSpecStore` and in the `MysqlSpecStore` so ordering on spec_uri should be unique. Let me add a comment in the docstring of the paginate function. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org