You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/10/13 12:10:01 UTC
[GitHub] [iotdb] Beyyes commented on a diff in pull request #7575: [IOTDB-4589] Register/recover trigger when registering/restart a datanode
Beyyes commented on code in PR #7575:
URL: https://github.com/apache/iotdb/pull/7575#discussion_r994533440
##########
confignode/src/main/java/org/apache/iotdb/confignode/consensus/response/DataNodeRegisterResp.java:
##########
@@ -70,6 +72,14 @@ public void setTemplateInfo(byte[] templateInfo) {
this.templateInfo = templateInfo;
}
+ public List<ByteBuffer> getTriggerInformation() {
Review Comment:
getAllTriggerInformation
##########
server/src/main/java/org/apache/iotdb/db/service/DataNode.java:
##########
@@ -101,6 +109,11 @@ private DataNode() {
private static final RegisterManager registerManager = new RegisterManager();
public static ServiceProvider serviceProvider;
+ /** store the list when registering in config node for preparing trigger related resources */
+ private List<TriggerInformation> triggerInformationList;
Review Comment:
Maybe it's better to maintain the `triggerInformationList` (informations about trigger) in an independent class? Because in the future CQ and other features are integrated, the variables amount of `DataNode` maybe large.
##########
server/src/main/java/org/apache/iotdb/db/service/DataNode.java:
##########
@@ -397,6 +418,102 @@ private void initTriggerRelatedInstance() throws StartupException {
}
}
+ private void prepareTriggerResources() throws StartupException {
+ initTriggerRelatedInstance();
+ if (triggerInformationList == null || triggerInformationList.isEmpty()) {
+ return;
+ }
+
+ // get jars from config node
+ List<TriggerInformation> triggerNeedJarList = getTriggerNeedJarList();
+ int index = 0;
+ while (index < triggerNeedJarList.size()) {
+ List<TriggerInformation> curList = new ArrayList<>();
+ int offset = 0;
+ while (offset < JAR_NUM_OF_ONE_RPC && index + offset < triggerNeedJarList.size()) {
+ curList.add(triggerNeedJarList.get(index + offset));
+ offset++;
+ }
+ index += (offset + 1);
+ getJarOfTriggers(curList);
+ }
+
+ // create instances of triggers and do registration
+ try {
+ for (TriggerInformation triggerInformation : triggerInformationList) {
+ TriggerManagementService.getInstance().doRegister(triggerInformation);
+ }
+ } catch (Exception e) {
+ throw new StartupException(e);
+ }
+ logger.debug("successfully registered all the triggers");
+ for (TriggerInformation triggerInformation :
+ TriggerManagementService.getInstance().getAllTriggerInformationInTriggerTable()) {
+ logger.debug("get trigger: {}", triggerInformation.getTriggerName());
+ }
+ for (TriggerExecutor triggerExecutor :
+ TriggerManagementService.getInstance().getAllTriggerExecutors()) {
+ logger.debug(
+ "get trigger executor: {}", triggerExecutor.getTriggerInformation().getTriggerName());
+ }
+ }
+
+ private void getJarOfTriggers(List<TriggerInformation> triggerInformationList)
+ throws StartupException {
+ try (ConfigNodeClient configNodeClient = new ConfigNodeClient()) {
+ List<String> jarNameList =
+ triggerInformationList.stream()
+ .map(TriggerInformation::getJarName)
+ .collect(Collectors.toList());
+ TGetTriggerJarResp resp = configNodeClient.getTriggerJar(new TGetTriggerJarReq(jarNameList));
+ if (resp.getStatus().getCode() == TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()) {
+ throw new StartupException("Failed to get trigger jar from config node.");
+ }
+ List<ByteBuffer> jarList = resp.getJarList();
+ for (int i = 0, n = triggerInformationList.size(); i < n; i++) {
Review Comment:
```suggestion
for (int i = 0; i < triggerInformationList.size(); i++) {
``` is tidy
##########
server/src/main/java/org/apache/iotdb/db/service/DataNode.java:
##########
@@ -134,6 +147,8 @@ protected void doAddNode() {
prepareDataNode();
// register current DataNode to ConfigNode
registerInConfigNode();
+ // get resources for trigger,udf...
+ prepareResources();
Review Comment:
Do we have any tests on real cluster? I'm not sure if there exists problems if put `prepareResources()` ahead of `active()` and `setUpRPCService()`
##########
server/src/main/java/org/apache/iotdb/db/service/DataNode.java:
##########
@@ -397,6 +418,102 @@ private void initTriggerRelatedInstance() throws StartupException {
}
}
+ private void prepareTriggerResources() throws StartupException {
+ initTriggerRelatedInstance();
+ if (triggerInformationList == null || triggerInformationList.isEmpty()) {
+ return;
+ }
+
+ // get jars from config node
+ List<TriggerInformation> triggerNeedJarList = getTriggerNeedJarList();
+ int index = 0;
+ while (index < triggerNeedJarList.size()) {
+ List<TriggerInformation> curList = new ArrayList<>();
+ int offset = 0;
+ while (offset < JAR_NUM_OF_ONE_RPC && index + offset < triggerNeedJarList.size()) {
+ curList.add(triggerNeedJarList.get(index + offset));
+ offset++;
+ }
+ index += (offset + 1);
+ getJarOfTriggers(curList);
+ }
+
+ // create instances of triggers and do registration
+ try {
+ for (TriggerInformation triggerInformation : triggerInformationList) {
+ TriggerManagementService.getInstance().doRegister(triggerInformation);
+ }
+ } catch (Exception e) {
+ throw new StartupException(e);
+ }
+ logger.debug("successfully registered all the triggers");
+ for (TriggerInformation triggerInformation :
+ TriggerManagementService.getInstance().getAllTriggerInformationInTriggerTable()) {
+ logger.debug("get trigger: {}", triggerInformation.getTriggerName());
+ }
+ for (TriggerExecutor triggerExecutor :
+ TriggerManagementService.getInstance().getAllTriggerExecutors()) {
+ logger.debug(
+ "get trigger executor: {}", triggerExecutor.getTriggerInformation().getTriggerName());
+ }
+ }
+
+ private void getJarOfTriggers(List<TriggerInformation> triggerInformationList)
+ throws StartupException {
+ try (ConfigNodeClient configNodeClient = new ConfigNodeClient()) {
+ List<String> jarNameList =
+ triggerInformationList.stream()
+ .map(TriggerInformation::getJarName)
+ .collect(Collectors.toList());
+ TGetTriggerJarResp resp = configNodeClient.getTriggerJar(new TGetTriggerJarReq(jarNameList));
+ if (resp.getStatus().getCode() == TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()) {
+ throw new StartupException("Failed to get trigger jar from config node.");
+ }
+ List<ByteBuffer> jarList = resp.getJarList();
+ for (int i = 0, n = triggerInformationList.size(); i < n; i++) {
+ TriggerExecutableManager.getInstance()
+ .writeToLibDir(jarList.get(i), triggerInformationList.get(i).getJarName());
+ }
+ } catch (IOException | TException e) {
+ throw new StartupException(e);
+ }
+ }
+
+ private List<TriggerInformation> getTriggerNeedJarList() {
Review Comment:
```suggestion
private List<TriggerInformation> getJarListForTrigger() {
```
is better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org