You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@airavata.apache.org by "machristie (via GitHub)" <gi...@apache.org> on 2023/08/31 18:15:50 UTC

[GitHub] [airavata] machristie commented on a diff in pull request #433: Airavata 3594 admin dashboard additions

machristie commented on code in PR #433:
URL: https://github.com/apache/airavata/pull/433#discussion_r1312007360


##########
modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/JobStatusRepository.java:
##########
@@ -113,4 +117,51 @@ public List<JobStatus> getDistinctListofJobStatus(String gatewayId,String status
         return  jobStatusRepository.selectWithNativeQuery(QueryConstants.FIND_JOB_COUNT_NATIVE_QUERY,
                 gatewayId,status,String.valueOf(time));
     }
+
+    public List<CpuUsage> getCpuUsages(String gatewayId, long fromTime, long toTime) {
+        JobStatusRepository jobStatusRepository = new JobStatusRepository();
+        List<CpuUsage> cpuUsages = new ArrayList<>();
+        List<JobStatus> startedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+                QueryConstants.FIND_JOB_STATUS_STARTED_BEFORE_TIME, gatewayId, JobState.ACTIVE.name(),
+                String.valueOf(toTime));
+        ;
+        List<JobStatus> finishedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+                QueryConstants.FIND_JOB_STATUS_FINISHED_AFTER_TIME, gatewayId, JobState.COMPLETE.name(),
+                JobState.FAILED.name(), JobState.CANCELED.name(), String.valueOf(fromTime));
+        Mapper mapper = ObjectMapperSingleton.getInstance();
+        for (JobStatus startedJobStatus : startedJobStatusList) {
+            for (JobStatus finishedJobStatus : finishedJobStatusList) {
+                JobStatusEntity startedJobStatusEntity = mapper.map(startedJobStatus, JobStatusEntity.class);
+                JobStatusEntity finishedJobStatusEntity = mapper.map(finishedJobStatus, JobStatusEntity.class);
+                if (startedJobStatusEntity.getJobId() == finishedJobStatusEntity.getJobId()) {

Review Comment:
   I'm pretty sure this won't work. Mapping a JobStatus back to an Entity won't populate the extra fields (like `getJobId()`) that are stored on the database. But see my comment above, if that approach works you won't need to do this.



##########
modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/JobStatusRepository.java:
##########
@@ -113,4 +117,51 @@ public List<JobStatus> getDistinctListofJobStatus(String gatewayId,String status
         return  jobStatusRepository.selectWithNativeQuery(QueryConstants.FIND_JOB_COUNT_NATIVE_QUERY,
                 gatewayId,status,String.valueOf(time));
     }
+
+    public List<CpuUsage> getCpuUsages(String gatewayId, long fromTime, long toTime) {
+        JobStatusRepository jobStatusRepository = new JobStatusRepository();
+        List<CpuUsage> cpuUsages = new ArrayList<>();
+        List<JobStatus> startedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+                QueryConstants.FIND_JOB_STATUS_STARTED_BEFORE_TIME, gatewayId, JobState.ACTIVE.name(),
+                String.valueOf(toTime));
+        ;
+        List<JobStatus> finishedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+                QueryConstants.FIND_JOB_STATUS_FINISHED_AFTER_TIME, gatewayId, JobState.COMPLETE.name(),
+                JobState.FAILED.name(), JobState.CANCELED.name(), String.valueOf(fromTime));
+        Mapper mapper = ObjectMapperSingleton.getInstance();
+        for (JobStatus startedJobStatus : startedJobStatusList) {
+            for (JobStatus finishedJobStatus : finishedJobStatusList) {
+                JobStatusEntity startedJobStatusEntity = mapper.map(startedJobStatus, JobStatusEntity.class);
+                JobStatusEntity finishedJobStatusEntity = mapper.map(finishedJobStatus, JobStatusEntity.class);
+                if (startedJobStatusEntity.getJobId() == finishedJobStatusEntity.getJobId()) {
+                    String jobId = startedJobStatusEntity.getJobId();
+                    List<ExperimentModel> experiments = (new ExperimentRepository())
+                            .selectWithNativeQuery(QueryConstants.FIND_EXPERIMENT_WITH_JOB_ID, jobId);
+                    if (!experiments.isEmpty()) {
+                        ExperimentModel experiment = experiments.get(0);
+                        List<Integer> res = (new ExperimentRepository().selectWithNativeQuery(

Review Comment:
   Do you need this query? You've already loaded the ExperimentModel so I think you can just use `experiment.getUserConfigurationData().getComputationalResourceScheduling().getTotalCPUCount()`.



##########
modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/JobStatusRepository.java:
##########
@@ -113,4 +117,51 @@ public List<JobStatus> getDistinctListofJobStatus(String gatewayId,String status
         return  jobStatusRepository.selectWithNativeQuery(QueryConstants.FIND_JOB_COUNT_NATIVE_QUERY,
                 gatewayId,status,String.valueOf(time));
     }
+
+    public List<CpuUsage> getCpuUsages(String gatewayId, long fromTime, long toTime) {
+        JobStatusRepository jobStatusRepository = new JobStatusRepository();
+        List<CpuUsage> cpuUsages = new ArrayList<>();
+        List<JobStatus> startedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+                QueryConstants.FIND_JOB_STATUS_STARTED_BEFORE_TIME, gatewayId, JobState.ACTIVE.name(),
+                String.valueOf(toTime));
+        ;
+        List<JobStatus> finishedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+                QueryConstants.FIND_JOB_STATUS_FINISHED_AFTER_TIME, gatewayId, JobState.COMPLETE.name(),
+                JobState.FAILED.name(), JobState.CANCELED.name(), String.valueOf(fromTime));
+        Mapper mapper = ObjectMapperSingleton.getInstance();
+        for (JobStatus startedJobStatus : startedJobStatusList) {

Review Comment:
   This double for loop isn't going to scale well for gateways with a large number of jobs. It would be better to do this all in the query. I would suggest something like this:
   
   ```sql
   select j.* from JOB j
   inner join PROCESS p on p.PROCESS_ID = j.PROCESS_ID
   inner join EXPERIMENT e on e.EXPERIMENT_ID = p.EXPERIMENT_ID
   where e.GATEWAY_ID = ?
   and exists (select 1 from JOB_STATUS js where js.JOB_ID = j.JOB_ID and js.STATE = ? and js.TIME_OF_STATE_CHANGE <= ?)
   and exists (select 1 from JOB_STATUS js where js.JOB_ID = j.JOB_ID and (js.STATE = ? or js.STATE = ? or js.STATE = ?) and js.TIME_OF_STATE_CHANGE >= ?)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@airavata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org