You are viewing a plain text version of this content. The canonical link for it is here.
Posted to custos@airavata.apache.org by "SauravKumarJhaNITW (via GitHub)" <gi...@apache.org> on 2023/08/12 19:27:34 UTC
[GitHub] [airavata] SauravKumarJhaNITW opened a new pull request, #433: Airavata 3594 admin dashboard additions
SauravKumarJhaNITW opened a new pull request, #433:
URL: https://github.com/apache/airavata/pull/433
added a method to return UserGroups filtered by creation date
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: custos-unsubscribe@airavata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airavata] machristie commented on a diff in pull request #433: Airavata 3594 admin dashboard additions
Posted by "machristie (via GitHub)" <gi...@apache.org>.
machristie commented on code in PR #433:
URL: https://github.com/apache/airavata/pull/433#discussion_r1312007360
##########
modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/JobStatusRepository.java:
##########
@@ -113,4 +117,51 @@ public List<JobStatus> getDistinctListofJobStatus(String gatewayId,String status
return jobStatusRepository.selectWithNativeQuery(QueryConstants.FIND_JOB_COUNT_NATIVE_QUERY,
gatewayId,status,String.valueOf(time));
}
+
+ public List<CpuUsage> getCpuUsages(String gatewayId, long fromTime, long toTime) {
+ JobStatusRepository jobStatusRepository = new JobStatusRepository();
+ List<CpuUsage> cpuUsages = new ArrayList<>();
+ List<JobStatus> startedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+ QueryConstants.FIND_JOB_STATUS_STARTED_BEFORE_TIME, gatewayId, JobState.ACTIVE.name(),
+ String.valueOf(toTime));
+ ;
+ List<JobStatus> finishedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+ QueryConstants.FIND_JOB_STATUS_FINISHED_AFTER_TIME, gatewayId, JobState.COMPLETE.name(),
+ JobState.FAILED.name(), JobState.CANCELED.name(), String.valueOf(fromTime));
+ Mapper mapper = ObjectMapperSingleton.getInstance();
+ for (JobStatus startedJobStatus : startedJobStatusList) {
+ for (JobStatus finishedJobStatus : finishedJobStatusList) {
+ JobStatusEntity startedJobStatusEntity = mapper.map(startedJobStatus, JobStatusEntity.class);
+ JobStatusEntity finishedJobStatusEntity = mapper.map(finishedJobStatus, JobStatusEntity.class);
+ if (startedJobStatusEntity.getJobId() == finishedJobStatusEntity.getJobId()) {
Review Comment:
I'm pretty sure this won't work. Mapping a JobStatus back to an Entity won't populate the extra fields (like `getJobId()`) that are stored on the database. But see my comment above, if that approach works you won't need to do this.
##########
modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/JobStatusRepository.java:
##########
@@ -113,4 +117,51 @@ public List<JobStatus> getDistinctListofJobStatus(String gatewayId,String status
return jobStatusRepository.selectWithNativeQuery(QueryConstants.FIND_JOB_COUNT_NATIVE_QUERY,
gatewayId,status,String.valueOf(time));
}
+
+ public List<CpuUsage> getCpuUsages(String gatewayId, long fromTime, long toTime) {
+ JobStatusRepository jobStatusRepository = new JobStatusRepository();
+ List<CpuUsage> cpuUsages = new ArrayList<>();
+ List<JobStatus> startedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+ QueryConstants.FIND_JOB_STATUS_STARTED_BEFORE_TIME, gatewayId, JobState.ACTIVE.name(),
+ String.valueOf(toTime));
+ ;
+ List<JobStatus> finishedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+ QueryConstants.FIND_JOB_STATUS_FINISHED_AFTER_TIME, gatewayId, JobState.COMPLETE.name(),
+ JobState.FAILED.name(), JobState.CANCELED.name(), String.valueOf(fromTime));
+ Mapper mapper = ObjectMapperSingleton.getInstance();
+ for (JobStatus startedJobStatus : startedJobStatusList) {
+ for (JobStatus finishedJobStatus : finishedJobStatusList) {
+ JobStatusEntity startedJobStatusEntity = mapper.map(startedJobStatus, JobStatusEntity.class);
+ JobStatusEntity finishedJobStatusEntity = mapper.map(finishedJobStatus, JobStatusEntity.class);
+ if (startedJobStatusEntity.getJobId() == finishedJobStatusEntity.getJobId()) {
+ String jobId = startedJobStatusEntity.getJobId();
+ List<ExperimentModel> experiments = (new ExperimentRepository())
+ .selectWithNativeQuery(QueryConstants.FIND_EXPERIMENT_WITH_JOB_ID, jobId);
+ if (!experiments.isEmpty()) {
+ ExperimentModel experiment = experiments.get(0);
+ List<Integer> res = (new ExperimentRepository().selectWithNativeQuery(
Review Comment:
Do you need this query? You've already loaded the ExperimentModel so I think you can just use `experiment.getUserConfigurationData().getComputationalResourceScheduling().getTotalCPUCount()`.
##########
modules/registry/registry-core/src/main/java/org/apache/airavata/registry/core/repositories/expcatalog/JobStatusRepository.java:
##########
@@ -113,4 +117,51 @@ public List<JobStatus> getDistinctListofJobStatus(String gatewayId,String status
return jobStatusRepository.selectWithNativeQuery(QueryConstants.FIND_JOB_COUNT_NATIVE_QUERY,
gatewayId,status,String.valueOf(time));
}
+
+ public List<CpuUsage> getCpuUsages(String gatewayId, long fromTime, long toTime) {
+ JobStatusRepository jobStatusRepository = new JobStatusRepository();
+ List<CpuUsage> cpuUsages = new ArrayList<>();
+ List<JobStatus> startedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+ QueryConstants.FIND_JOB_STATUS_STARTED_BEFORE_TIME, gatewayId, JobState.ACTIVE.name(),
+ String.valueOf(toTime));
+ ;
+ List<JobStatus> finishedJobStatusList = jobStatusRepository.selectWithNativeQuery(
+ QueryConstants.FIND_JOB_STATUS_FINISHED_AFTER_TIME, gatewayId, JobState.COMPLETE.name(),
+ JobState.FAILED.name(), JobState.CANCELED.name(), String.valueOf(fromTime));
+ Mapper mapper = ObjectMapperSingleton.getInstance();
+ for (JobStatus startedJobStatus : startedJobStatusList) {
Review Comment:
This double for loop isn't going to scale well for gateways with a large number of jobs. It would be better to do this all in the query. I would suggest something like this:
```sql
select j.* from JOB j
inner join PROCESS p on p.PROCESS_ID = j.PROCESS_ID
inner join EXPERIMENT e on e.EXPERIMENT_ID = p.EXPERIMENT_ID
where e.GATEWAY_ID = ?
and exists (select 1 from JOB_STATUS js where js.JOB_ID = j.JOB_ID and js.STATE = ? and js.TIME_OF_STATE_CHANGE <= ?)
and exists (select 1 from JOB_STATUS js where js.JOB_ID = j.JOB_ID and (js.STATE = ? or js.STATE = ? or js.STATE = ?) and js.TIME_OF_STATE_CHANGE >= ?)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@airavata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Airavata 3594 admin dashboard additions [airavata]
Posted by "SauravKumarJhaNITW (via GitHub)" <gi...@apache.org>.
SauravKumarJhaNITW commented on PR #433:
URL: https://github.com/apache/airavata/pull/433#issuecomment-1837153656
Thanks a lot @machristie for your comments. Comments were very easy to follow along, really appreciated. I have made changes based on these. Please review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@airavata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Airavata 3594 admin dashboard additions [airavata]
Posted by "lahirujayathilake (via GitHub)" <gi...@apache.org>.
lahirujayathilake closed pull request #433: Airavata 3594 admin dashboard additions
URL: https://github.com/apache/airavata/pull/433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: custos-unsubscribe@airavata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org