You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sdap.apache.org by "Frank Greguska (JIRA)" <ji...@apache.org> on 2019/06/26 16:08:00 UTC
[jira] [Commented] (SDAP-144) DOMS to support filtering by quality
[ https://issues.apache.org/jira/browse/SDAP-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873473#comment-16873473 ]
Frank Greguska commented on SDAP-144:
-------------------------------------
This class ([https://github.com/apache/incubator-sdap-nexus/blob/584b2f5b91300fd0952cb4de32068a008c551936/analysis/webservice/algorithms_spark/Matchup.py#L323]) is used as a generic “DomsPoint” that can represent either an in situ or a satellite measurement. The “from_edge_point” [https://github.com/apache/incubator-sdap-nexus/blob/584b2f5b91300fd0952cb4de32068a008c551936/analysis/webservice/algorithms_spark/Matchup.py#L398] function is responsible for translating from a document from edge into a DomsPoint.
This function then goes from DomsPoint to dict: [https://github.com/apache/incubator-sdap-nexus/blob/master/analysis/webservice/algorithms_spark/Matchup.py#L301]
Which is what eventually gets passed to the storage module: [https://github.com/apache/incubator-sdap-nexus/blob/584b2f5b91300fd0952cb4de32068a008c551936/analysis/webservice/algorithms/doms/ResultsStorage.py#L65]
Additionally, the doms data table will have to have a new column added to it in order to store the quality flag [https://github.com/apache/incubator-sdap-nexus/blob/584b2f5b91300fd0952cb4de32068a008c551936/analysis/webservice/algorithms/doms/DomsInitialization.py#L111]
All of these will need to be updated to keep track of the quality flags in order to be able to put them in the output file.
> DOMS to support filtering by quality
> ------------------------------------
>
> Key: SDAP-144
> URL: https://issues.apache.org/jira/browse/SDAP-144
> Project: Apache Science Data Analytics Platform
> Issue Type: New Feature
> Components: doms, nexus
> Reporter: Frank Greguska
> Assignee: Elizabeth Yam
> Priority: Major
>
> The matchup algorithm can be modified to utilize quality filtering of in situ data.
>
> This method is a good place to start because it is used to query for insitu data using EDGE.
> https://github.com/apache/incubator-sdap-nexus/blob/725ed5a7bb7db96662e5351f8cd373f4c54862d6/analysis/webservice/algorithms_spark/Matchup.py#L650
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)