You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@sdap.apache.org by "skorper (via GitHub)" <gi...@apache.org> on 2024/02/16 02:02:36 UTC

[PR] SDAP-505: Support DOMS insitu API [incubator-sdap-nexus]

skorper opened a new pull request, #299:
URL: https://github.com/apache/incubator-sdap-nexus/pull/299

   - Cast platform to string. SDAP requires platform to be a string (db col is str) but the DOMS insitu API seems to return an integer.
   - Add mechanism for not supporting insitu schema. As far as I know no schema endpoint exists in DOMS, so in this case we'd set the schema endpoint to None. 
   - Removed cloud insitu nodes from conf. Added DOMS insitu nodes
   
   ---
   
   ### Testing
   
   All testing was done on a local instance of SDAP where Cass/Solr are running locally via Docker. I ingested AVHRR_OI_L4_GHRSST_NCEI 2018-08-01 - 2018-08-16 data prior to running this test.
   
   Test query:
   
   ```bash
   localhost:8083/match_spark?primary=AVHRR_OI_L4_GHRSST_NCEI&secondary=SAMOS&startTime=2018-08-01T00%3A00%3A00Z&endTime=2018-08-01T23%3A59%3A59Z&b=-122%2C%2026%2C%20-114%2C%2034&platforms=30&depthMin=-1&depthMax=1&tt=43200&rt=18750&matchOnce=true&prioritizeDistance=true
   ```
   
   Job completes successfully. Results (JSON `/cdmsresults` endpoint)
   
   ```json
   {
   	"executionId": "fe87f866-a415-494e-851a-7a666fdfe5f3",
   	"data": [{
   		"platform": "orbiting satellite",
   		"device": "radiometers",
   		"lon": "-117.375",
   		"lat": "32.625",
   		"point": "Point(-117.375 32.625)",
   		"time": 1533081600,
   		"depth": null,
   		"fileurl": "20180801120000-NCEI-L4_GHRSST-SSTblend-AVHRR_OI-GLOB-v02.0-fv02.0.nc",
   		"id": "9793ad5b-5d06-3163-95b2-b5f020280044[[0, 90, 50]]",
   		"source": "AVHRR_OI_L4_GHRSST_NCEI",
   		"primary": [{
   			"variable_name": "analysed_sst",
   			"cf_variable_name": "sea_surface_temperature",
   			"variable_value": 23.899993896484375,
   			"variable_unit": null
   		}],
   		"matches": [{
   			"platform": "30",
   			"device": null,
   			"lon": "-117.2368",
   			"lat": "32.7076",
   			"point": "Point(-117.237 32.708)",
   			"time": 1533081240,
   			"depth": -99999,
   			"fileurl": null,
   			"id": "WSQ2674_20180731v20001_1434-99999.0@-99999.0",
   			"source": "SAMOS",
   			"secondary": [{
   				"variable_name": "relative_humidity",
   				"cf_variable_name": "relative_humidity",
   				"variable_value": 71,
   				"variable_unit": null
   			}, {
   				"variable_name": "relative_humidity_quality",
   				"cf_variable_name": "relative_humidity_quality",
   				"variable_value": 1,
   				"variable_unit": null
   			}, {
   				"variable_name": "sea_water_temperature",
   				"cf_variable_name": "sea_water_temperature",
   				"variable_value": 23.2,
   				"variable_unit": null
   			}, {
   				"variable_name": "sea_water_temperature_quality",
   				"cf_variable_name": "sea_water_temperature_quality",
   				"variable_value": 1,
   				"variable_unit": null
   			}]
   		}]
   	}, {
   		"platform": "orbiting satellite",
   		"device": "radiometers",
   		"lon": "-117.125",
   		"lat": "32.625",
   		"point": "Point(-117.125 32.625)",
   		"time": 1533081600,
   		"depth": null,
   		"fileurl": "20180801120000-NCEI-L4_GHRSST-SSTblend-AVHRR_OI-GLOB-v02.0-fv02.0.nc",
   		"id": "9793ad5b-5d06-3163-95b2-b5f020280044[[0, 90, 51]]",
   		"source": "AVHRR_OI_L4_GHRSST_NCEI",
   		"primary": [{
   			"variable_name": "analysed_sst",
   			"cf_variable_name": "sea_surface_temperature",
   			"variable_value": 24.410003662109375,
   			"variable_unit": null
   		}],
   		"matches": [{
   			"platform": "30",
   			"device": null,
   			"lon": "-117.2367",
   			"lat": "32.7076",
   			"point": "Point(-117.237 32.708)",
   			"time": 1533077340,
   			"depth": -99999,
   			"fileurl": null,
   			"id": "WSQ2674_20180731v20001_1369-99999.0@-99999.0",
   			"source": "SAMOS",
   			"secondary": [{
   				"variable_name": "relative_humidity",
   				"cf_variable_name": "relative_humidity",
   				"variable_value": 72.8,
   				"variable_unit": null
   			}, {
   				"variable_name": "relative_humidity_quality",
   				"cf_variable_name": "relative_humidity_quality",
   				"variable_value": 1,
   				"variable_unit": null
   			}, {
   				"variable_name": "sea_water_temperature",
   				"cf_variable_name": "sea_water_temperature",
   				"variable_value": 23.2,
   				"variable_unit": null
   			}, {
   				"variable_name": "sea_water_temperature_quality",
   				"cf_variable_name": "sea_water_temperature_quality",
   				"variable_value": 1,
   				"variable_unit": null
   			}]
   		}]
   	}],
   	"params": {
   		"primary": "AVHRR_OI_L4_GHRSST_NCEI",
   		"matchup": "SAMOS",
   		"startTime": 1533081600,
   		"endTime": 1533167999,
   		"bbox": "-122, 26, -114, 34",
   		"timeTolerance": 43200,
   		"radiusTolerance": 18750,
   		"platforms": "30",
   		"parameter": null,
   		"depthMin": -1,
   		"depthMax": 1
   	},
   	"bounds": {},
   	"count": 2,
   	"details": {
   		"timeToComplete": 9,
   		"numSecondaryMatched": 2,
   		"numPrimaryMatched": 2,
   		"numUniqueSecondaries": 2,
   		"pageNum": 1,
   		"pageSize": 1000
   	}
   }
   ```
   
   Have not yet tested NCAR insitu endpoint. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@sdap.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org