You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@eagle.apache.org by "Hao Chen (JIRA)" <ji...@apache.org> on 2016/11/11 06:09:58 UTC
[jira] [Resolved] (EAGLE-635) Refactor policy parser and validator
for richer plan details and better performance
[ https://issues.apache.org/jira/browse/EAGLE-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hao Chen resolved EAGLE-635.
----------------------------
Resolution: Fixed
> Refactor policy parser and validator for richer plan details and better performance
> -----------------------------------------------------------------------------------
>
> Key: EAGLE-635
> URL: https://issues.apache.org/jira/browse/EAGLE-635
> Project: Eagle
> Issue Type: Improvement
> Affects Versions: v0.5.0
> Reporter: Hao Chen
> Assignee: Hao Chen
> Fix For: v0.5.0
>
>
> h2. Changes
> * Refactor policy parser and validator for richer plan details and better performance
> * Decouple PolicyExecutionPlan and PolicyValidation
> h2. API
> * Parse API
> {code}
> POST /metadata/policies/parse
> Accept-Type: text
> from HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX#window.timeBatch(2 min) select cmd, user, count() as total_count group by cmd,user insert into HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX_OUT"
> {code}
> * Validation API
> {code}
> POST /metadata/policies/validate
> Accept-Type: application/json
> {
> "name": "hdfsPolicy",
> "description": "hdfsPolicy",
> "inputStreams": [
> "HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX"
> ],
> "outputStreams": [
> "HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX_OUT"
> ],
> "definition": {
> "type": "siddhi",
> "value": "from HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX#window.timeBatch(2 min) select cmd, user, count() as total_count group by cmd,user insert into HDFS_AUDIT_LOG_ENRICHED_STREAM_SANDBOX_OUT "
> },
> "partitionSpec": [
> {
> "streamId": "hdfs_audit_log_enriched_stream",
> "type": "GROUPBY",
> "columns" : [
> "cmd"
> ]
> }
> ],
> "parallelismHint": 2
> }
> {code}
> h2. Use Cases
> * *parse*: could continuously call `parse` API aside during user keeps typing to verify the syntax and automatically generate input/output/partition , as it won't call back-end db, so will be very fast.
> * *validate*: when user finishes defining policy, the api will validate the metadata end2end
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)