You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@eagle.apache.org by "G.C. Chen" <ch...@gmail.com> on 2015/11/10 12:28:44 UTC
Eagle Storm execution get 403 error
Hello everyone,
Glad I am the second to post a thread in this mailist:)
I am very interested in using Eagle in our production Hadoop cluster.
However, I met a bug which is produced when running storm topology of
hdfs_audit_log: many storm blots reports 403 error when invoking Eagle
restful api, could you please guide me how to debug this issue please? I
also attached my storm configuration in the end of this mail. Thanks a lot
for your help!
2015-11-10T18:59:57.546+0800 o.q.i.StdSchedulerFactory [INFO] Quartz
scheduler 'DefaultQuartzScheduler' initialized from default resource file
in Quartz package: 'quartz.properties'
2015-11-10T18:59:57.546+0800 o.q.i.StdSchedulerFactory [INFO] Quartz
scheduler version: 2.2.1
2015-11-10T18:59:57.546+0800 o.q.c.QuartzScheduler [INFO] Scheduler
DefaultQuartzScheduler_$_NON_CLUSTERED started.
2015-11-10T18:59:57.553+0800 b.s.d.executor [INFO] Prepared bolt
JavaStormExecutorForAlertWrapper_5:(6)
2015-11-10T18:59:57.553+0800 b.s.d.executor [INFO] Prepared bolt
FileSensitivityDataJoinExecutor_2:(5)
2015-11-10T18:59:57.563+0800 e.s.a.t.FileSensitivityPollingJob [INFO] Load
file sensitivity information from eagle service
gateway.hdp.010170250047.ali.bj:9099
2015-11-10T18:59:57.565+0800 e.s.a.t.IPZonePollingJob [INFO] Load ip zone
information from eagle service gateway.hdp.010170250047.ali.bj:9099
2015-11-10T18:59:57.590+0800 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client environment:
host.name=gateway.hdp.010170250047.ali.bj
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.version=1.7.0_85
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.vendor=Oracle Corporation
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.home=/oneapm/local/jdk1.7.0_85/jre
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.class.path=/oneapm/local/storm-0.9.5/lib/reflectasm-1.07-shaded.jar:/oneapm/local/storm-0.9.5/lib/carbonite-1.4.0.jar:/oneapm/local/storm-0.9.5/lib/ring-core-1.1.5.jar:/oneapm/local/storm-0.9.5/lib/tools.cli-0.2.4.jar:/oneapm/local/storm-0.9.5/lib/minlog-1.2.jar:/oneapm/local/storm-0.9.5/lib/core.incubator-0.1.0.jar:/oneapm/local/storm-0.9.5/lib/json-simple-1.1.jar:/oneapm/local/storm-0.9.5/lib/ring-devel-0.3.11.jar:/oneapm/local/storm-0.9.5/lib/asm-4.0.jar:/oneapm/local/storm-0.9.5/lib/jline-2.11.jar:/oneapm/local/storm-0.9.5/lib/joda-time-2.0.jar:/oneapm/local/storm-0.9.5/lib/commons-lang-2.5.jar:/oneapm/local/storm-0.9.5/lib/kryo-2.21.jar:/oneapm/local/storm-0.9.5/lib/commons-io-2.4.jar:/oneapm/local/storm-0.9.5/lib/commons-logging-1.1.3.jar:/oneapm/local/storm-0.9.5/lib/commons-fileupload-1.2.1.jar:/oneapm/local/storm-0.9.5/lib/logback-classic-1.0.13.jar:/oneapm/local/storm-0.9.5/lib/servlet-api-2.5.jar:/oneapm/local/storm-0.9.5/lib/log4j-over-slf4j-1.6.6.jar:/oneapm/local/storm-0.9.5/lib/objenesis-1.2.jar:/oneapm/local/storm-0.9.5/lib/math.numeric-tower-0.0.1.jar:/oneapm/local/storm-0.9.5/lib/compojure-1.1.3.jar:/oneapm/local/storm-0.9.5/lib/hiccup-0.3.6.jar:/oneapm/local/storm-0.9.5/lib/disruptor-2.10.1.jar:/oneapm/local/storm-0.9.5/lib/jgrapht-core-0.9.0.jar:/oneapm/local/storm-0.9.5/lib/tools.macro-0.1.0.jar:/oneapm/local/storm-0.9.5/lib/snakeyaml-1.11.jar:/oneapm/local/storm-0.9.5/lib/slf4j-api-1.7.5.jar:/oneapm/local/storm-0.9.5/lib/clj-time-0.4.1.jar:/oneapm/local/storm-0.9.5/lib/ring-servlet-0.3.11.jar:/oneapm/local/storm-0.9.5/lib/ring-jetty-adapter-0.3.11.jar:/oneapm/local/storm-0.9.5/lib/storm-core-0.9.5.jar:/oneapm/local/storm-0.9.5/lib/clojure-1.5.1.jar:/oneapm/local/storm-0.9.5/lib/clj-stacktrace-0.2.2.jar:/oneapm/local/storm-0.9.5/lib/commons-codec-1.6.jar:/oneapm/local/storm-0.9.5/lib/tools.logging-0.2.3.jar:/oneapm/local/storm-0.9.5/lib/clout-1.0.1.jar:/oneapm/local/storm-0.9.5/lib/jetty-util-6.1.26.jar:/oneapm/local/storm-0.9.5/lib/chill-java-0.3.5.jar:/oneapm/local/storm-0.9.5/lib/logback-core-1.0.13.jar:/oneapm/local/storm-0.9.5/lib/jetty-6.1.26.jar:/oneapm/local/storm-0.9.5/lib/commons-exec-1.1.jar:/oneapm/local/storm-0.9.5/conf:/oneapm/data/storm/supervisor/stormdist/hdp-hdfsAuditLog-topology-7-1447153112/stormjar.jar
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.library.path=/oneapm/data/storm/supervisor/stormdist/hdp-hdfsAuditLog-topology-7-1447153112/resources/Linux-amd64:/oneapm/data/storm/supervisor/stormdist/hdp-hdfsAuditLog-topology-7-1447153112/resources:/usr/local/lib:/opt/local/lib:/usr/lib
2015-11-10T18:59:57.600+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.io.tmpdir=/tmp
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client
environment:java.compiler=<NA>
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client environment:
os.name=Linux
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client
environment:os.arch=amd64
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client
environment:os.version=3.10.0-123.9.3.el7.x86_64
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client environment:
user.name=hadoop
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client
environment:user.home=/home/hadoop
2015-11-10T18:59:57.601+0800 o.a.z.ZooKeeper [INFO] Client
environment:user.dir=/oneapm/local/storm-0.9.5
2015-11-10T18:59:57.602+0800 o.a.z.ZooKeeper [INFO] Initiating client
connection, connectString=10.172.171.229:2181,10.171.84.196:2181,
10.170.197.109:2181,10.172.177.98:2181,10.171.22.201:2181,
sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@7c2095b4
2015-11-10T18:59:57.619+0800 o.a.z.ClientCnxn [INFO] Opening socket
connection to server 10.172.171.229/10.172.171.229:2181. Will not attempt
to authenticate using SASL (unknown error)
2015-11-10T18:59:57.619+0800 o.a.z.ClientCnxn [INFO] Socket connection
established to 10.172.171.229/10.172.171.229:2181, initiating session
2015-11-10T18:59:57.626+0800 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-11-10T18:59:57.626+0800 o.a.z.ClientCnxn [INFO] Session establishment
complete on server 10.172.171.229/10.172.171.229:2181, sessionid =
0x14f22830eb29ed6, negotiated timeout = 20000
2015-11-10T18:59:57.626+0800 o.a.z.ZooKeeper [INFO] Initiating client
connection, connectString=10.172.171.229:2181/mix sessionTimeout=20000
watcher=org.apache.curator.ConnectionState@61ae975a
2015-11-10T18:59:57.627+0800 o.a.z.ClientCnxn [INFO] Opening socket
connection to server 10.172.171.229/10.172.171.229:2181. Will not attempt
to authenticate using SASL (unknown error)
2015-11-10T18:59:57.628+0800 o.a.z.ClientCnxn [INFO] Socket connection
established to 10.172.171.229/10.172.171.229:2181, initiating session
2015-11-10T18:59:57.633+0800 o.a.c.f.s.ConnectionStateManager [INFO] State
change: CONNECTED
2015-11-10T18:59:57.635+0800 o.a.z.ClientCnxn [INFO] Session establishment
complete on server 10.172.171.229/10.172.171.229:2181, sessionid =
0x14f22830eb29ed7, negotiated timeout = 20000
2015-11-10T18:59:57.635+0800 o.a.c.f.s.ConnectionStateManager [INFO] State
change: CONNECTED
2015-11-10T18:59:57.669+0800 s.k.DynamicBrokersReader [INFO] Read partition
info from zookeeper: GlobalPartitionInformation{partitionMap={0=
10.251.213.182:9092}}
2015-11-10T18:59:57.671+0800 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-11-10T18:59:57.671+0800 o.a.z.ZooKeeper [INFO] Initiating client
connection, connectString=10.172.171.229:2181/mix sessionTimeout=20000
watcher=org.apache.curator.ConnectionState@900fb9a
2015-11-10T18:59:57.673+0800 o.a.z.ClientCnxn [INFO] Opening socket
connection to server 10.172.171.229/10.172.171.229:2181. Will not attempt
to authenticate using SASL (unknown error)
2015-11-10T18:59:57.673+0800 o.a.z.ClientCnxn [INFO] Socket connection
established to 10.172.171.229/10.172.171.229:2181, initiating session
2015-11-10T18:59:57.674+0800 b.s.d.executor [INFO] Opened spout
kafkaMsgConsumer:(9)
2015-11-10T18:59:57.676+0800 b.s.d.executor [INFO] Activating spout
kafkaMsgConsumer:(9)
2015-11-10T18:59:57.677+0800 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing
partition manager connections
2015-11-10T18:59:57.681+0800 o.a.z.ClientCnxn [INFO] Session establishment
complete on server 10.172.171.229/10.172.171.229:2181, sessionid =
0x14f22830eb29ed8, negotiated timeout = 20000
2015-11-10T18:59:57.681+0800 o.a.c.f.s.ConnectionStateManager [INFO] State
change: CONNECTED
2015-11-10T18:59:57.684+0800 s.k.DynamicBrokersReader [INFO] Read partition
info from zookeeper: GlobalPartitionInformation{partitionMap={0=
10.251.213.182:9092}}
2015-11-10T18:59:57.686+0800 s.k.KafkaUtils [INFO] Task [1/1] assigned
[Partition{host=10.251.213.182:9092, partition=0}]
2015-11-10T18:59:57.686+0800 s.k.ZkCoordinator [INFO] Task [1/1] Deleted
partition managers: []
2015-11-10T18:59:57.686+0800 s.k.ZkCoordinator [INFO] Task [1/1] New
partition managers: [Partition{host=10.251.213.182:9092, partition=0}]
2015-11-10T18:59:57.855+0800 e.s.c.i.EagleServiceClientImpl [INFO] Going to
query service:
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=IPZoneService%5B%5D%7B*%7D&pageSize=2147483647&treeAgg=false
2015-11-10T18:59:57.855+0800 e.s.c.i.EagleServiceClientImpl [INFO] Going to
query service:
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=false
2015-11-10T18:59:57.855+0800 e.s.c.i.EagleServiceClientImpl [INFO] Going to
query service:
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=FileSensitivityService%5B%5D%7B*%7D&pageSize=2147483647&treeAgg=false
2015-11-10T18:59:57.855+0800 e.s.c.i.EagleServiceClientImpl [INFO] Going to
query service:
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=false
2015-11-10T18:59:57.855+0800 e.s.c.i.EagleServiceClientImpl [INFO] Going to
query service:
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=false
2015-11-10T18:59:58.001+0800 s.k.PartitionManager [INFO] Read partition
information from:
/eagle_hdfs/hdp_hdfs_audit_log/eagle.hdfsaudit.consumer/partition_0 -->
null
2015-11-10T18:59:58.047+0800 e.s.a.t.FileSensitivityPollingJob [ERROR] Fail
loading file sensitivity data
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=FileSensitivityService%5B%5D%7B*%7D&pageSize=2147483647&treeAgg=false
returned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.security.auditlog.timer.FileSensitivityPollingJob.load(FileSensitivityPollingJob.java:73)
~[stormjar.jar:na]
at
eagle.security.auditlog.timer.FileSensitivityPollingJob.execute(FileSensitivityPollingJob.java:46)
~[stormjar.jar:na]
at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [stormjar.jar:na]
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
[stormjar.jar:na]
2015-11-10T18:59:58.048+0800 e.a.d.AlertDefinitionDAOImpl [ERROR] Got an
exception when query alert Def service
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=false
returned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:70)
[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefsGroupbyAlertExecutorId(AlertDefinitionDAOImpl.java:93)
[stormjar.jar:na]
at
eagle.alert.dedup.AlertDeduplicationExecutorBase.init(AlertDeduplicationExecutorBase.java:91)
[stormjar.jar:na]
at
eagle.datastream.JavaStormBoltWrapper.prepare(JavaStormBoltWrapper.scala:35)
[stormjar.jar:na]
at
backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461)
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
2015-11-10T18:59:58.048+0800 e.a.d.AlertDefinitionDAOImpl [ERROR] Got an
exception when query alert Def service
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=false
returned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:70)
[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefsGroupbyAlertExecutorId(AlertDefinitionDAOImpl.java:93)
[stormjar.jar:na]
at
eagle.alert.dedup.AlertDeduplicationExecutorBase.init(AlertDeduplicationExecutorBase.java:91)
[stormjar.jar:na]
at
eagle.datastream.JavaStormBoltWrapper.prepare(JavaStormBoltWrapper.scala:35)
[stormjar.jar:na]
at
backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461)
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
2015-11-10T18:59:58.048+0800 e.a.d.AlertDefinitionDAOImpl [ERROR] Got an
exception when query alert Def service
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=false
returned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:70)
[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefsGroupbyAlertExecutorId(AlertDefinitionDAOImpl.java:93)
[stormjar.jar:na]
at
eagle.alert.notification.AlertNotificationExecutor.init(AlertNotificationExecutor.java:112)
[stormjar.jar:na]
at
eagle.datastream.JavaStormBoltWrapper.prepare(JavaStormBoltWrapper.scala:35)
[stormjar.jar:na]
at
backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461)
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
2015-11-10T18:59:58.047+0800 e.s.a.t.IPZonePollingJob [ERROR] Fail loading
ip zones data
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=IPZoneService%5B%5D%7B*%7D&pageSize=2147483647&treeAgg=false
returned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.security.auditlog.timer.IPZonePollingJob.load(IPZonePollingJob.java:74)
~[stormjar.jar:na]
at
eagle.security.auditlog.timer.IPZonePollingJob.execute(IPZonePollingJob.java:47)
~[stormjar.jar:na]
at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [stormjar.jar:na]
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
[stormjar.jar:na]
2015-11-10T18:59:58.049+0800 e.a.d.AlertDeduplicationExecutorBase [ERROR]
fail to initialize initialAlertDefs:
java.lang.IllegalStateException:
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=falsereturned
a response status of 403 Forbidden
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:87)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefsGroupbyAlertExecutorId(AlertDefinitionDAOImpl.java:93)
~[stormjar.jar:na]
at
eagle.alert.dedup.AlertDeduplicationExecutorBase.init(AlertDeduplicationExecutorBase.java:91)
~[stormjar.jar:na]
at
eagle.datastream.JavaStormBoltWrapper.prepare(JavaStormBoltWrapper.scala:35)
[stormjar.jar:na]
at
backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461)
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
Caused by: com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=falsereturned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:70)
~[stormjar.jar:na]
... 7 common frames omitted
2015-11-10T18:59:58.051+0800 e.a.n.AlertNotificationExecutor [ERROR] fail
to initialize initialAlertDefs:
java.lang.IllegalStateException:
com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=falsereturned
a response status of 403 Forbidden
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:87)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefsGroupbyAlertExecutorId(AlertDefinitionDAOImpl.java:93)
~[stormjar.jar:na]
at
eagle.alert.notification.AlertNotificationExecutor.init(AlertNotificationExecutor.java:112)
~[stormjar.jar:na]
at
eagle.datastream.JavaStormBoltWrapper.prepare(JavaStormBoltWrapper.scala:35)
[stormjar.jar:na]
at
backtype.storm.daemon.executor$fn__6647$fn__6659.invoke(executor.clj:692)
[storm-core-0.9.5.jar:0.9.5]
at backtype.storm.util$async_loop$fn__459.invoke(util.clj:461)
[storm-core-0.9.5.jar:0.9.5]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
Caused by: com.sun.jersey.api.client.UniformInterfaceException: GET
http://gateway.hdp.010170250047.ali.bj:9099/eagle-service/rest/entities?query=AlertDefinitionService%5B%40site%3D%22hdp%22+AND+%40dataSource%3D%22hdfsAuditLog%22%5D%7B*%7D&pageSize=2147483647&treeAgg=falsereturned
a response status of 403 Forbidden
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:676)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
~[stormjar.jar:na]
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
~[stormjar.jar:na]
at
eagle.service.client.impl.EagleServiceClientImpl.search(EagleServiceClientImpl.java:192)
~[stormjar.jar:na]
at
eagle.service.client.impl.SearchRequestBuilder.send(SearchRequestBuilder.java:95)
~[stormjar.jar:na]
at
eagle.alert.dao.AlertDefinitionDAOImpl.findActiveAlertDefs(AlertDefinitionDAOImpl.java:70)
~[stormjar.jar:na]
... 7 common frames omitted
Below is the storm job conf I used:
[hadoop@gateway eagle-0.1.0]$ cat conf/hdfsAuditLog-application.conf
{
"envContextConfig" : {
"env" : "storm",
"mode" : "cluster",
"topologyName" : "hdp-hdfsAuditLog-topology",
"stormConfigFile" : "security-auditlog-storm.yaml",
"parallelismConfig" : {
"kafkaMsgConsumer" : 1,
"hdfsAuditLogAlertExecutor*" : 1
}
},
"dataSourceConfig": {
"topic" : "hdp_hdfs_audit_log",
"zkConnection" : "10.172.171.229:2181/mix",
"zkConnectionTimeoutMS" : 15000,
"fetchSize" : 1048586,
"deserializerClass" :
"eagle.security.auditlog.HdfsAuditLogKafkaDeserializer",
"transactionZKServers" :
"10.172.171.229,10.171.84.196,10.170.197.109,10.172.177.98,10.171.22.201",
"transactionZKPort" : 2181,
"transactionZKRoot" : "/eagle_hdfs",
"consumerGroupId" : "eagle.hdfsaudit.consumer",
"transactionStateUpdateMS" : 2000
},
"alertExecutorConfigs" : {
"hdfsAuditLogAlertExecutor" : {
"parallelism" : 1,
"partitioner" : "eagle.alert.policy.DefaultPolicyPartitioner",
"needValidation" : "true"
}
},
"eagleProps" : {
"site" : "hdp",
"dataSource": "hdfsAuditLog",
"dataJoinPollIntervalSec" : 30,
"mailHost" : "mailHost.com",
"mailSmtpPort":"25",
"mailDebug" : "true",
"eagleService": {
"host": "gateway.hdp.010170250047.ali.bj",
"port": 9099
"username": "admin",
"password": "admin"
}
},
"dynamicConfigSource" : {
"enabled" : true,
"initDelayMillis" : 0,
"delayMillis" : 30000
}
}
Best Regards,
- G.C.
--
G.C. Guancheng Chen
Principal Data Engineer at OneAPM.com
LinkedIn: www.linkedin.com/in/guanchengchen
Re: Eagle Storm execution get 403 error
Posted by Libin Sun <ab...@gmail.com>.
Hi Guan cheng,
Good to know it works, we have the two copies because the one in
eagleSecurity.xml is for sever side, and the one in application.conf is for
client side
But thanks for your feedback, we will add noteinto our document that there
are two config for username/password in service and client side
respectively
Thanks
Libin
2015-11-11 0:24 GMT+08:00 Guan-Cheng Chen <ch...@gmail.com>:
> Hi Libin,
>
> Cool, after change password back to “secret”, it just works!
>
> BTW, it should be better for user to config their own password. What do
> you think pls? Thanks!
>
> Best Regards,
>
> G.C.
> Director and Principle Data Engineer
> OneAPM.com
> http://www.linkedin.com/in/guanchengchen
>
>
>
>
>
> On Nov 10, 2015, at 20:00, Libin Sun <ab...@gmail.com> wrote:
>
> This is because the user/password is configured as admin/secret in eagleSecurity.xml at eagle service side
>
> <scr:user name="admin" password="$2a$10$TwALMRHpSetDaeTurg9rj.DnIdOde4fkQGBSPG3fVqtH.G5ZH8sQK" authorities="ROLE_ADMIN, ROLE_USER" />
>
> ($2a$10$TwALMRHpSetDaeTurg9rj.DnIdOde4fkQGBSPG3fVqtH.G5ZH8sQK is base64 encoding for "admin:secret")
>
> You can change the password to "secret", it should works
>
> Best Regards,
> Libin
>
>
>
Re: Eagle Storm execution get 403 error
Posted by Guan-Cheng Chen <ch...@gmail.com>.
Hi Libin,
Cool, after change password back to “secret”, it just works!
BTW, it should be better for user to config their own password. What do you think pls? Thanks!
Best Regards,
G.C.
Director and Principle Data Engineer
OneAPM.com
http://www.linkedin.com/in/guanchengchen <http://www.linkedin.com/in/guanchengchen>
> On Nov 10, 2015, at 20:00, Libin Sun <ab...@gmail.com> wrote:
>
> This is because the user/password is configured as admin/secret in eagleSecurity.xml at eagle service side
>
> <scr:user name="admin" password="$2a$10$TwALMRHpSetDaeTurg9rj.DnIdOde4fkQGBSPG3fVqtH.G5ZH8sQK" authorities="ROLE_ADMIN, ROLE_USER" />
>
> ($2a$10$TwALMRHpSetDaeTurg9rj.DnIdOde4fkQGBSPG3fVqtH.G5ZH8sQK is base64 encoding for "admin:secret")
>
> You can change the password to "secret", it should works
>
> Best Regards,
> Libin
Re: Eagle Storm execution get 403 error
Posted by Libin Sun <ab...@gmail.com>.
This is because the user/password is configured as admin/secret in
eagleSecurity.xml at eagle service side
<scr:user name="admin"
password="$2a$10$TwALMRHpSetDaeTurg9rj.DnIdOde4fkQGBSPG3fVqtH.G5ZH8sQK"
authorities="ROLE_ADMIN, ROLE_USER" />
($2a$10$TwALMRHpSetDaeTurg9rj.DnIdOde4fkQGBSPG3fVqtH.G5ZH8sQK is
base64 encoding for "admin:secret")
You can change the password to "secret", it should works
Best Regards,
Libin