You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by charlesporter <gi...@git.apache.org> on 2015/12/14 06:11:59 UTC
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
GitHub user charlesporter opened a pull request:
https://github.com/apache/incubator-metron/pull/2
replace opensoc-streaming version 0.4BETA with 0.6BETA 8e7a6b4ad9febbc…
…4ea47ba7810c42cc94d4dee37
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/charlesporter/incubator-metron master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/2.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2
----
commit a919cc191a207373c7aa854dfe4949f9f0fe9daa
Author: charlesporter <pu...@gmail.com>
Date: 2015-12-14T05:04:53Z
replace opensoc-steaming version 0.4BETA with 0.6BETA 8e7a6b4ad9febbc4ea47ba7810c42cc94d4dee37
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by omalley <gi...@git.apache.org>.
Github user omalley commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-164870709
Yeah, you need a commit that includes "closes apache/#2" in the commit message.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47837646
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -185,10 +184,10 @@ public void execute(Tuple tuple) {
JSONArray uuid_list = new JSONArray();
if (alerts_list == null || alerts_list.isEmpty()) {
- LOG.trace("[OpenSOC] No alerts detected in: "
+ System.out.println("[OpenSOC] No alerts detected in: "
--- End diff --
I think it's ok to have system.outs in the runners that launch the topology, but it shouldn't exist in the bolts themselves. you are correct. we should change it back to log.trace
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by ptgoetz <gi...@git.apache.org>.
Github user ptgoetz commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-164869295
@james-sirota The github mirrors of apache repos are read-only. You can comment, etc. but you can't use the "merge" button. To merge you have to pull from the repo of the person that opened the request, then push the result to the official apache repo.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by omalley <gi...@git.apache.org>.
Github user omalley commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-165274029
Yeah, System.exit is exactly right when a thread has failed and can't recover. You want to bring down the whole process and not leave the process partially running.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47837339
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/adapters/AllAlertAdapter.java ---
@@ -23,64 +21,96 @@
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
import org.json.simple.JSONObject;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
+import org.apache.log4j.Logger;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.opensoc.alerts.interfaces.AlertsAdapter;
+@SuppressWarnings("serial")
public class AllAlertAdapter implements AlertsAdapter, Serializable {
+ HTableInterface blacklist_table;
HTableInterface whitelist_table;
InetAddressValidator ipvalidator = new InetAddressValidator();
String _whitelist_table_name;
- // String _blacklist_table_name;
+ String _blacklist_table_name;
String _quorum;
String _port;
String _topologyname;
Configuration conf = null;
- protected Cache<String, String> cache;
-
- Map<String, String> id_list = new HashMap<String, String>();
+ Cache<String, String> cache;
+ String _topology_name;
Set<String> loaded_whitelist = new HashSet<String>();
Set<String> loaded_blacklist = new HashSet<String>();
- String _topology_name;
-
- protected static final Logger LOG = LoggerFactory
+ protected static final Logger LOG = Logger
.getLogger(AllAlertAdapter.class);
- public AllAlertAdapter(String whitelist_table_name,
- String blacklist_table_name, String quorum, String port,
- int _MAX_TIME_RETAIN, int _MAX_CACHE_SIZE) {
-
- _whitelist_table_name = whitelist_table_name;
-
- _quorum = quorum;
- _port = port;
-
- cache = CacheBuilder.newBuilder().maximumSize(_MAX_CACHE_SIZE)
- .expireAfterWrite(_MAX_TIME_RETAIN, TimeUnit.MINUTES).build();
+ public AllAlertAdapter(Map<String, String> config) {
+ try {
+ if(!config.containsKey("whitelist_table_name"))
+ throw new Exception("Whitelist table name is missing");
+
+ _whitelist_table_name = config.get("whitelist_table_name");
+
+ if(!config.containsKey("blacklist_table_name"))
+ throw new Exception("Blacklist table name is missing");
+
+ _blacklist_table_name = config.get("blacklist_table_name");
+
+ if(!config.containsKey("quorum"))
+ throw new Exception("Quorum name is missing");
+
+ _quorum = config.get("quorum");
+
+ if(!config.containsKey("port"))
+ throw new Exception("port name is missing");
+
+ _port = config.get("port");
+ if(!config.containsKey("_MAX_CACHE_SIZE_OBJECTS_NUM"))
+ throw new Exception("_MAX_CACHE_SIZE_OBJECTS_NUM name is missing");
+
+ int _MAX_CACHE_SIZE_OBJECTS_NUM = Integer.parseInt(config
+ .get("_MAX_CACHE_SIZE_OBJECTS_NUM"));
+
+ if(!config.containsKey("_MAX_TIME_RETAIN_MINUTES"))
+ throw new Exception("_MAX_TIME_RETAIN_MINUTES name is missing");
+
+ int _MAX_TIME_RETAIN_MINUTES = Integer.parseInt(config
+ .get("_MAX_TIME_RETAIN_MINUTES"));
+ cache = CacheBuilder.newBuilder().maximumSize(_MAX_CACHE_SIZE_OBJECTS_NUM)
+ .expireAfterWrite(_MAX_TIME_RETAIN_MINUTES, TimeUnit.MINUTES)
+ .build();
+ } catch (Exception e) {
+ System.out.println("Could not initialize Alerts Adapter");
+ e.printStackTrace();
+ System.exit(0);
--- End diff --
The reason this was done is because this bolt has dependencies on Hbase and Zookeeper. If something went wrong in the initialization the bolt didn't explicitly fail, but timed out instead. So the topology would half start and then started blasting packets into a bolt that wasn't fully initialized or had an error. Having it explicitly exit like this failed the whole topology right away so you immediately knew you had a problem and didn't have to go looking for it in the logs when the topology blew up 5 minutes later
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-metron pull request: replace opensoc-streaming
version 0...
Posted by charles porter <op...@gmail.com>.
This pull request is a little atypical:
The assertion of this pull request is only to bring Metron up-to-date with
the tip of the code that CISCO published.
https://github.com/OpenSOC/opensoc-streaming
There is no assertion that it is ideal code.
A as Owen suggested a few comments back, my hope is that a reviewer will
confirm that the pull request really does match the tip of
https://github.com/OpenSOC/opensoc-streaming,
Once we get it merged we can start fixing the bugs and beating into shape
as deployable code.
After stability, we can start adding features, which is what a lot of
people really want to do.
(Alternatively, someone could object to the premise that we want Metron to
baseline at the tip of Cisco code. Then we would have a discussion on
where to start the Metron project)
On Wed, Dec 16, 2015 at 2:59 PM, james-sirota <gi...@git.apache.org> wrote:
> Github user james-sirota commented on a diff in the pull request:
>
> https://github.com/apache/incubator-metron/pull/2#discussion_r47825924
>
> --- Diff:
> opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java
> ---
> @@ -185,10 +184,10 @@ public void execute(Tuple tuple) {
> JSONArray uuid_list = new JSONArray();
>
> if (alerts_list == null || alerts_list.isEmpty()) {
> - LOG.trace("[OpenSOC] No alerts detected
> in: "
> + System.out.println("[OpenSOC] No alerts
> detected in: "
> --- End diff --
>
> I think this is a piece of debugging that someone forgot to comment
> out. We can ignore this
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47825924
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -185,10 +184,10 @@ public void execute(Tuple tuple) {
JSONArray uuid_list = new JSONArray();
if (alerts_list == null || alerts_list.isEmpty()) {
- LOG.trace("[OpenSOC] No alerts detected in: "
+ System.out.println("[OpenSOC] No alerts detected in: "
--- End diff --
I think this is a piece of debugging that someone forgot to comment out. We can ignore this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by ptgoetz <gi...@git.apache.org>.
Github user ptgoetz commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47845155
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/adapters/AllAlertAdapter.java ---
@@ -23,64 +21,96 @@
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
import org.json.simple.JSONObject;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
+import org.apache.log4j.Logger;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.opensoc.alerts.interfaces.AlertsAdapter;
+@SuppressWarnings("serial")
public class AllAlertAdapter implements AlertsAdapter, Serializable {
+ HTableInterface blacklist_table;
HTableInterface whitelist_table;
InetAddressValidator ipvalidator = new InetAddressValidator();
String _whitelist_table_name;
- // String _blacklist_table_name;
+ String _blacklist_table_name;
String _quorum;
String _port;
String _topologyname;
Configuration conf = null;
- protected Cache<String, String> cache;
-
- Map<String, String> id_list = new HashMap<String, String>();
+ Cache<String, String> cache;
+ String _topology_name;
Set<String> loaded_whitelist = new HashSet<String>();
Set<String> loaded_blacklist = new HashSet<String>();
- String _topology_name;
-
- protected static final Logger LOG = LoggerFactory
+ protected static final Logger LOG = Logger
.getLogger(AllAlertAdapter.class);
- public AllAlertAdapter(String whitelist_table_name,
- String blacklist_table_name, String quorum, String port,
- int _MAX_TIME_RETAIN, int _MAX_CACHE_SIZE) {
-
- _whitelist_table_name = whitelist_table_name;
-
- _quorum = quorum;
- _port = port;
-
- cache = CacheBuilder.newBuilder().maximumSize(_MAX_CACHE_SIZE)
- .expireAfterWrite(_MAX_TIME_RETAIN, TimeUnit.MINUTES).build();
+ public AllAlertAdapter(Map<String, String> config) {
+ try {
+ if(!config.containsKey("whitelist_table_name"))
+ throw new Exception("Whitelist table name is missing");
+
+ _whitelist_table_name = config.get("whitelist_table_name");
+
+ if(!config.containsKey("blacklist_table_name"))
+ throw new Exception("Blacklist table name is missing");
+
+ _blacklist_table_name = config.get("blacklist_table_name");
+
+ if(!config.containsKey("quorum"))
+ throw new Exception("Quorum name is missing");
+
+ _quorum = config.get("quorum");
+
+ if(!config.containsKey("port"))
+ throw new Exception("port name is missing");
+
+ _port = config.get("port");
+ if(!config.containsKey("_MAX_CACHE_SIZE_OBJECTS_NUM"))
+ throw new Exception("_MAX_CACHE_SIZE_OBJECTS_NUM name is missing");
+
+ int _MAX_CACHE_SIZE_OBJECTS_NUM = Integer.parseInt(config
+ .get("_MAX_CACHE_SIZE_OBJECTS_NUM"));
+
+ if(!config.containsKey("_MAX_TIME_RETAIN_MINUTES"))
+ throw new Exception("_MAX_TIME_RETAIN_MINUTES name is missing");
+
+ int _MAX_TIME_RETAIN_MINUTES = Integer.parseInt(config
+ .get("_MAX_TIME_RETAIN_MINUTES"));
+ cache = CacheBuilder.newBuilder().maximumSize(_MAX_CACHE_SIZE_OBJECTS_NUM)
+ .expireAfterWrite(_MAX_TIME_RETAIN_MINUTES, TimeUnit.MINUTES)
+ .build();
+ } catch (Exception e) {
+ System.out.println("Could not initialize Alerts Adapter");
+ e.printStackTrace();
+ System.exit(0);
--- End diff --
An alternative approach that the idiomatic storm way of doing things, is to throw a RuntimeException, which will cause (currently) the worker JVM to exit.
This could be important in the future as storm will likely switch to a model where uncaught the RuntimeExceptions only cause the specific task to be restarted, not the entire worker JVM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-metron/pull/2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-164857934
I actually don't have permissions to approve this pull request. let me see if i can figure out how to add the PMC members to a group that has these privs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by dev-warlord <gi...@git.apache.org>.
Github user dev-warlord commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47831309
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -185,10 +184,10 @@ public void execute(Tuple tuple) {
JSONArray uuid_list = new JSONArray();
if (alerts_list == null || alerts_list.isEmpty()) {
- LOG.trace("[OpenSOC] No alerts detected in: "
+ System.out.println("[OpenSOC] No alerts detected in: "
--- End diff --
James, do you mean that this particular line will be reverted back to LOG.trace? We as a group should not have code in the master branch with System.out.*
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by dev-warlord <gi...@git.apache.org>.
Github user dev-warlord commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47831355
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -196,11 +195,11 @@ public void execute(Tuple tuple) {
LOG.trace("[OpenSOC] Checking alerts cache: " + alert);
if (cache.getIfPresent(alert) == null) {
- LOG.trace("[OpenSOC]: Alert not found in cache: " + alert);
+ System.out.println("[OpenSOC]: Alert not found in cache: " + alert);
--- End diff --
see above
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by omalley <gi...@git.apache.org>.
Github user omalley commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-164867594
Since you guys picked RTC, you should get a review of it to ensure that it matches the code from https://github.com/OpenSOC/opensoc-streaming at 8e7a6b4ad9febbc4ea47ba7810c42cc94d4dee37
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47832617
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -196,11 +195,11 @@ public void execute(Tuple tuple) {
LOG.trace("[OpenSOC] Checking alerts cache: " + alert);
if (cache.getIfPresent(alert) == null) {
- LOG.trace("[OpenSOC]: Alert not found in cache: " + alert);
+ System.out.println("[OpenSOC]: Alert not found in cache: " + alert);
--- End diff --
yeah it just needs to be removed as it serves no purpose. the log is already logging the same line. there is no reason to print it out
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by sirsean <gi...@git.apache.org>.
Github user sirsean commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47825454
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -185,10 +184,10 @@ public void execute(Tuple tuple) {
JSONArray uuid_list = new JSONArray();
if (alerts_list == null || alerts_list.isEmpty()) {
- LOG.trace("[OpenSOC] No alerts detected in: "
+ System.out.println("[OpenSOC] No alerts detected in: "
--- End diff --
Why would we replace `LOG.trace` with `System.out.println`? Isn't that exactly the opposite of what we want?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by sirsean <gi...@git.apache.org>.
Github user sirsean commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47835948
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/adapters/AllAlertAdapter.java ---
@@ -23,64 +21,96 @@
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
import org.json.simple.JSONObject;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
+import org.apache.log4j.Logger;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.opensoc.alerts.interfaces.AlertsAdapter;
+@SuppressWarnings("serial")
public class AllAlertAdapter implements AlertsAdapter, Serializable {
+ HTableInterface blacklist_table;
HTableInterface whitelist_table;
InetAddressValidator ipvalidator = new InetAddressValidator();
String _whitelist_table_name;
- // String _blacklist_table_name;
+ String _blacklist_table_name;
String _quorum;
String _port;
String _topologyname;
Configuration conf = null;
- protected Cache<String, String> cache;
-
- Map<String, String> id_list = new HashMap<String, String>();
+ Cache<String, String> cache;
+ String _topology_name;
Set<String> loaded_whitelist = new HashSet<String>();
Set<String> loaded_blacklist = new HashSet<String>();
- String _topology_name;
-
- protected static final Logger LOG = LoggerFactory
+ protected static final Logger LOG = Logger
.getLogger(AllAlertAdapter.class);
- public AllAlertAdapter(String whitelist_table_name,
- String blacklist_table_name, String quorum, String port,
- int _MAX_TIME_RETAIN, int _MAX_CACHE_SIZE) {
-
- _whitelist_table_name = whitelist_table_name;
-
- _quorum = quorum;
- _port = port;
-
- cache = CacheBuilder.newBuilder().maximumSize(_MAX_CACHE_SIZE)
- .expireAfterWrite(_MAX_TIME_RETAIN, TimeUnit.MINUTES).build();
+ public AllAlertAdapter(Map<String, String> config) {
+ try {
+ if(!config.containsKey("whitelist_table_name"))
+ throw new Exception("Whitelist table name is missing");
+
+ _whitelist_table_name = config.get("whitelist_table_name");
+
+ if(!config.containsKey("blacklist_table_name"))
+ throw new Exception("Blacklist table name is missing");
+
+ _blacklist_table_name = config.get("blacklist_table_name");
+
+ if(!config.containsKey("quorum"))
+ throw new Exception("Quorum name is missing");
+
+ _quorum = config.get("quorum");
+
+ if(!config.containsKey("port"))
+ throw new Exception("port name is missing");
+
+ _port = config.get("port");
+ if(!config.containsKey("_MAX_CACHE_SIZE_OBJECTS_NUM"))
+ throw new Exception("_MAX_CACHE_SIZE_OBJECTS_NUM name is missing");
+
+ int _MAX_CACHE_SIZE_OBJECTS_NUM = Integer.parseInt(config
+ .get("_MAX_CACHE_SIZE_OBJECTS_NUM"));
+
+ if(!config.containsKey("_MAX_TIME_RETAIN_MINUTES"))
+ throw new Exception("_MAX_TIME_RETAIN_MINUTES name is missing");
+
+ int _MAX_TIME_RETAIN_MINUTES = Integer.parseInt(config
+ .get("_MAX_TIME_RETAIN_MINUTES"));
+ cache = CacheBuilder.newBuilder().maximumSize(_MAX_CACHE_SIZE_OBJECTS_NUM)
+ .expireAfterWrite(_MAX_TIME_RETAIN_MINUTES, TimeUnit.MINUTES)
+ .build();
+ } catch (Exception e) {
+ System.out.println("Could not initialize Alerts Adapter");
+ e.printStackTrace();
+ System.exit(0);
--- End diff --
We should really avoid ever, ever calling `System.exit`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by sirsean <gi...@git.apache.org>.
Github user sirsean commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/2#discussion_r47833458
--- Diff: opensoc-streaming/OpenSOC-Alerts/src/main/java/com/opensoc/alerts/TelemetryAlertsBolt.java ---
@@ -196,11 +195,11 @@ public void execute(Tuple tuple) {
LOG.trace("[OpenSOC] Checking alerts cache: " + alert);
if (cache.getIfPresent(alert) == null) {
- LOG.trace("[OpenSOC]: Alert not found in cache: " + alert);
+ System.out.println("[OpenSOC]: Alert not found in cache: " + alert);
--- End diff --
Well the log isn't logging that same line any more -- the diff shows that the call to the logger was replaced by System.out, which we probably want to switch back.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by ptgoetz <gi...@git.apache.org>.
Github user ptgoetz commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-165505353
@james-sirota
Here's the process:
1. Add the official Apache repo as a remote in your local clone (one time step):
`git remote add apache https://git-wip-us.apache.org/repos/asf/incubator-metron.git`
2. checkout whatever branch you want to merge it into:
`git checkout master`
3. Merge the pull request:
`git pull git@github.com:charlesporter/incubator-metron.git master`
4. Push the results to the Apache repo:
`git push apache master`
A git hook will automatically close the github pull request when the repos get synced.
If there is a pull request that you want to close without merging, you can simply ask the person who opened it to close it. If that doesn't work, you can use a commit message to close it.
To do that, make a trivial change, like adding a space to README.md, and commit and push the change. In the commit message add the text "this closes #N" where N is the number of the pull request. When the repos sync, the pull request will get closed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by charlesporter <gi...@git.apache.org>.
Github user charlesporter commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-164855096
Can someone have a look this? I would like it if this could get merged very soon.
It simply contains the tip of opensoc-streaming from the cisco opensource
This is the point from which the hortonworks forks started.
Owen had indicated that we cannot easily start from the hortonworks repo .
He had created metron from a very old versio because that is what was hard-linked as sub-repos in the top-level opensoc repo.
We need to get this Pull Request ok's and merged before we can start on even the most basic improvements, like making it work with recent builds of the hadoop infrastructure .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request: replace opensoc-streaming version 0...
Posted by james-sirota <gi...@git.apache.org>.
Github user james-sirota commented on the pull request:
https://github.com/apache/incubator-metron/pull/2#issuecomment-165335811
hi can anyone walk me through how to commit this the first time i do it? then i'll publish a wiki so everyone can do it. i want to make sure i don't screw it up the first time
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---