[jira] [Commented] (METRON-2230) Update typosquatting use case to reflect new OBJECT_GET max file size constraint property in global config

["Michael Miklavcic (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913686#comment-16913686 ] 

Michael Miklavcic commented on METRON-2230:
-------------------------------------------

The following config option needs to be added to global.json in order for the [typsquatting|[https://github.com/apache/metron/tree/master/use-cases/typosquat_detection]] use case to work.

{code:java}
"object.cache.max.file.size" : 5000000
{code}


Failure to provide the larger maximum file size will result in the following error:

{code:java}
[Stellar]>>> BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle')
[!] Unable to parse: BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle') due to: File at path '/tmp/reference/alexa10k_filter.ser' is larger than the configured max file size of 1048576
org.apache.metron.stellar.dsl.ParseException: Unable to parse: BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle') due to: File at path '/tmp/reference/alexa10k_filter.ser' is larger than the configured max file size of 1048576
	at org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:166)
	at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154)
	at org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.executeStellar(DefaultStellarShellExecutor.java:407)
	at org.apache.metron.stellar.common.shell.DefaultStellarShellExecutor.execute(DefaultStellarShellExecutor.java:257)
	at org.apache.metron.stellar.common.shell.cli.StellarShell.execute(StellarShell.java:359)
	at org.jboss.aesh.console.AeshProcess.run(AeshProcess.java:53)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: File at path '/tmp/reference/alexa10k_filter.ser' is larger than the configured max file size of 1048576
	at org.apache.metron.enrichment.cache.ObjectCache$Loader.load(ObjectCache.java:72)
	at org.apache.metron.enrichment.cache.ObjectCache$Loader.load(ObjectCache.java:46)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache.lambda$new$0(BoundedLocalCache.java:3366)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2039)
	at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2037)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2020)
	at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
	at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:67)
	at org.apache.metron.enrichment.cache.ObjectCache.get(ObjectCache.java:82)
	at org.apache.metron.enrichment.stellar.ObjectGet.apply(ObjectGet.java:66)
	at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664)
	at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259)
	at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151)
	... 7 more
{code}

The default max file size is 1MB while the bloom filter in the use case is over 4MB.

{code:java}
# hdfs dfs -ls /tmp/reference/alexa10k_filter.ser
-rw-r--r--   1 root hdfs    4710254 2019-08-22 19:16 /tmp/reference/alexa10k_filter.ser
{code}


 

> Update typosquatting use case to reflect new OBJECT_GET max file size constraint property in global config
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: METRON-2230
>                 URL: https://issues.apache.org/jira/browse/METRON-2230
>             Project: Metron
>          Issue Type: Bug
>            Reporter: Michael Miklavcic
>            Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)