You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by GitBox <gi...@apache.org> on 2019/10/01 21:53:51 UTC
[GitHub] [metron] mmiklavc commented on issue #1523: METRON-2232 Upgrade to
Hadoop 3.1.1
mmiklavc commented on issue #1523: METRON-2232 Upgrade to Hadoop 3.1.1
URL: https://github.com/apache/metron/pull/1523#issuecomment-537249252
A few brief notes:
- In order to get around https://issues.apache.org/jira/browse/HBASE-22394 in our integration tests, we had to copy source files into our test dirs. I still need to document this and make it clear what the issue is.
- I found a bug in the MR mode for the flatfile loader. Here's the stack trace:
```
2019-10-01 19:50:50,570 INFO [IPC Server handler 10 on 39246] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1569879390951_0002_m_000000_1000: Error: org.codehaus.jackson.map.JsonMappingException: org.apache.metron.stellar.dsl.Context$Capabilities (through reference chain: org.apache.metron.dataloads.extractor.ExtractorHandler["extractor"])
at org.codehaus.jackson.map.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:345)
at org.codehaus.jackson.map.deser.SettableBeanProperty._throwAsIOE(SettableBeanProperty.java:328)
at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.set(SettableBeanProperty.java:424)
at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
at org.apache.metron.dataloads.extractor.ExtractorHandler.load(ExtractorHandler.java:84)
at org.apache.metron.dataloads.extractor.ExtractorHandler.load(ExtractorHandler.java:93)
at org.apache.metron.dataloads.extractor.ExtractorHandler.load(ExtractorHandler.java:100)
at org.apache.metron.dataloads.hbase.mr.BulkLoadMapper.initialize(BulkLoadMapper.java:60)
at org.apache.metron.dataloads.hbase.mr.BulkLoadMapper.setup(BulkLoadMapper.java:45)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:799)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.lang.ClassNotFoundException: org.apache.metron.stellar.dsl.Context$Capabilities
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.metron.dataloads.extractor.Extractors.create(Extractors.java:51)
at org.apache.metron.dataloads.extractor.ExtractorHandler.setExtractor(ExtractorHandler.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.set(SettableBeanProperty.java:422)
... 18 more
```
I inspected the jar file for stellar-common included on the jar classpath and the class in question is there. Still investigating this. It's possible this is a regression in master as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services