You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jani Arvonen <ja...@floobs.com> on 2007/08/21 14:59:22 UTC
newbie problem: Using DFS from Java program
Hello everybody!
I've been trying to use hadoop distributed file system from my java
spring web application but without any good results :). We have one
server where hadoop namenode are datanode are succesfully running (so
they all are running on a single node). I managed to configure it with
the help of documentation page found from hadoop website.
Now i need to make my web application to use this distributed file
system running on the other server. I've tried to place the same
hadoop-site.xml found from the hadoop master server in the programs
classpath and using DistributedFileSystem but without no luck. Always
when trying to use for example copyFromLocalFile it throws a
NullPointerException. Im getting really lost with this issue because i
havent found any examples or tutorials related to this :)
I would really appreciate if somebody could help me so that i can
connect to this hadoop master server from my java spring web application
and to use distributed file system.
best regards,
Jani
Re: newbie problem: Using DFS from Java program
Posted by Sagar Naik <sa...@visvo.com>.
Hey
I think its more of a classpath issue.
Can u pl check whether u r hadoop config (hadoop-site.xml , etc) files
are present in classpath when u start from webserver.
I dont think u wud have errors in hadoop log files, but its good to chk
Jani Arvonen wrote:
> Oh yes, of course. Our web application throws this exception when trying
> to use copyFromLocalFile() through DistributedFileSystem. The local file
> im trying to copy to the HDFS do exists...
>
> And we have also tried to use LocalFileSystem which works fine but that
> is not exactly what we are trying to achieve :)
>
> stacktrace:
>
> java.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:155)
> at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.copyFromLocalFile(ChecksumFileSystem.java:576)
> at
> com.floobs.service.media.DefaultMediaStorage.copyFromLocalFile(DefaultMediaStorage.java:23)
> at
> com.floobs.service.media.DefaultMediaStorage.copyFromLocalFile(DefaultMediaStorage.java:18)
> at
> com.floobs.web.media.MediaUploadFormController.storeFile(MediaUploadFormController.java:155)
> at com.floobs.web.media.MediaUploadFormController.access
> $500(MediaUploadFormController.java:36)
> at com.floobs.web.media.MediaUploadFormController
> $2.processFile(MediaUploadFormController.java:120)
> at com.floobs.web.media.MediaUploadFormController
> $UploadFileProcessor.execute(MediaUploadFormController.java:144)
> at
> com.floobs.web.media.MediaUploadFormController.processFileUploads(MediaUploadFormController.java:114)
> at
> com.floobs.web.media.MediaUploadFormController.onSubmit(MediaUploadFormController.java:65)
> at
> org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)
> at
> org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:250)
> at
> org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
> at
> org.riotfamily.cachius.spring.CacheableControllerHandlerAdapter.handle(CacheableControllerHandlerAdapter.java:118)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
> at
> org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
> at
> org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
> at
> org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
> at
> org.riotfamily.pages.component.impl.IncludeComponent.renderInternal(IncludeComponent.java:66)
> at
> org.riotfamily.pages.component.impl.AbstractComponent.render(AbstractComponent.java:98)
> at
> org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponent(AbstractRenderStrategy.java:243)
> at
> org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponentVersion(AbstractRenderStrategy.java:207)
> at
> org.riotfamily.pages.component.render.EditModeRenderStrategy.renderContainer(EditModeRenderStrategy.java:169)
> at
> org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponents(AbstractRenderStrategy.java:156)
> at
> org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponentList(AbstractRenderStrategy.java:137)
> at
> org.riotfamily.pages.component.render.EditModeRenderStrategy.renderComponentList(EditModeRenderStrategy.java:93)
> at
> org.riotfamily.pages.component.render.AbstractRenderStrategy.render(AbstractRenderStrategy.java:87)
> at
> org.riotfamily.pages.component.render.AbstractRenderStrategy.render(AbstractRenderStrategy.java:81)
> at org.riotfamily.pages.component.ComponentListController
> $1.doInTransactionWithoutResult(ComponentListController.java:202)
> at
> org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
> at
> org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)
> at
> org.riotfamily.pages.component.ComponentListController.handleRequest(ComponentListController.java:199)
> at
> org.riotfamily.cachius.spring.CacheableControllerHandlerAdapter.handle(CacheableControllerHandlerAdapter.java:118)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
> at
> org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
> at
> org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
> at
> org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
> at
> org.riotfamily.common.web.view.freemarker.IncludeMethod.exec(IncludeMethod.java:37)
> at
> org.riotfamily.common.web.view.freemarker.AbstractSimpleMethod.exec(AbstractSimpleMethod.java:38)
> at freemarker.core.MethodCall._getAsTemplateModel(MethodCall.java:93)
> at freemarker.core.Expression.getAsTemplateModel(Expression.java:89)
> at freemarker.core.Expression.getStringValue(Expression.java:93)
> at freemarker.core.DollarVariable.accept(DollarVariable.java:76)
> at freemarker.core.Environment.visit(Environment.java:196)
> at freemarker.core.MixedContent.accept(MixedContent.java:92)
> at freemarker.core.Environment.visit(Environment.java:196)
> at freemarker.core.Macro$Context.runMacro(Macro.java:164)
> at freemarker.core.Environment.visit(Environment.java:537)
> at freemarker.core.UnifiedCall.accept(UnifiedCall.java:128)
> at freemarker.core.Environment.visit(Environment.java:196)
> at freemarker.core.MixedContent.accept(MixedContent.java:92)
> at freemarker.core.Environment.visit(Environment.java:196)
> at freemarker.core.Environment.process(Environment.java:176)
> at freemarker.template.Template.process(Template.java:232)
> at
> org.springframework.web.servlet.view.freemarker.FreeMarkerView.processTemplate(FreeMarkerView.java:333)
> at
> org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.doRender(RiotFreeMarkerView.java:113)
> at org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView
> $1.doInContext(RiotFreeMarkerView.java:93)
> at
> org.riotfamily.common.web.view.ViewContext.execute(ViewContext.java:45)
> at
> org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.renderMergedTemplateModel(RiotFreeMarkerView.java:91)
> at
> org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:179)
> at
> org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:247)
> at
> org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.render(RiotFreeMarkerView.java:75)
> at
> org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1123)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:860)
> at
> org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:60)
> at
> org.riotfamily.pages.page.support.FolderFilterPlugin.doFilter(FolderFilterPlugin.java:82)
> at
> org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:57)
> at
> org.riotfamily.riot.security.AccessControlFilterPlugin.doFilter(AccessControlFilterPlugin.java:58)
> at
> org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:57)
> at
> org.riotfamily.common.web.filter.PluginFilter.doFilterInternal(PluginFilter.java:123)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.riotfamily.common.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:95)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
> at org.apache.coyote.http11.Http11BaseProtocol
> $Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
> at
> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
> at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
> at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:685)
> at java.lang.Thread.run(Thread.java:619)
>
>
>
> And our hadoop server logfile (datanode log) contains this several times
> with different timestamp:
>
> 2007-08-21 00:00:23,393 INFO org.apache.hadoop.dfs.DataNode: using
> BLOCKREPORT_INTERVAL of 3356962msec
> 2007-08-21 00:00:23,407 ERROR org.apache.hadoop.dfs.DataNode: Exception:
> java.lang.NullPointerException
> at org.apache.hadoop.dfs.FSDataset
> $FSDir.getBlockInfo(FSDataset.java:148)
> at org.apache.hadoop.dfs.FSDataset
> $FSVolume.getBlockInfo(FSDataset.java:353)
> at org.apache.hadoop.dfs.FSDataset
> $FSVolumeSet.getBlockInfo(FSDataset.java:419)
> at
> org.apache.hadoop.dfs.FSDataset.getBlockReport(FSDataset.java:636)
> at
> org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:523)
> at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1310)
> at java.lang.Thread.run(Thread.java:619)
>
>
> but im not sure if this has anything to do with this particular problem.
> Any help would be really appreciated!
>
> Best regards,
> Jani
>
> On Tue, 2007-08-21 at 11:53 -0500, Sagar Naik wrote:
>
>> Hello
>> Can u provide some more information.
>> Like a stackstrace from log files
>>
>> - Sagar
>> Jani Arvonen wrote:
>>
>>> Hello everybody!
>>>
>>> I've been trying to use hadoop distributed file system from my java
>>> spring web application but without any good results :). We have one
>>> server where hadoop namenode are datanode are succesfully running (so
>>> they all are running on a single node). I managed to configure it with
>>> the help of documentation page found from hadoop website.
>>>
>>> Now i need to make my web application to use this distributed file
>>> system running on the other server. I've tried to place the same
>>> hadoop-site.xml found from the hadoop master server in the programs
>>> classpath and using DistributedFileSystem but without no luck. Always
>>> when trying to use for example copyFromLocalFile it throws a
>>> NullPointerException. Im getting really lost with this issue because i
>>> havent found any examples or tutorials related to this :)
>>>
>>> I would really appreciate if somebody could help me so that i can
>>> connect to this hadoop master server from my java spring web application
>>> and to use distributed file system.
>>>
>>>
>>> best regards,
>>> Jani
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
--
This message has been scanned for viruses and
dangerous content and is believed to be clean.
Re: newbie problem: Using DFS from Java program
Posted by Jani Arvonen <ja...@floobs.com>.
Oh yes, of course. Our web application throws this exception when trying
to use copyFromLocalFile() through DistributedFileSystem. The local file
im trying to copy to the HDFS do exists...
And we have also tried to use LocalFileSystem which works fine but that
is not exactly what we are trying to achieve :)
stacktrace:
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:155)
at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
at
org.apache.hadoop.fs.ChecksumFileSystem.copyFromLocalFile(ChecksumFileSystem.java:576)
at
com.floobs.service.media.DefaultMediaStorage.copyFromLocalFile(DefaultMediaStorage.java:23)
at
com.floobs.service.media.DefaultMediaStorage.copyFromLocalFile(DefaultMediaStorage.java:18)
at
com.floobs.web.media.MediaUploadFormController.storeFile(MediaUploadFormController.java:155)
at com.floobs.web.media.MediaUploadFormController.access
$500(MediaUploadFormController.java:36)
at com.floobs.web.media.MediaUploadFormController
$2.processFile(MediaUploadFormController.java:120)
at com.floobs.web.media.MediaUploadFormController
$UploadFileProcessor.execute(MediaUploadFormController.java:144)
at
com.floobs.web.media.MediaUploadFormController.processFileUploads(MediaUploadFormController.java:114)
at
com.floobs.web.media.MediaUploadFormController.onSubmit(MediaUploadFormController.java:65)
at
org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)
at
org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:250)
at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at
org.riotfamily.cachius.spring.CacheableControllerHandlerAdapter.handle(CacheableControllerHandlerAdapter.java:118)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
at
org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
at
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
at
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
at
org.riotfamily.pages.component.impl.IncludeComponent.renderInternal(IncludeComponent.java:66)
at
org.riotfamily.pages.component.impl.AbstractComponent.render(AbstractComponent.java:98)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponent(AbstractRenderStrategy.java:243)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponentVersion(AbstractRenderStrategy.java:207)
at
org.riotfamily.pages.component.render.EditModeRenderStrategy.renderContainer(EditModeRenderStrategy.java:169)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponents(AbstractRenderStrategy.java:156)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.renderComponentList(AbstractRenderStrategy.java:137)
at
org.riotfamily.pages.component.render.EditModeRenderStrategy.renderComponentList(EditModeRenderStrategy.java:93)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.render(AbstractRenderStrategy.java:87)
at
org.riotfamily.pages.component.render.AbstractRenderStrategy.render(AbstractRenderStrategy.java:81)
at org.riotfamily.pages.component.ComponentListController
$1.doInTransactionWithoutResult(ComponentListController.java:202)
at
org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
at
org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)
at
org.riotfamily.pages.component.ComponentListController.handleRequest(ComponentListController.java:199)
at
org.riotfamily.cachius.spring.CacheableControllerHandlerAdapter.handle(CacheableControllerHandlerAdapter.java:118)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
at
org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
at
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:594)
at
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:505)
at
org.riotfamily.common.web.view.freemarker.IncludeMethod.exec(IncludeMethod.java:37)
at
org.riotfamily.common.web.view.freemarker.AbstractSimpleMethod.exec(AbstractSimpleMethod.java:38)
at freemarker.core.MethodCall._getAsTemplateModel(MethodCall.java:93)
at freemarker.core.Expression.getAsTemplateModel(Expression.java:89)
at freemarker.core.Expression.getStringValue(Expression.java:93)
at freemarker.core.DollarVariable.accept(DollarVariable.java:76)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.Macro$Context.runMacro(Macro.java:164)
at freemarker.core.Environment.visit(Environment.java:537)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:128)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:196)
at freemarker.core.Environment.process(Environment.java:176)
at freemarker.template.Template.process(Template.java:232)
at
org.springframework.web.servlet.view.freemarker.FreeMarkerView.processTemplate(FreeMarkerView.java:333)
at
org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.doRender(RiotFreeMarkerView.java:113)
at org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView
$1.doInContext(RiotFreeMarkerView.java:93)
at
org.riotfamily.common.web.view.ViewContext.execute(ViewContext.java:45)
at
org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.renderMergedTemplateModel(RiotFreeMarkerView.java:91)
at
org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:179)
at
org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:247)
at
org.riotfamily.common.web.view.freemarker.RiotFreeMarkerView.render(RiotFreeMarkerView.java:75)
at
org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1123)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:860)
at
org.riotfamily.common.web.servlet.ReloadableDispatcherServlet.doDispatch(ReloadableDispatcherServlet.java:107)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:774)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:460)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:425)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:60)
at
org.riotfamily.pages.page.support.FolderFilterPlugin.doFilter(FolderFilterPlugin.java:82)
at
org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:57)
at
org.riotfamily.riot.security.AccessControlFilterPlugin.doFilter(AccessControlFilterPlugin.java:58)
at
org.riotfamily.common.web.filter.PluginChain.doFilter(PluginChain.java:57)
at
org.riotfamily.common.web.filter.PluginFilter.doFilterInternal(PluginFilter.java:123)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:70)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.riotfamily.common.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:95)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
at org.apache.coyote.http11.Http11BaseProtocol
$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:619)
And our hadoop server logfile (datanode log) contains this several times
with different timestamp:
2007-08-21 00:00:23,393 INFO org.apache.hadoop.dfs.DataNode: using
BLOCKREPORT_INTERVAL of 3356962msec
2007-08-21 00:00:23,407 ERROR org.apache.hadoop.dfs.DataNode: Exception:
java.lang.NullPointerException
at org.apache.hadoop.dfs.FSDataset
$FSDir.getBlockInfo(FSDataset.java:148)
at org.apache.hadoop.dfs.FSDataset
$FSVolume.getBlockInfo(FSDataset.java:353)
at org.apache.hadoop.dfs.FSDataset
$FSVolumeSet.getBlockInfo(FSDataset.java:419)
at
org.apache.hadoop.dfs.FSDataset.getBlockReport(FSDataset.java:636)
at
org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:523)
at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1310)
at java.lang.Thread.run(Thread.java:619)
but im not sure if this has anything to do with this particular problem.
Any help would be really appreciated!
Best regards,
Jani
On Tue, 2007-08-21 at 11:53 -0500, Sagar Naik wrote:
> Hello
> Can u provide some more information.
> Like a stackstrace from log files
>
> - Sagar
> Jani Arvonen wrote:
> > Hello everybody!
> >
> > I've been trying to use hadoop distributed file system from my java
> > spring web application but without any good results :). We have one
> > server where hadoop namenode are datanode are succesfully running (so
> > they all are running on a single node). I managed to configure it with
> > the help of documentation page found from hadoop website.
> >
> > Now i need to make my web application to use this distributed file
> > system running on the other server. I've tried to place the same
> > hadoop-site.xml found from the hadoop master server in the programs
> > classpath and using DistributedFileSystem but without no luck. Always
> > when trying to use for example copyFromLocalFile it throws a
> > NullPointerException. Im getting really lost with this issue because i
> > havent found any examples or tutorials related to this :)
> >
> > I would really appreciate if somebody could help me so that i can
> > connect to this hadoop master server from my java spring web application
> > and to use distributed file system.
> >
> >
> > best regards,
> > Jani
> >
> >
> >
>
>
> --
> Always vizz it us @ visvo.com
>
>
Re: newbie problem: Using DFS from Java program
Posted by Sagar Naik <sa...@visvo.com>.
Hello
Can u provide some more information.
Like a stackstrace from log files
- Sagar
Jani Arvonen wrote:
> Hello everybody!
>
> I've been trying to use hadoop distributed file system from my java
> spring web application but without any good results :). We have one
> server where hadoop namenode are datanode are succesfully running (so
> they all are running on a single node). I managed to configure it with
> the help of documentation page found from hadoop website.
>
> Now i need to make my web application to use this distributed file
> system running on the other server. I've tried to place the same
> hadoop-site.xml found from the hadoop master server in the programs
> classpath and using DistributedFileSystem but without no luck. Always
> when trying to use for example copyFromLocalFile it throws a
> NullPointerException. Im getting really lost with this issue because i
> havent found any examples or tutorials related to this :)
>
> I would really appreciate if somebody could help me so that i can
> connect to this hadoop master server from my java spring web application
> and to use distributed file system.
>
>
> best regards,
> Jani
>
>
>
--
Always vizz it us @ visvo.com
--
This message has been scanned for viruses and
dangerous content and is believed to be clean.