You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Denis Mekhanikov <dm...@gmail.com> on 2019/04/02 14:31:57 UTC
Re: Error in running wordcount hadoop example in ignite
Hi!
As far as I can see, you tried running examples from Hadoop 2.7.7
But Ignite uses Hadoop version 2.4.1 internally.
So, I would start from checking the same examples for the matching version
of Hadoop (2.4.1).
Denis
чт, 28 февр. 2019 г. в 08:58, mehdi sey <se...@gmail.com>:
> hi
> i want to execute wordcount example of hadoop in apache ignite. i have used
> apache-ignite-hadoop-2.6.0-bin for execute map reduce tasks. my
> default-config.xml in apache-ignite-hadoop-2.6.0-bin/config folder is just
> as bellow:
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:util="http://www.springframework.org/schema/util"
> xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd">
>
>
> <description>
> Spring file for Ignite node configuration with IGFS and Apache
> Hadoop map-reduce support enabled.
> Ignite node will start with this configuration by default.
> </description>
>
>
> <bean id="propertyConfigurer"
>
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
> <property name="systemPropertiesModeName"
> value="SYSTEM_PROPERTIES_MODE_FALLBACK"/>
> <property name="searchSystemEnvironment" value="true"/>
> </bean>
>
>
> <bean id="grid.cfg"
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> <property name="connectorConfiguration">
> <bean
> class="org.apache.ignite.configuration.ConnectorConfiguration">
> <property name="port" value="11211"/>
> </bean>
> </property>
>
> <property name="includeEventTypes">
> <list>
>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_JOB_MAPPED"/>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
> <util:constant
>
> static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>
>
>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
> <util:constant
> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
> <util:constant
>
> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
> </list>
> </property>
>
> i have run an ignite node with bellow command in command line in linux
> ubuntu:
> */usr/local/apache-ignite-hadoop-2.6.0-bin/bin/ignite.sh
> /usr/local/apache-ignite-hadoop-2.6.0-bin/config/default-config.xml*
>
> after starting ignite node i execute a wordcount example in hadoop for
> ruuning in ignite with bellow command:
>
> *./hadoop jar
>
> /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.7.jar
> wordcount /input/hadoop output2*
>
> but after executing above command i have encounter an error just as
> attached
> image. please help about solving problem. i have seen also below link but
> it
> could not help.
>
> http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tc2146.html#a2183
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h2.png>
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h1.png>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Error in running wordcount hadoop example in ignite
Posted by Mehdi Seydali <se...@gmail.com>.
thanks that works with 2.4
On Tue, Apr 2, 2019 at 7:32 AM Denis Mekhanikov <dm...@gmail.com>
wrote:
> Hi!
>
> As far as I can see, you tried running examples from Hadoop 2.7.7
> But Ignite uses Hadoop version 2.4.1 internally.
> So, I would start from checking the same examples for the matching version
> of Hadoop (2.4.1).
>
> Denis
>
> чт, 28 февр. 2019 г. в 08:58, mehdi sey <se...@gmail.com>:
>
>> hi
>> i want to execute wordcount example of hadoop in apache ignite. i have
>> used
>> apache-ignite-hadoop-2.6.0-bin for execute map reduce tasks. my
>> default-config.xml in apache-ignite-hadoop-2.6.0-bin/config folder is just
>> as bellow:
>> <beans xmlns="http://www.springframework.org/schema/beans"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xmlns:util="http://www.springframework.org/schema/util"
>> xsi:schemaLocation="http://www.springframework.org/schema/beans
>> http://www.springframework.org/schema/beans/spring-beans.xsd
>> http://www.springframework.org/schema/util
>> http://www.springframework.org/schema/util/spring-util.xsd">
>>
>>
>> <description>
>> Spring file for Ignite node configuration with IGFS and Apache
>> Hadoop map-reduce support enabled.
>> Ignite node will start with this configuration by default.
>> </description>
>>
>>
>> <bean id="propertyConfigurer"
>>
>> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
>> <property name="systemPropertiesModeName"
>> value="SYSTEM_PROPERTIES_MODE_FALLBACK"/>
>> <property name="searchSystemEnvironment" value="true"/>
>> </bean>
>>
>>
>> <bean id="grid.cfg"
>> class="org.apache.ignite.configuration.IgniteConfiguration">
>>
>> <property name="connectorConfiguration">
>> <bean
>> class="org.apache.ignite.configuration.ConnectorConfiguration">
>> <property name="port" value="11211"/>
>> </bean>
>> </property>
>>
>> <property name="includeEventTypes">
>> <list>
>>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_JOB_MAPPED"/>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
>> <util:constant
>>
>> static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>
>>
>>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
>> <util:constant
>> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
>> <util:constant
>>
>> static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
>> </list>
>> </property>
>>
>> i have run an ignite node with bellow command in command line in linux
>> ubuntu:
>> */usr/local/apache-ignite-hadoop-2.6.0-bin/bin/ignite.sh
>> /usr/local/apache-ignite-hadoop-2.6.0-bin/config/default-config.xml*
>>
>> after starting ignite node i execute a wordcount example in hadoop for
>> ruuning in ignite with bellow command:
>>
>> *./hadoop jar
>>
>> /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.7.jar
>> wordcount /input/hadoop output2*
>>
>> but after executing above command i have encounter an error just as
>> attached
>> image. please help about solving problem. i have seen also below link but
>> it
>> could not help.
>>
>> http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tc2146.html#a2183
>> <http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h2.png>
>> <http://apache-ignite-users.70518.x6.nabble.com/file/t2160/h1.png>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>