You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by akshay naidu <ak...@gmail.com> on 2017/01/06 14:38:05 UTC
unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Hello,
I am not able to run *kafka-to-HDFS-sync* app.
apache kafka server on my ubuntu machine is up and running. I have created
a topic called transactions which is created from a csv file. I want to
load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app and generate
graphic display just to analyse.
I am successfully able to import the app, but when i launch the app it runs
showing the state as *ACCEPTED* but after few seconds the *STATE* is
changed to *FAILED*.
I have attached a screenshot of *launch properties* and *Application
overview* page.
Kindly assist me.
Thank You.
Re: unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Posted by akshay naidu <ak...@gmail.com>.
Hi,
I added the following lines in yarn-site and also i switched to kafka-0.9
.. looks like one of these two or both changes worked for me.. now its
working absolutely fine..
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
<description>Whether virtual memory limits will be enforced for
containers</description>
</property>
Thank you very much to all of you for your help.
I am trying to create a graphic display similar to the attached image. Is
it possible to do that in dataTorrent..
Thanks.
On Sat, Jan 7, 2017 at 12:17 PM, Chaitanya Chebolu <
chaitanya@datatorrent.com> wrote:
> Hi Akshay,
>
> Issue is exceeding the virtual limits. Please refer the below link
> about how to fix this issue:
> http://docs.datatorrent.com/configuration/#yarn-vmem-pmem-ratio-tuning
>
> Please install 0.9* version of Kafka. Because, the App which you are
> trying is compatible with 0.9* version of Kafka.
>
> Regards,
> Chaitanya
>
> On Sat, Jan 7, 2017 at 11:28 AM, akshay naidu <ak...@gmail.com>
> wrote:
>
>> Hello Ajay,
>> dt.logs is not accessible , I am attaching yarn-user-resourcema
>> nager-akshay-300E4Z-300E5Z-300E7Z.log
>> I hope this will also help.
>>
>> On Sat, Jan 7, 2017 at 9:42 AM, AJAY GUPTA <aj...@gmail.com> wrote:
>>
>>> Hi Akshay,
>>>
>>> Can you attach the dtgateway.log file found in ~/.dt/logs
>>>
>>> Ajay
>>>
>>> On Fri, Jan 6, 2017 at 10:24 PM, akshay naidu <ak...@gmail.com>
>>> wrote:
>>>
>>>> Hello Sandesh,
>>>> its kafka_2.10-0.10.0.1
>>>>
>>>> On Fri, Jan 6, 2017 at 8:44 PM, Sandesh Hegde <sa...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Which version of the Kafka are you using?
>>>>>
>>>>>
>>>>> On Fri, Jan 6, 2017 at 6:38 AM akshay naidu <ak...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>> I am not able to run *kafka-to-HDFS-sync* app.
>>>>>> apache kafka server on my ubuntu machine is up and running. I have
>>>>>> created a topic called transactions which is created from a csv file. I
>>>>>> want to load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app
>>>>>> and generate graphic display just to analyse.
>>>>>> I am successfully able to import the app, but when i launch the app
>>>>>> it runs showing the state as *ACCEPTED* but after few seconds the
>>>>>> *STATE* is changed to *FAILED*.
>>>>>> I have attached a screenshot of *launch properties* and *Application
>>>>>> overview* page.
>>>>>> Kindly assist me.
>>>>>> Thank You.
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Posted by Chaitanya Chebolu <ch...@datatorrent.com>.
Hi Akshay,
Issue is exceeding the virtual limits. Please refer the below link about
how to fix this issue:
http://docs.datatorrent.com/configuration/#yarn-vmem-pmem-ratio-tuning
Please install 0.9* version of Kafka. Because, the App which you are trying
is compatible with 0.9* version of Kafka.
Regards,
Chaitanya
On Sat, Jan 7, 2017 at 11:28 AM, akshay naidu <ak...@gmail.com>
wrote:
> Hello Ajay,
> dt.logs is not accessible , I am attaching yarn-user-
> resourcemanager-akshay-300E4Z-300E5Z-300E7Z.log
> I hope this will also help.
>
> On Sat, Jan 7, 2017 at 9:42 AM, AJAY GUPTA <aj...@gmail.com> wrote:
>
>> Hi Akshay,
>>
>> Can you attach the dtgateway.log file found in ~/.dt/logs
>>
>> Ajay
>>
>> On Fri, Jan 6, 2017 at 10:24 PM, akshay naidu <ak...@gmail.com>
>> wrote:
>>
>>> Hello Sandesh,
>>> its kafka_2.10-0.10.0.1
>>>
>>> On Fri, Jan 6, 2017 at 8:44 PM, Sandesh Hegde <sa...@datatorrent.com>
>>> wrote:
>>>
>>>> Which version of the Kafka are you using?
>>>>
>>>>
>>>> On Fri, Jan 6, 2017 at 6:38 AM akshay naidu <ak...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>> I am not able to run *kafka-to-HDFS-sync* app.
>>>>> apache kafka server on my ubuntu machine is up and running. I have
>>>>> created a topic called transactions which is created from a csv file. I
>>>>> want to load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app
>>>>> and generate graphic display just to analyse.
>>>>> I am successfully able to import the app, but when i launch the app it
>>>>> runs showing the state as *ACCEPTED* but after few seconds the *STATE* is
>>>>> changed to *FAILED*.
>>>>> I have attached a screenshot of *launch properties* and *Application
>>>>> overview* page.
>>>>> Kindly assist me.
>>>>> Thank You.
>>>>>
>>>>
>>>
>>
>
Re: unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Posted by akshay naidu <ak...@gmail.com>.
Hello Ajay,
dt.logs is not accessible , I am
attaching yarn-user-resourcemanager-akshay-300E4Z-300E5Z-300E7Z.log
I hope this will also help.
On Sat, Jan 7, 2017 at 9:42 AM, AJAY GUPTA <aj...@gmail.com> wrote:
> Hi Akshay,
>
> Can you attach the dtgateway.log file found in ~/.dt/logs
>
> Ajay
>
> On Fri, Jan 6, 2017 at 10:24 PM, akshay naidu <ak...@gmail.com>
> wrote:
>
>> Hello Sandesh,
>> its kafka_2.10-0.10.0.1
>>
>> On Fri, Jan 6, 2017 at 8:44 PM, Sandesh Hegde <sa...@datatorrent.com>
>> wrote:
>>
>>> Which version of the Kafka are you using?
>>>
>>>
>>> On Fri, Jan 6, 2017 at 6:38 AM akshay naidu <ak...@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>> I am not able to run *kafka-to-HDFS-sync* app.
>>>> apache kafka server on my ubuntu machine is up and running. I have
>>>> created a topic called transactions which is created from a csv file. I
>>>> want to load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app and
>>>> generate graphic display just to analyse.
>>>> I am successfully able to import the app, but when i launch the app it
>>>> runs showing the state as *ACCEPTED* but after few seconds the *STATE* is
>>>> changed to *FAILED*.
>>>> I have attached a screenshot of *launch properties* and *Application
>>>> overview* page.
>>>> Kindly assist me.
>>>> Thank You.
>>>>
>>>
>>
>
Re: unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Posted by AJAY GUPTA <aj...@gmail.com>.
Hi Akshay,
Can you attach the dtgateway.log file found in ~/.dt/logs
Ajay
On Fri, Jan 6, 2017 at 10:24 PM, akshay naidu <ak...@gmail.com>
wrote:
> Hello Sandesh,
> its kafka_2.10-0.10.0.1
>
> On Fri, Jan 6, 2017 at 8:44 PM, Sandesh Hegde <sa...@datatorrent.com>
> wrote:
>
>> Which version of the Kafka are you using?
>>
>>
>> On Fri, Jan 6, 2017 at 6:38 AM akshay naidu <ak...@gmail.com>
>> wrote:
>>
>>> Hello,
>>> I am not able to run *kafka-to-HDFS-sync* app.
>>> apache kafka server on my ubuntu machine is up and running. I have
>>> created a topic called transactions which is created from a csv file. I
>>> want to load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app and
>>> generate graphic display just to analyse.
>>> I am successfully able to import the app, but when i launch the app it
>>> runs showing the state as *ACCEPTED* but after few seconds the *STATE* is
>>> changed to *FAILED*.
>>> I have attached a screenshot of *launch properties* and *Application
>>> overview* page.
>>> Kindly assist me.
>>> Thank You.
>>>
>>
>
Re: unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Posted by akshay naidu <ak...@gmail.com>.
Hello Sandesh,
its kafka_2.10-0.10.0.1
On Fri, Jan 6, 2017 at 8:44 PM, Sandesh Hegde <sa...@datatorrent.com>
wrote:
> Which version of the Kafka are you using?
>
>
> On Fri, Jan 6, 2017 at 6:38 AM akshay naidu <ak...@gmail.com>
> wrote:
>
>> Hello,
>> I am not able to run *kafka-to-HDFS-sync* app.
>> apache kafka server on my ubuntu machine is up and running. I have
>> created a topic called transactions which is created from a csv file. I
>> want to load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app and
>> generate graphic display just to analyse.
>> I am successfully able to import the app, but when i launch the app it
>> runs showing the state as *ACCEPTED* but after few seconds the *STATE* is
>> changed to *FAILED*.
>> I have attached a screenshot of *launch properties* and *Application
>> overview* page.
>> Kindly assist me.
>> Thank You.
>>
>
Re: unable to lunch Kafka-to-HDFS-sync app in DataTorrent.
Posted by Sandesh Hegde <sa...@datatorrent.com>.
Which version of the Kafka are you using?
On Fri, Jan 6, 2017 at 6:38 AM akshay naidu <ak...@gmail.com> wrote:
> Hello,
> I am not able to run *kafka-to-HDFS-sync* app.
> apache kafka server on my ubuntu machine is up and running. I have created
> a topic called transactions which is created from a csv file. I want to
> load this topic in *dataTorrent* in Kafka-to-Hdfs-sync app and generate
> graphic display just to analyse.
> I am successfully able to import the app, but when i launch the app it
> runs showing the state as *ACCEPTED* but after few seconds the *STATE* is
> changed to *FAILED*.
> I have attached a screenshot of *launch properties* and *Application
> overview* page.
> Kindly assist me.
> Thank You.
>