You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Samarth Gahire (Created) (JIRA)" <ji...@apache.org> on 2012/01/13 13:14:40 UTC
[jira] [Created] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
------------------------------------------------------------------------------
Key: CASSANDRA-3740
URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1
Reporter: Samarth Gahire
Fix For: 1.1
I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
But still when I run the job i get the following error:
{
12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
}
Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0002-Prevent-loading-from-yaml.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Erik Forsberg (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198796#comment-13198796 ]
Erik Forsberg commented on CASSANDRA-3740:
------------------------------------------
>Actually we are experiencing the problem while loading the data where it fails to connect if the host the M/R job is running on is dualstack, i.e. has both IPv4 >and IPv6.
>Also it works when cassandra.yaml is provided ,may be it is reading listen address or something from cassandra.yaml.
Described this problem in more detail in CASSANDRA-3839.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0004-update-BOF-for-new-dir-layout.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: 0003-use-output-partitioner.txt
Fix incorrect variable scope causing NPE.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3740) While using
BulkOutputFormat unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207472#comment-13207472 ]
Brandon Williams edited comment on CASSANDRA-3740 at 2/14/12 3:03 AM:
----------------------------------------------------------------------
bq. That has a scoping issue, where you set File output, so scope becomes local.
Are you sure you tried the latest patch on this ticket?
Edit: nevermind, I squashed 004 into 003 and never deleted it.
was (Author: brandon.williams):
bq. That has a scoping issue, where you set File output, so scope becomes local.
Are you sure you tried the latest patch on this ticket?
Edit: nevermind, I uploaded the wrong file, despite commenting that I fixed it :)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198823#comment-13198823 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
bq. Described this problem in more detail in CASSANDRA-3839.
I see, let's address this there.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Reviewer: lenn0x (was: jbellis)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207472#comment-13207472 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
bq. That has a scoping issue, where you set File output, so scope becomes local.
Are you sure you tried the latest patch on this ticket?
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185772#comment-13185772 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
Are you trying this on trunk? Because specifically patch 0002 on CASSANDRA-3045 addresses this problem.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186185#comment-13186185 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
One More issue:
I can set the "Partitioner" into jobconf using "ConfigHelper" ,but it is no where used to set into the "DatabaseDescriptor" and not even initialized into the "DatabaseDescriptor.initDefaultsOnly()".
But "AbstractSSTableSimpleWriter" uses partioner (at line number 94) which will result into the null pointer.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Erik Forsberg (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207714#comment-13207714 ]
Erik Forsberg commented on CASSANDRA-3740:
------------------------------------------
bq. Samarth/Erik, How does this patch look?
Overall, it's looking good. I haven't tested the latest version of this patch on a large data set yet, hoping to find time to do that tomorrow.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0006-BWR-uses-any-if.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3740:
--------------------------------------
Reviewer: jbellis (was: _spa)
Remaining Estimate: (was: 72h)
Original Estimate: (was: 72h)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, configuration, hadoop, mapreduce, test
> Fix For: 1.1
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188320#comment-13188320 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
Apart From these issues I do not think that we are considering the TTL case in BulkOutputFormat.
Have looked into the code and can only see the "addColumn()" method and the "addExpiringColumn()" is not used anywhere.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206969#comment-13206969 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
Samarth/Eric,
How does this patch look?
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: 0004-update-BOF-for-new-dir-layout.txt
0003-use-output-partitioner.txt
0002-Prevent-loading-from-yaml.txt
0001-Make-DD-the-canonical-partitioner-source.txt
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0003-use-output-partitioner.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0004-update-BOF-for-new-dir-layout.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0001-Make-DD-the-canonical-partitioner-source.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0004-update-BOF-for-new-dir-layout.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186086#comment-13186086 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
I am using "apache-cassandra-2011-12-27_22-01-50-bin.tar.gz" build from jenkins . Let me know if I am using the correct binaries .
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Jonathan Ellis (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-3740:
-----------------------------------------
Assignee: Brandon Williams
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, configuration, hadoop, mapreduce, test
> Fix For: 1.1
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0003-use-output-partitioner.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: 0003-use-output-partitioner.txt
0002-Prevent-loading-from-yaml.txt
0001-Make-DD-the-canonical-partitioner-source.txt
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198577#comment-13198577 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
Cool! Its Working Perfect with the updated patches.
Can you please explain
1) what is the significance of "INPUT_INITIAL_THRIFT_ADDRESS" for BulkOutPutFormat.
2) What am I suppose to provide there?(If it is needed)
3) Is there any need to provide Listen address of the Hadoop Nodes for BulkOutputFormat if yes How to provide the same?
Actually we are experiencing the problem while loading the data where it fails to connect if the host the M/R job is running on is dualstack, i.e. has both IPv4 and IPv6.
Also it works when cassandra.yaml is provided ,may be it is reading listen address or something from cassandra.yaml.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: 0006-BWR-uses-any-if.txt
0005-Allow-using-any-interface-for-outgoing-connections.txt
Two more patches to address CASSANDRA-3839
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-Allow-using-any-interface-for-outgoing-connections.txt, 0006-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0001-Make-DD-the-canonical-partitioner-source.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186140#comment-13186140 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
I have checked into the code and I can see following issues:
1) "org.apache.cassandra.config.Config" do not initialize the all the properties and which results into the null pointer exception in a static block of class "DatabaseDescriptor" for example "conf.commitlog_sync"
2) I cant see any method in ConfigHelper to specify that I am using "Supercolumn" and clustername(dont know if cluster name is really needed)
3) Also there is no method to specify comparator and subcomparator in ConfigHelper and it seems like comparators have been hard coded to "BytesType"
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197745#comment-13197745 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
I Tested the patches all the patches except the yaml one works fine.
I have checked the Yaml patch and my job still look for the cassandra.yaml file and fails.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189205#comment-13189205 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
bq. 1) "org.apache.cassandra.config.Config" do not initialize the all the properties and which results into the null pointer exception in a static block of class "DatabaseDescriptor" for example "conf.commitlog_sync"
Patches to address that.
bq. 2) I cant see any method in ConfigHelper to specify that I am using "Supercolumn"
"mapreduce.output.bulkoutputformat.issuper" controls that.
bq. 3) Also there is no method to specify comparator and subcomparator in ConfigHelper and it seems like comparators have been hard coded to "BytesType"
BytesType will sort correctly, the comparators are in the schema on the remote nodes.
bq. Apart From these issues I do not think that we are considering the TTL case in BulkOutputFormat.
CASSANDRA-3754 will handle this.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Chris Goffinet (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207521#comment-13207521 ]
Chris Goffinet commented on CASSANDRA-3740:
-------------------------------------------
Brandon, other than that, I +1 this patch. If you don't mind reviewing my other patches :) I am about to post another for making compression work with bulk writer too, and it depends on this going in.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3740) While using
BulkOutputFormat unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207472#comment-13207472 ]
Brandon Williams edited comment on CASSANDRA-3740 at 2/14/12 3:00 AM:
----------------------------------------------------------------------
bq. That has a scoping issue, where you set File output, so scope becomes local.
Are you sure you tried the latest patch on this ticket?
Edit: nevermind, I uploaded the wrong file, despite commenting that I fixed it :)
was (Author: brandon.williams):
bq. That has a scoping issue, where you set File output, so scope becomes local.
Are you sure you tried the latest patch on this ticket?
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Samarth Gahire (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206986#comment-13206986 ]
Samarth Gahire commented on CASSANDRA-3740:
-------------------------------------------
First 4 patches working fine.
About the patch related to CASSANDRA-3839 Erik can explain properly.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0003-use-output-partitioner.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0005-Allow-using-any-interface-for-outgoing-connections.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198095#comment-13198095 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
I added a way to get around the yaml, then forgot to make BOF use it. Updated patches should work.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: 0004-update-BOF-for-new-dir-layout.txt
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: (was: 0002-Prevent-loading-from-yaml.txt)
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Chris Goffinet (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207469#comment-13207469 ]
Chris Goffinet commented on CASSANDRA-3740:
-------------------------------------------
Brandon,
I just filed CASSANDRA-3906, this fixes counter column NPE. Also we tested out these patches and based on patch: 0004-update-BOF-for-new-dir-layout.txt
That has a scoping issue, where you set File output, so scope becomes local.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3740) While using
BulkOutputFormat unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206969#comment-13206969 ]
Brandon Williams edited comment on CASSANDRA-3740 at 2/13/12 4:36 PM:
----------------------------------------------------------------------
Samarth/Erik,
How does this patch look?
was (Author: brandon.williams):
Samarth/Eric,
How does this patch look?
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1.0
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1.0
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198793#comment-13198793 ]
Brandon Williams commented on CASSANDRA-3740:
---------------------------------------------
bq. what is the significance of "INPUT_INITIAL_THRIFT_ADDRESS" for BulkOutPutFormat.
For an output format, this won't be used, it's only for input formats.
bq. Is there any need to provide Listen address of the Hadoop Nodes for BulkOutputFormat if yes How to provide the same?
I'm not sure what you mean, hadoop nodes themselves won't have a listen address, and BOF will discover the cassandra nodes' listen address via thrift.
bq. Actually we are experiencing the problem while loading the data where it fails to connect if the host the M/R job is running on is dualstack, i.e. has both IPv4 and IPv6. Also it works when cassandra.yaml is provided ,may be it is reading listen address or something from cassandra.yaml.
Hmm, I can't think of any reason that would work with the yaml, can you give more details of the setup?
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3740) While using BulkOutputFormat
unneccessarily look for the cassandra.yaml file.
Posted by "Brandon Williams (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3740:
----------------------------------------
Attachment: 0005-BWR-uses-any-if.txt
0004-update-BOF-for-new-dir-layout.txt
0003-use-output-partitioner.txt
0002-Prevent-loading-from-yaml.txt
0001-Make-DD-the-canonical-partitioner-source.txt
Rebased.
> While using BulkOutputFormat unneccessarily look for the cassandra.yaml file.
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-3740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3740
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.1
> Reporter: Samarth Gahire
> Assignee: Brandon Williams
> Labels: cassandra, hadoop, mapreduce
> Fix For: 1.1
>
> Attachments: 0001-Make-DD-the-canonical-partitioner-source.txt, 0002-Prevent-loading-from-yaml.txt, 0003-use-output-partitioner.txt, 0004-update-BOF-for-new-dir-layout.txt, 0005-BWR-uses-any-if.txt
>
>
> I am trying to use BulkOutputFormat to stream the data from map of Hadoop job. I have set the cassandra related configuration using ConfigHelper ,Also have looked into Cassandra code seems Cassandra has taken care that it should not look for the cassandra.yaml file.
> But still when I run the job i get the following error:
> {
> 12/01/13 11:30:04 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
> 12/01/13 11:30:04 INFO input.FileInputFormat: Total input paths to process : 1
> 12/01/13 11:30:04 INFO mapred.JobClient: Running job: job_201201130910_0015
> 12/01/13 11:30:05 INFO mapred.JobClient: map 0% reduce 0%
> 12/01/13 11:30:23 INFO mapred.JobClient: Task Id : attempt_201201130910_0015_m_000000_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
> attempt_201201130910_0015_m_000000_0: Cannot locate cassandra.yaml
> attempt_201201130910_0015_m_000000_0: Fatal configuration error; unable to start server.
> }
> Also let me know how can i make this cassandra.yaml file available to Hadoop mapreduce job?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira