You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "David Greco (JIRA)" <ji...@apache.org> on 2009/11/18 12:34:52 UTC
[jira] Created: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
A Camel component for writing into an Hadoop filesystem
-------------------------------------------------------
Key: CAMEL-2197
URL: https://issues.apache.org/activemq/browse/CAMEL-2197
Project: Apache Camel
Issue Type: New Feature
Affects Versions: 2.1.0
Reporter: David Greco
This component (in attachment) provides support for writing data into an Hadoop filesystem.
You can give a look to the following blog post showing some info about its usage:
http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56704#action_56704 ]
Willem Jiang commented on CAMEL-2197:
-------------------------------------
Hi David,
I just took a quick look at this camel-hadoop component, the code is shaped well, I will commit your patch shortly , so you can keep on working it within camel trunk :)
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2197) A Camel component for
writing into an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56698#action_56698 ]
David Greco edited comment on CAMEL-2197 at 12/30/09 9:52 AM:
--------------------------------------------------------------
Hello Willem,
thanks a lot for your suggestion, I integrated that dependency into the component's pom file and I attached the patch.
I deleted also the previous ones for wasting useless space.
Thanks,
David
was (Author: dgreco):
Latest version
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56534#action_56534 ]
David Greco commented on CAMEL-2197:
------------------------------------
Claus,
this component is almost done, so as soon as you are ready to release the 2.2 I could add the latest patch for this component, now I'm just adding minor stuff and some additional test. It's already pretty stable and I think the code quality should be fine, I'm using sonar for code quality check and I have 100% rule compliancy :)
Do I have to put a documentation page on the Camel wiki or is it to early ? Let me know
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56525#action_56525 ]
David Greco commented on CAMEL-2197:
------------------------------------
Claus,
yes, I'm still working on it, I have a new version with the capability to poll the hdfs file system and better handling of erroneous situation plus addition tests.
What do you suggest ? Do I need to post regular updates ? Off course having commit right it would be easier for me to keep up to date the development of this component. Maybe I could consider to put it under another repository, fusesource ?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56713#action_56713 ]
David Greco commented on CAMEL-2197:
------------------------------------
Thanks Claus,
if I ever find a solution for solving this problem I'll let you know
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55553#action_55553 ]
Hadrian Zbarcea commented on CAMEL-2197:
----------------------------------------
David, awesome patch!
We generally don't put other jars in our svn repo and prefer to take them from maven repo. According to this [Hadoop issue|http://issues.apache.org/jira/browse/HADOOP-6331], the hadoop-core-21 will be pushed in a public repo. Would it be ok to wait until then?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55558#action_55558 ]
Hadrian Zbarcea commented on CAMEL-2197:
----------------------------------------
Let's see how long it takes for Hadoop to release the next version. If it takes to long maybe we'll publish ver 20 somewhere.
Btw, hadoop is getting a lot of attention as of late, good timing.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Greco updated CAMEL-2197:
-------------------------------
Attachment: (was: patchfile.txt)
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56691#action_56691 ]
Willem Jiang commented on CAMEL-2197:
-------------------------------------
Hi David,
Although Hadoop doesn't use maven to build [], but it publish the jars into the maven repository [HADOOP-5107|http://issues.apache.org/jira/browse/HADOOP-5107].
And you can find the hadoop-core module with this dependency.
{code}
<dependency>
<groupId>org.apache.mahout.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.1</version>
</dependency>
{code}
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56725#action_56725 ]
David Greco commented on CAMEL-2197:
------------------------------------
Adrian, Willem,
so, what I'm going to do is to start the process for adding a new project under forge, then at any time when you think it's safe you could put that stuff into the main trunk.
As soon I have this project set up under forge I will put a new page under the wiki.
Does it work for you ?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56829#action_56829 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
The documentation is better. I think you should emphasize the problem is that Hadoop requires JDK1.6+ which cannot work together with 1.5 that Camel uses.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2197) A Camel component for
writing into an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56691#action_56691 ]
Willem Jiang edited comment on CAMEL-2197 at 12/30/09 1:56 AM:
---------------------------------------------------------------
Hi David,
Although Hadoop doesn't use maven to build, but it publish the jars into the maven repository [HADOOP-5107|http://issues.apache.org/jira/browse/HADOOP-5107].
And you can find the hadoop-core module with this dependency.
{code}
<dependency>
<groupId>org.apache.mahout.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.1</version>
</dependency>
{code}
was (Author: njiang):
Hi David,
Although Hadoop doesn't use maven to build [], but it publish the jars into the maven repository [HADOOP-5107|http://issues.apache.org/jira/browse/HADOOP-5107].
And you can find the hadoop-core module with this dependency.
{code}
<dependency>
<groupId>org.apache.mahout.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.1</version>
</dependency>
{code}
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56705#action_56705 ]
Willem Jiang commented on CAMEL-2197:
-------------------------------------
Hi David,
I just tried to applied your patch, and found some issues here.
1. Checkstyle error, there are some CS complains about the ASF header and some package import errors.
It's not a big deal.
2. It looks like the hadoop core is require JDK 1.6, so you had to tell maven compile plugin to compile the source with 1.6.
I try to remove the that configure and got this error.
{code}
/Users/jiangning/Work/camel/trunk/components/camel-hdfs/src/main/java/org/apache/camel/component/hdfs/HdfsWritableFactories.java:[23,20] package javax.xml.ws does not exist
/Users/jiangning/Work/camel/trunk/components/camel-hdfs/src/main/java/org/apache/camel/component/hdfs/HdfsConsumer.java:[25,-1] cannot access org.apache.hadoop.fs.FileStatus
bad class file: /Users/jiangning/.m2/repository/org/apache/mahout/hadoop/hadoop-core/0.20.1/hadoop-core-0.20.1.jar(org/apache/hadoop/fs/FileStatus.class)
class file has wrong version 50.0, should be 49.0
{code}
It will cause some trouble if we put it into Camel trunk, as we still use JDK 1.5.x for our daily build.
Before we figure out how do resolve issue 2, I can't submit your patch into Camel trunk :(
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Greco updated CAMEL-2197:
-------------------------------
Attachment: patchfile.txt
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-2197:
-------------------------------
Fix Version/s: 2.3.0
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56530#action_56530 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
David I think the ball is on your turf. We would like to keep 2.2 as a short release and e.g. release it in Jan/Feb next year. So I am just scoping if you need like 1-2 months to get this done to a ready for Camel end users to use, then we should scope it for 2.3.
Its also a good idea to host it elsewhere where you can work on it as much you like and be committer.
We can only include it in the core dist at Apache if there are no license issues and that the component has a broad appeal and the quality is good :)
Which I am sure as you have done Camel components in the past.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56514#action_56514 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
David is this feature complete or do you plan adding more stuff to it?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-2197.
--------------------------------
Resolution: Fixed
Fix Version/s: (was: Future)
2.2.0
David lets close this ticket. Then when JDK 1.6 is minim then we can migrate the project to Apache.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: 2.2.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-2197:
-------------------------------
Fix Version/s: (was: 2.3.0)
Future
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55560#action_55560 ]
David Greco commented on CAMEL-2197:
------------------------------------
Hadoop is awesome and Camel as well, I think that they can complete each other. Let me try ...
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56685#action_56685 ]
Willem Jiang commented on CAMEL-2197:
-------------------------------------
Hi David,
I just check the big patch file , I found you put the hadoop core jar into your patch file.
Can you use the maven instead of putting the jar into the component lib ?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56830#action_56830 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
David I have not looked at the actual source code, but I picked up the HdsfConsumer just to glaze it.
I think you should consider extending {{ScheduledPollConsumer}} if you want the consumer to be schedule based. Then you got all that for free from the camel-core to setup the intervals etc.
And the camel version is more robust as it use ExecutorService and have reliable start/shutdown etc.
All you need to do is to implement the {{poll}} method.
Also camel-core have DEBUG/TRACE logging for starting/stopping a consumer so that is often not needed either.
And I wonder when you read the file you may need a try .. finally block to ensure calling the unlock method.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56727#action_56727 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
David I think that is a good plan
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56810#action_56810 ]
David Greco commented on CAMEL-2197:
------------------------------------
Hi Claus,
last update:
1) I created a project camel-hadoop under github: http://github.com/dgreco/camel-hadoop
2) I created a new page under the Camel wiki (work in progress): http://cwiki.apache.org/confluence/display/CAMEL/HDFS with the proper link to the github project
I'm gonna fill up that page with proper documentation
Hoping it's fine
regards,
David
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Greco updated CAMEL-2197:
-------------------------------
Attachment: (was: patchfile.txt.zip)
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56723#action_56723 ]
Willem Jiang commented on CAMEL-2197:
-------------------------------------
@Hadrian,
If we can make sure the deploy box is running with JDK 1.6, the camel-hdfs can be deployed without an issue.
If we can't do that, hosting the component in the forge could be a better solution.
@David
We can host the camel-hdfs component wiki page in camel.apache.org wherever the camel-hdfs module is :)
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56826#action_56826 ]
David Greco commented on CAMEL-2197:
------------------------------------
Claus, done. Let me know it=f what I wrote is enough.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Greco updated CAMEL-2197:
-------------------------------
Attachment: patchfile.txt.zip
This new patch contains the latest version of the camel-hdfs component, it contains also the consumer part, I'm going to put some documentation on my blog www.davidgreco.it. As soon, if ever, the component will be put into the camel distribution I'll add a manual page to the official camel wiki.
This patch has been produced with camel 2.2-SNAPSHOT
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56719#action_56719 ]
David Greco commented on CAMEL-2197:
------------------------------------
Hi Hadrian,
the point is that Hadoop is compiled and published to the public maven rep using jdk1.6 that seems mandatory. So, at the end the forge interim solution could be fine.
Maybe it would be easier for me to get the commit Karma for the forge repository, since I'm improving and adding new features this would be the best solution for me.
Let me know.
Regards and Happy New Year to you all !
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56708#action_56708 ]
David Greco commented on CAMEL-2197:
------------------------------------
Hi Willem,
I checked the Hadoop prerequisites and it seems that jdk1.6 is mandatory. So, I doubt it's possible to solve that issue, it would mean trying to recompile hadoop core itself trying to sort out the
any problem it could arise :(
It's really a pity. Anyway, do you have any plan to deprecate 1.5 given the fact that it had been already deprecated by SUN putting it in EOL ?
In the meanwhile, the only thing I could do is to make available the work I'm doing on the integration of Camel and Hadoop on another repository, I'm working also on the integration of the map/reduce part. The idea is to allow Camel to drive an entire M/R job from the input production till reading the output produced by the job itself.
Anyway, thanks for your work.
Regards,
David
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56714#action_56714 ]
Willem Jiang commented on CAMEL-2197:
-------------------------------------
Maybe we can add a profile to include camel-hdfs module when we use JDK1.6, and make sure we deploy the kit with a JDK 1.6 box.
Then we can still host the camel-hdfs in the camel trunk :)
Any thought?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2197) A Camel component for
writing into an Hadoop filesystem
Posted by "Willem Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56691#action_56691 ]
Willem Jiang edited comment on CAMEL-2197 at 12/30/09 1:57 AM:
---------------------------------------------------------------
Hi David,
Although Hadoop doesn't use maven to build, but it publish the snapshot jars into the maven repository [HADOOP-5107|http://issues.apache.org/jira/browse/HADOOP-5107].
And you can find the hadoop-core module with this dependency.
{code}
<dependency>
<groupId>org.apache.mahout.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.1</version>
</dependency>
{code}
was (Author: njiang):
Hi David,
Although Hadoop doesn't use maven to build, but it publish the jars into the maven repository [HADOOP-5107|http://issues.apache.org/jira/browse/HADOOP-5107].
And you can find the hadoop-core module with this dependency.
{code}
<dependency>
<groupId>org.apache.mahout.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.1</version>
</dependency>
{code}
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56712#action_56712 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
No no plan to drop JDK1.5 there are just too many users still on it.
Its actually a problem that Hadoop is so dependent on JDK1.6 at it offers little extra over JDK1.5.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Hadrian Zbarcea (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56717#action_56717 ]
Hadrian Zbarcea commented on CAMEL-2197:
----------------------------------------
I have mixed feelings about this. The kits are built with jdk 1.5 so this will result in camel-hdfs not being included in the kit, unless we jump through serious hoops.
Is there any other dependency we can add for package {{javax.xml.ws}} so it at least compile with 1.5? Another (interim) solution is to host camel-hdfs at the forge.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-2197:
-------------------------------
Affects Version/s: (was: 2.1.0)
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55556#action_55556 ]
David Greco commented on CAMEL-2197:
------------------------------------
Hi Hadrian,
I'm glad you like it, and yes, sure, I can wait till then. I could also have time for adding new features ;)
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56686#action_56686 ]
David Greco commented on CAMEL-2197:
------------------------------------
Hi Willem,
I couldn't find any maven public repository with that jar file. It seems that the Hadoop people don't like maven ;)
I have a new version ready, just tell me if you want me t attach the patch.
How can we solve this maven problem ?
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt, patchfile.txt.zip
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56823#action_56823 ]
Claus Ibsen commented on CAMEL-2197:
------------------------------------
David you should document on the wiki page where this project is hosted. And a reason why its there, eg. it requires Java 1.6 etc.
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2197) A Camel component for writing into an
Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Greco updated CAMEL-2197:
-------------------------------
Attachment: patchfile.txt
Latest version
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Affects Versions: 2.1.0
> Reporter: David Greco
> Fix For: 2.3.0
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2197) A Camel component for writing into
an Hadoop filesystem
Posted by "David Greco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56716#action_56716 ]
David Greco commented on CAMEL-2197:
------------------------------------
Willem,
yes, I like your idea. Off course I'm biased here, but I think that the integration of Camel with Hadoop/HDFS is interesting and useful.
So, I would be glad if I could work directly on the main trunk.
Regards,
David
> A Camel component for writing into an Hadoop filesystem
> -------------------------------------------------------
>
> Key: CAMEL-2197
> URL: https://issues.apache.org/activemq/browse/CAMEL-2197
> Project: Apache Camel
> Issue Type: New Feature
> Reporter: David Greco
> Fix For: Future
>
> Attachments: patchfile.txt
>
>
> This component (in attachment) provides support for writing data into an Hadoop filesystem.
> You can give a look to the following blog post showing some info about its usage:
> http://www.davidgreco.it/MySite/Blog/Entries/2009/11/17_When_a_Camel_encounters_an_Elephant.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.