You are viewing a plain text version of this content. The canonical link for it is here.
Posted to droids-commits@incubator.apache.org by to...@apache.org on 2013/02/08 15:24:12 UTC
svn commit: r1444064 - in
/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core:
ContentEntity.java LinkedTask.java
Author: tobr
Date: Fri Feb 8 15:24:12 2013
New Revision: 1444064
URL: http://svn.apache.org/r1444064
Log:
moved anchortext from linkedtask to content entity
Changed collection type of getTo to LinkedTask
Modified:
incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java
Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java?rev=1444064&r1=1444063&r2=1444064&view=diff
==============================================================================
--- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java (original)
+++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java Fri Feb 8 15:24:12 2013
@@ -27,6 +27,9 @@ import java.util.Set;
* The predefined members are shortcuts to default fields.
* Otherwise you can put your meta data in the Map using your own key.
*
+ * TODO Storing the data in a map does not seem to be the best way.
+ * Maybe we should split it up to separate members?
+ *
* @version 1.0
*/
public class ContentEntity {
@@ -37,6 +40,7 @@ public class ContentEntity {
public final static String CONTENT_TYPE = "content-type";
public final static String CONTENT_LENGTH = "content-length";
public final static String LINKS = "links";
+ public final static String ANCHORTEXT = "hreftitle";
public final static String DEFAULT_CHARSET = "UTF-8";
@@ -193,6 +197,25 @@ public class ContentEntity {
return Long.getLong(this.getValue(CONTENT_LENGTH).toString()).longValue();
}
+
+ /**
+ * Set the title of an link
+ *
+ * @param anchorText the text
+ */
+ public void setAnchorText(String anchorText) {
+ this.put(ANCHORTEXT, anchorText);
+ }
+
+ /**
+ * Get the title of an anchor text
+ *
+ * @return the Anchor text for this link
+ */
+ public String getAnchorText() {
+ return this.getValue(ANCHORTEXT).toString();
+ }
+
/**
* Returns the data map.
*
Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java
URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java?rev=1444064&r1=1444063&r2=1444064&view=diff
==============================================================================
--- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java (original)
+++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java Fri Feb 8 15:24:12 2013
@@ -18,12 +18,8 @@ package org.apache.droids.core;
*/
-import java.net.URI;
import java.util.Collection;
-import org.apache.droids.core.Task;
-
-
/**
* Simple extension of a {@link Task}. Adding from/to link
* to the task object.
@@ -32,11 +28,6 @@ import org.apache.droids.core.Task;
*/
public interface LinkedTask extends Task {
/**
- * @return the Anchor text for this link
- */
- String getAnchorText();
-
- /**
* From where the link was created.
*
* @return the parent link from where the link was coming from
@@ -48,6 +39,6 @@ public interface LinkedTask extends Task
*
* @return the location where the link is pointing to
*/
- Collection<URI> getTo();
+ Collection<LinkedTask> getTo();
}
Re: svn commit: r1444064 - in /incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core:
ContentEntity.java LinkedTask.java
Posted by Tobias Rübner <to...@apache.org>.
Hi Thorsten,
you are right, we should see the ContentEntity as the raw data store.
So I would like to add a new ParserData interface for representing
the data extracted by the parser.
Each parser could come up with there own implementation and so you could
easily define your own extracted data fields based on the parser
implementation.
Having a build bot for both the current trunk and the branch is a great
idea.
I could also maintain the builds at jenkins, but I don't have the rights to
do that.
So if you like, you could also grant access to me.
Another great improvement could be to add droids to the apache git mirrors.
http://git.apache.org/
Maybe that could attract developers to contribute to the project.
Tobias
On Mon, Feb 11, 2013 at 2:33 PM, Thorsten Scherler <sc...@gmail.com>wrote:
> On 02/08/2013 04:24 PM, tobr@apache.org wrote:
> > Author: tobr
> > Date: Fri Feb 8 15:24:12 2013
> > New Revision: 1444064
> >
> > URL: http://svn.apache.org/r1444064
> > Log:
> > moved anchortext from linkedtask to content entity
> > Changed collection type of getTo to LinkedTask
> >
> > Modified:
> >
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
> >
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java
> >
> > Modified:
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
> > URL:
> http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java?rev=1444064&r1=1444063&r2=1444064&view=diff
> >
> ==============================================================================
> > ---
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
> (original)
> > +++
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
> Fri Feb 8 15:24:12 2013
> > @@ -27,6 +27,9 @@ import java.util.Set;
> > * The predefined members are shortcuts to default fields.
> > * Otherwise you can put your meta data in the Map using your own key.
> > *
> > + * TODO Storing the data in a map does not seem to be the best way.
> > + * Maybe we should split it up to separate members?
> >
>
> Well, that is the reason why we had them in the task and not in the
> content entity. I thing that the content entity should be a a simple
> rep/wrapper of the url response. What we are now doing is to use it as
> storage unit but do not really have any reserved keys listed that cannot
> be used or to be more general no logic to control/manage the contentEntity.
>
> I think that the task is the best place to store things that we extract
> in the parser besides the request associated parameters.
>
> BTW we may look into setting up a jenkins/build bot for the branch on
> the asf, for now I am monitoring with our company jenkins the build but
> had not configured well the mail notifications so the above commit slipped.
>
> salu2
>
> --
> Thorsten Scherler <scherler.at.gmail.com>
> codeBusters S.L. - web based systems
> <consulting, training and solutions>
>
> http://www.codebusters.es/
>
>
Re: svn commit: r1444064 - in /incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core:
ContentEntity.java LinkedTask.java
Posted by Thorsten Scherler <sc...@gmail.com>.
On 02/08/2013 04:24 PM, tobr@apache.org wrote:
> Author: tobr
> Date: Fri Feb 8 15:24:12 2013
> New Revision: 1444064
>
> URL: http://svn.apache.org/r1444064
> Log:
> moved anchortext from linkedtask to content entity
> Changed collection type of getTo to LinkedTask
>
> Modified:
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
> incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/LinkedTask.java
>
> Modified: incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java
> URL: http://svn.apache.org/viewvc/incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java?rev=1444064&r1=1444063&r2=1444064&view=diff
> ==============================================================================
> --- incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java (original)
> +++ incubator/droids/branches/0.2.x-cleanup/droids-core/src/main/java/org/apache/droids/core/ContentEntity.java Fri Feb 8 15:24:12 2013
> @@ -27,6 +27,9 @@ import java.util.Set;
> * The predefined members are shortcuts to default fields.
> * Otherwise you can put your meta data in the Map using your own key.
> *
> + * TODO Storing the data in a map does not seem to be the best way.
> + * Maybe we should split it up to separate members?
>
Well, that is the reason why we had them in the task and not in the
content entity. I thing that the content entity should be a a simple
rep/wrapper of the url response. What we are now doing is to use it as
storage unit but do not really have any reserved keys listed that cannot
be used or to be more general no logic to control/manage the contentEntity.
I think that the task is the best place to store things that we extract
in the parser besides the request associated parameters.
BTW we may look into setting up a jenkins/build bot for the branch on
the asf, for now I am monitoring with our company jenkins the build but
had not configured well the mail notifications so the above commit slipped.
salu2
--
Thorsten Scherler <scherler.at.gmail.com>
codeBusters S.L. - web based systems
<consulting, training and solutions>
http://www.codebusters.es/