You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Karl Wright <da...@gmail.com> on 2011/05/02 09:04:27 UTC
Re: Connector Transaction Data
Oh, but you might glean something from the Chapter 9 example, with is
still under development but may have enough stuff in it to be
interesting.
http://manifoldcfinaction.googlecode.com/svn/trunk/edition_1/output_connector_example/src/org/apache/manifoldcf/agents/output/docs4u/Docs4UOutputConnector.java
Karl
On Sat, Apr 30, 2011 at 3:12 AM, daddywri@gmail.com <da...@gmail.com> wrote:
> Chapter 9 of manifoldcf in action may help here but it is not yet available. i am happy to review your code though.
>
> Karl
>
> Sent from my Nokia phone
> -----Original Message-----
> From: hokie@farzad.net
> Sent: 29/04/2011, 3:24 PM
> To: Connectors dev
> Subject: Connector Transaction Data
>
>
> I'm connecting Manifold to another repository, much like Solr, to
> process the files. I have to pass a session id that I get after logon.
> I used db tables to store persistent data, but the session id is
> pertenent to the connector trying to insert the data. Any thoughts on
> the correct design pattern? Should the connector implement a pool
> concept too? I read the page about writing output connectors, saw the
> warning about not stepping out of bounds :) so I'm asking.
>
> Thanks,
> Farzad.
>
Re: Connector Transaction Data
Posted by Karl Wright <da...@gmail.com>.
Thread contexts should not be linked cross-thread. For managing
sessions, it seems to me that there are two possible models, as
described below:
(1) For any given configuration, there's ONE session ID shared among
all connector class instances which share that configuration.
OR
(2) Each connector class instance has its own session ID.
If you want to code (1), then my recommendation is to simply create a
class that maps a ConfigParams object to a session ID. Make sure the
methods are all synchronized. Then, include an instance of that
mapping class as a static member variable in your connector class.
If you want to code (2), then just have your session ID be a member
variable of the connector class, and set up the session on demand.
I don't understand why you'd need to do anything more complicated than
that, but then I may not understand your problem completely.
Karl
On Mon, May 2, 2011 at 8:09 PM, <ho...@farzad.net> wrote:
> Last time I was working hard at this, it seems that I added the ability to
> store data in the threadContext object. See code below. Can't remember if
> this is the correct way, or I figured a work around? : ) If it is correct,
> it would be right vehicle for the session id too. Thanks!
>
> public void setThreadContext(IThreadContext threadContext) {
> try {
> super.setThreadContext(threadContext);
> } catch (ManifoldCFException e) {
> e.printStackTrace();
> }
> if (threadContext != null) {
> setThreadID();
> }
> }
>
> private void setThreadID() {
> if (currentContext != null) {
> Object id = currentContext.get("id");
> if (id == null) {
> currentContext.save("id", new
> Integer(idNum));
> idNum++;
>
> System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()
> + " new ID assigned [" + currentContext.get("id") + "]");
> } else {
>
> //System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()
> + " already has an id set to [" + currentContext.get("id") + "]");
> }
> }
> }
>
> On Mon, 2 May 2011 03:04:27 -0400, Karl Wright <da...@gmail.com> wrote:
>>
>> Oh, but you might glean something from the Chapter 9 example, with is
>> still under development but may have enough stuff in it to be
>> interesting.
>>
>>
>>
>> http://manifoldcfinaction.googlecode.com/svn/trunk/edition_1/output_connector_example/src/org/apache/manifoldcf/agents/output/docs4u/Docs4UOutputConnector.java
>>
>> Karl
>>
>> On Sat, Apr 30, 2011 at 3:12 AM, daddywri@gmail.com
>> <da...@gmail.com> wrote:
>>>
>>> Chapter 9 of manifoldcf in action may help here but it is not yet
>>> available. i am happy to review your code though.
>>>
>>> Karl
>>>
>>> Sent from my Nokia phone
>>> -----Original Message-----
>>> From: hokie@farzad.net
>>> Sent: 29/04/2011, 3:24 PM
>>> To: Connectors dev
>>> Subject: Connector Transaction Data
>>>
>>>
>>> I'm connecting Manifold to another repository, much like Solr, to
>>> process the files. I have to pass a session id that I get after logon.
>>> I used db tables to store persistent data, but the session id is
>>> pertenent to the connector trying to insert the data. Any thoughts on
>>> the correct design pattern? Should the connector implement a pool
>>> concept too? I read the page about writing output connectors, saw the
>>> warning about not stepping out of bounds :) so I'm asking.
>>>
>>> Thanks,
>>> Farzad.
>>>
>
>
Re: Connector Transaction Data
Posted by ho...@farzad.net.
Last time I was working hard at this, it seems that I added the ability
to store data in the threadContext object. See code below. Can't
remember if this is the correct way, or I figured a work around? : ) If
it is correct, it would be right vehicle for the session id too.
Thanks!
public void setThreadContext(IThreadContext threadContext) {
try {
super.setThreadContext(threadContext);
} catch (ManifoldCFException e) {
e.printStackTrace();
}
if (threadContext != null) {
setThreadID();
}
}
private void setThreadID() {
if (currentContext != null) {
Object id = currentContext.get("id");
if (id == null) {
currentContext.save("id", new Integer(idNum));
idNum++;
System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()
+ " new ID assigned [" + currentContext.get("id") + "]");
} else {
//System.out.println(Thread.currentThread().getStackTrace()[1].getMethodName()
+ " already has an id set to [" + currentContext.get("id") + "]");
}
}
}
On Mon, 2 May 2011 03:04:27 -0400, Karl Wright <da...@gmail.com>
wrote:
> Oh, but you might glean something from the Chapter 9 example, with is
> still under development but may have enough stuff in it to be
> interesting.
>
>
> http://manifoldcfinaction.googlecode.com/svn/trunk/edition_1/output_connector_example/src/org/apache/manifoldcf/agents/output/docs4u/Docs4UOutputConnector.java
>
> Karl
>
> On Sat, Apr 30, 2011 at 3:12 AM, daddywri@gmail.com
> <da...@gmail.com> wrote:
>> Chapter 9 of manifoldcf in action may help here but it is not yet
>> available. i am happy to review your code though.
>>
>> Karl
>>
>> Sent from my Nokia phone
>> -----Original Message-----
>> From: hokie@farzad.net
>> Sent: 29/04/2011, 3:24 PM
>> To: Connectors dev
>> Subject: Connector Transaction Data
>>
>>
>> I'm connecting Manifold to another repository, much like Solr, to
>> process the files. I have to pass a session id that I get after
>> logon.
>> I used db tables to store persistent data, but the session id is
>> pertenent to the connector trying to insert the data. Any thoughts
>> on
>> the correct design pattern? Should the connector implement a pool
>> concept too? I read the page about writing output connectors, saw
>> the
>> warning about not stepping out of bounds :) so I'm asking.
>>
>> Thanks,
>> Farzad.
>>