You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Maciej Lizewski (JIRA)" <ji...@apache.org> on 2012/09/03 17:11:07 UTC
[jira] [Created] (CONNECTORS-518) Support for API authorization and
security tokens
Maciej Lizewski created CONNECTORS-518:
------------------------------------------
Summary: Support for API authorization and security tokens
Key: CONNECTORS-518
URL: https://issues.apache.org/jira/browse/CONNECTORS-518
Project: ManifoldCF
Issue Type: Bug
Components: Wiki connector
Reporter: Maciej Lizewski
Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (CONNECTORS-518) Wiki Connector support for
API authorization and security tokens
Posted by Maciej Liżewski <ma...@gmail.com>.
yes :)
2012/9/10 Karl Wright <da...@gmail.com>
> Ok, I understand - so the maximum number of connections you actually
> need are 2, not 20. ;-)
>
> Karl
>
> On Mon, Sep 10, 2012 at 8:36 AM, Maciej Liżewski
> <ma...@gmail.com> wrote:
> > OK. will try to explain:
> >
> > there is only one HttpClient for Connector instance. But it has tohandle
> > multiple request. During login process it has to handle 2 connections
> > (first try to login and if the server returns TokenRequired answer -
> second
> > connection but the first may not be disconnected that time). So I had to
> > increase setMaxTotalConnections, because oryginally there was a limit
> > setMaxTotalConnections(1),
> > which caused deadlocks in situations described earlier. I think there
> could
> > be "unlimited" option as well.
> >
> > As there is only one instance of HttpClient per Connector instance per
> > session - there are not so many "login" requests - in fact only one per
> > session when the HttpClient is instantiated and is maintained until
> > Connector::disconnect() is called (which kills HttpClient instance).
> >
> > When session expires there is no 40x response unfortunately, but there is
> > error message in payload:
> >
> > <?xml version="1.0"?>
> > <api><error info="You need read permission to use this module"
> > code="readapidenied"/></api>
> >
> > it could be used to handle "session expired" situation, but should be
> > checked on every API request...
> >
> >
> > 2012/9/10 Karl Wright (JIRA) <ji...@apache.org>
> >
> >>
> >> [
> >>
> https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451863#comment-13451863
> ]
> >>
> >> Karl Wright commented on CONNECTORS-518:
> >> ----------------------------------------
> >>
> >> I've had a look at the login functionality. Since you are keeping hold
> of
> >> an HttpClient instance for the entire lifetime of a session, cookies
> should
> >> stick to that instance. This should be OK. However, each individual
> >> HttpClient instance will need to log in itself, so you will have a lot
> of
> >> logins - but perhaps that is OK too.
> >>
> >> I am puzzled, however, why you have the following:
> >>
> >> {code}
> >> connectionManager.getParams().setMaxTotalConnections(20);
> >> {code}
> >>
> >> Since this is local to the connection instance, there should need to be
> >> one connection per instance, so it seems to me that you can leave this
> at 1
> >> and be fine. If you set up a global pool in Commons HttpClient 3.x, it
> >> segregates cookies only by server name, and there may be different
> >> credentials depending on which connection was in play, so we cannot use
> a
> >> global pool even if we wanted to.
> >>
> >> So I think the only other thing we need to fix is explicit support for
> >> session timeout. Am I correct in assuming that you get back a 401 error
> >> when session has timed out on a document fetch? Is there anything in
> the
> >> payload which would indicate session timeout instead of general
> >> inaccessibility?
> >>
> >>
> >> > Wiki Connector support for API authorization and security tokens
> >> > ----------------------------------------------------------------
> >> >
> >> > Key: CONNECTORS-518
> >> > URL:
> >> https://issues.apache.org/jira/browse/CONNECTORS-518
> >> > Project: ManifoldCF
> >> > Issue Type: Improvement
> >> > Components: Wiki connector
> >> > Affects Versions: ManifoldCF 0.6
> >> > Reporter: Maciej Lizewski
> >> > Assignee: Karl Wright
> >> > Fix For: ManifoldCF next
> >> >
> >> > Attachments: WikiConnector.java, wiki.zip
> >> >
> >> >
> >> > Wiki connector does not support API with restricted access (there is
> >> "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> >> > There is no "security" tab for forced authorization tokens or any
> other
> >> security implemented. There should be at least forced tokens tab or
> tokens
> >> assigned to wiki namespaces (second would be better)
> >>
> >> --
> >> This message is automatically generated by JIRA.
> >> If you think it was sent incorrectly, please contact your JIRA
> >> administrators
> >> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
> >>
>
Re: [jira] [Commented] (CONNECTORS-518) Wiki Connector support for
API authorization and security tokens
Posted by Karl Wright <da...@gmail.com>.
Ok, I understand - so the maximum number of connections you actually
need are 2, not 20. ;-)
Karl
On Mon, Sep 10, 2012 at 8:36 AM, Maciej Liżewski
<ma...@gmail.com> wrote:
> OK. will try to explain:
>
> there is only one HttpClient for Connector instance. But it has tohandle
> multiple request. During login process it has to handle 2 connections
> (first try to login and if the server returns TokenRequired answer - second
> connection but the first may not be disconnected that time). So I had to
> increase setMaxTotalConnections, because oryginally there was a limit
> setMaxTotalConnections(1),
> which caused deadlocks in situations described earlier. I think there could
> be "unlimited" option as well.
>
> As there is only one instance of HttpClient per Connector instance per
> session - there are not so many "login" requests - in fact only one per
> session when the HttpClient is instantiated and is maintained until
> Connector::disconnect() is called (which kills HttpClient instance).
>
> When session expires there is no 40x response unfortunately, but there is
> error message in payload:
>
> <?xml version="1.0"?>
> <api><error info="You need read permission to use this module"
> code="readapidenied"/></api>
>
> it could be used to handle "session expired" situation, but should be
> checked on every API request...
>
>
> 2012/9/10 Karl Wright (JIRA) <ji...@apache.org>
>
>>
>> [
>> https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451863#comment-13451863]
>>
>> Karl Wright commented on CONNECTORS-518:
>> ----------------------------------------
>>
>> I've had a look at the login functionality. Since you are keeping hold of
>> an HttpClient instance for the entire lifetime of a session, cookies should
>> stick to that instance. This should be OK. However, each individual
>> HttpClient instance will need to log in itself, so you will have a lot of
>> logins - but perhaps that is OK too.
>>
>> I am puzzled, however, why you have the following:
>>
>> {code}
>> connectionManager.getParams().setMaxTotalConnections(20);
>> {code}
>>
>> Since this is local to the connection instance, there should need to be
>> one connection per instance, so it seems to me that you can leave this at 1
>> and be fine. If you set up a global pool in Commons HttpClient 3.x, it
>> segregates cookies only by server name, and there may be different
>> credentials depending on which connection was in play, so we cannot use a
>> global pool even if we wanted to.
>>
>> So I think the only other thing we need to fix is explicit support for
>> session timeout. Am I correct in assuming that you get back a 401 error
>> when session has timed out on a document fetch? Is there anything in the
>> payload which would indicate session timeout instead of general
>> inaccessibility?
>>
>>
>> > Wiki Connector support for API authorization and security tokens
>> > ----------------------------------------------------------------
>> >
>> > Key: CONNECTORS-518
>> > URL:
>> https://issues.apache.org/jira/browse/CONNECTORS-518
>> > Project: ManifoldCF
>> > Issue Type: Improvement
>> > Components: Wiki connector
>> > Affects Versions: ManifoldCF 0.6
>> > Reporter: Maciej Lizewski
>> > Assignee: Karl Wright
>> > Fix For: ManifoldCF next
>> >
>> > Attachments: WikiConnector.java, wiki.zip
>> >
>> >
>> > Wiki connector does not support API with restricted access (there is
>> "login" method in API: https://www.mediawiki.org/wiki/API:Login)
>> > There is no "security" tab for forced authorization tokens or any other
>> security implemented. There should be at least forced tokens tab or tokens
>> assigned to wiki namespaces (second would be better)
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA
>> administrators
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
Re: [jira] [Commented] (CONNECTORS-518) Wiki Connector support for
API authorization and security tokens
Posted by Maciej Liżewski <ma...@gmail.com>.
OK. will try to explain:
there is only one HttpClient for Connector instance. But it has tohandle
multiple request. During login process it has to handle 2 connections
(first try to login and if the server returns TokenRequired answer - second
connection but the first may not be disconnected that time). So I had to
increase setMaxTotalConnections, because oryginally there was a limit
setMaxTotalConnections(1),
which caused deadlocks in situations described earlier. I think there could
be "unlimited" option as well.
As there is only one instance of HttpClient per Connector instance per
session - there are not so many "login" requests - in fact only one per
session when the HttpClient is instantiated and is maintained until
Connector::disconnect() is called (which kills HttpClient instance).
When session expires there is no 40x response unfortunately, but there is
error message in payload:
<?xml version="1.0"?>
<api><error info="You need read permission to use this module"
code="readapidenied"/></api>
it could be used to handle "session expired" situation, but should be
checked on every API request...
2012/9/10 Karl Wright (JIRA) <ji...@apache.org>
>
> [
> https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451863#comment-13451863]
>
> Karl Wright commented on CONNECTORS-518:
> ----------------------------------------
>
> I've had a look at the login functionality. Since you are keeping hold of
> an HttpClient instance for the entire lifetime of a session, cookies should
> stick to that instance. This should be OK. However, each individual
> HttpClient instance will need to log in itself, so you will have a lot of
> logins - but perhaps that is OK too.
>
> I am puzzled, however, why you have the following:
>
> {code}
> connectionManager.getParams().setMaxTotalConnections(20);
> {code}
>
> Since this is local to the connection instance, there should need to be
> one connection per instance, so it seems to me that you can leave this at 1
> and be fine. If you set up a global pool in Commons HttpClient 3.x, it
> segregates cookies only by server name, and there may be different
> credentials depending on which connection was in play, so we cannot use a
> global pool even if we wanted to.
>
> So I think the only other thing we need to fix is explicit support for
> session timeout. Am I correct in assuming that you get back a 401 error
> when session has timed out on a document fetch? Is there anything in the
> payload which would indicate session timeout instead of general
> inaccessibility?
>
>
> > Wiki Connector support for API authorization and security tokens
> > ----------------------------------------------------------------
> >
> > Key: CONNECTORS-518
> > URL:
> https://issues.apache.org/jira/browse/CONNECTORS-518
> > Project: ManifoldCF
> > Issue Type: Improvement
> > Components: Wiki connector
> > Affects Versions: ManifoldCF 0.6
> > Reporter: Maciej Lizewski
> > Assignee: Karl Wright
> > Fix For: ManifoldCF next
> >
> > Attachments: WikiConnector.java, wiki.zip
> >
> >
> > Wiki connector does not support API with restricted access (there is
> "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> > There is no "security" tab for forced authorization tokens or any other
> security implemented. There should be at least forced tokens tab or tokens
> assigned to wiki namespaces (second would be better)
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
[jira] [Commented] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449651#comment-13449651 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
I've created a branch - branches/CONNECTORS-518 - to work on integrating this contribution. Once the contribution is reformatted to match Apache standards I will commit it to this branch.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452573#comment-13452573 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
I revamped the code so that one request happens after the other, and lower the number of connections available back to 1. I also put in code that detects readapidenied errors and attempts a login if that is seen.
I need to still do something about the possibility of infinite login loops - will tackle that next.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453896#comment-13453896 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Thanks for the eval - I hope to correct these issues when I have a moment, later today perhaps.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CONNECTORS-518) Wiki Connector support for
API authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454045#comment-13454045 ]
Maciej Lizewski edited comment on CONNECTORS-518 at 9/13/12 1:57 AM:
---------------------------------------------------------------------
Here comes call stack.
"Startup thread"
java.lang.Object.wait(Object.java)
java.lang.Thread.join(Thread.java:1186)
java.lang.Thread.join(Thread.java:1239)
org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:673)
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:709)
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
org.apache.manifoldcf.core.database.DBInterfaceDerby.performModification(DBInterfaceDerby.java:833)
org.apache.manifoldcf.core.database.DBInterfaceDerby.performUpdate(DBInterfaceDerby.java:279)
org.apache.manifoldcf.core.database.BaseTable.performUpdate(BaseTable.java:80)
org.apache.manifoldcf.crawler.jobs.JobQueue.updateExistingRecordInitial(JobQueue.java:1078)
org.apache.manifoldcf.crawler.jobs.JobManager.addDocumentsInitial(JobManager.java:3502)
org.apache.manifoldcf.crawler.system.SeedingActivity.writeSeedDocuments(SeedingActivity.java:232)
org.apache.manifoldcf.crawler.system.SeedingActivity.addSeedDocument(SeedingActivity.java:94)
org.apache.manifoldcf.crawler.system.SeedingActivity.addSeedDocument(SeedingActivity.java:122)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.executeListPagesViaThread(WikiConnector.java:2003)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.listAllPages(WikiConnector.java:1971)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.addSeedDocuments(WikiConnector.java:705)
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.addSeedDocuments(BaseRepositoryConnector.java:150)
org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:179)
it seems that in listAllPages method:
String newLastTitle = executeListPagesViaThread(lastTitle,namespace,prefix,activities);
if (newLastTitle == null)
break;
newLastTitle is never null... possible cause:
ExecuteListPagesThread:run method:
StringBuilder lastPageTitleBuffer = new StringBuilder();
loginNeeded = parseListPagesResponse(is,pageBuffer,startPageTitle,lastPageTitleBuffer);
lastPageTitle = lastPageTitleBuffer.toString();
if there is no queryContinue part in API response, lastPageTitleBuffer.toString() returns empty string instead of null.
was (Author: redguy666):
Here comes call stack.
"Startup thread"
java.lang.Object.wait(Object.java)
java.lang.Thread.join(Thread.java:1186)
java.lang.Thread.join(Thread.java:1239)
org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:673)
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:709)
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
org.apache.manifoldcf.core.database.DBInterfaceDerby.performModification(DBInterfaceDerby.java:833)
org.apache.manifoldcf.core.database.DBInterfaceDerby.performUpdate(DBInterfaceDerby.java:279)
org.apache.manifoldcf.core.database.BaseTable.performUpdate(BaseTable.java:80)
org.apache.manifoldcf.crawler.jobs.JobQueue.updateExistingRecordInitial(JobQueue.java:1078)
org.apache.manifoldcf.crawler.jobs.JobManager.addDocumentsInitial(JobManager.java:3502)
org.apache.manifoldcf.crawler.system.SeedingActivity.writeSeedDocuments(SeedingActivity.java:232)
org.apache.manifoldcf.crawler.system.SeedingActivity.addSeedDocument(SeedingActivity.java:94)
org.apache.manifoldcf.crawler.system.SeedingActivity.addSeedDocument(SeedingActivity.java:122)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.executeListPagesViaThread(WikiConnector.java:2003)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.listAllPages(WikiConnector.java:1971)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.addSeedDocuments(WikiConnector.java:705)
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.addSeedDocuments(BaseRepositoryConnector.java:150)
org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:179)
it seems that in listAllPages method:
String newLastTitle = executeListPagesViaThread(lastTitle,namespace,prefix,activities);
if (newLastTitle == null)
break;
newLastTitle is never null...
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454849#comment-13454849 ]
Maciej Lizewski commented on CONNECTORS-518:
--------------------------------------------
Nice! did some checks on my own and I can confirm that everything works fine.
For future enhacement - on namespaces tab display user friendly name instead of namespace id (in chosen namespaces list) :)
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.7
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454051#comment-13454051 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Checked in a fix: r1383994
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453986#comment-13453986 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Checked in a fix for the login problem.
But now I'm looking at the "infinite loop" problem. It seems to me that there are two possible explanations and I need to distinguish between the two. Can you do the following:
- Run the job
- When it gets stuck in the "loop", can you generate a thread dump
- Attach the thread dump here
The way the list-documents code is supposed to work is that it keeps track of the last title it has seen, and requests only documents that are greater than that title on the next request. So the "apfrom" argument should keep changing, as should the documents being returned. It should, however, complete and start crawling after that. It is possible that the actual document fetches are not working, and that is why it appears to "hang". If so, you would see entries in the Simple History that would show documents being tried and retried.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453858#comment-13453858 ]
Maciej Lizewski commented on CONNECTORS-518:
--------------------------------------------
There is one problem (when I look at your changes).
According to http://www.mediawiki.org/wiki/API:Login. Token confiramtion (two step login) does not apply to all cases:
Confirm token
If the response to the above query was Success instead of NeedToken, you can skip this step. (This extra step was added in MediaWiki 1.15.3.) In MediaWiki 1.15.4, first phase of login in ApiLogin.php is broken, so login/sessionid parameter is not returned, thus token confirmation is impossible. Apply ApiLogin.php file from MediaWiki 1.15.5 to your installation as a quick workaround while you plan your upgrade to 1.15.5. ApiLogin.php from MediaWiki 1.16+ is incompatible with MediaWiki 1.15.3+.
so forcing Connector to throw exception when first step does not return token is not always right...
there is also something wrong with getting documents to index. seems like it is hanging in infinite loop when fetching document list:
192.168.1.1 - - [12/Sep/2012:11:19:57 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=Tutorial+Install+%28CRM+%2FIntranet%29&aplimit=500 HTTP/1.1" 200 5944 "-" "Jakarta Commons-HttpClient/3.1"
192.168.1.1 - - [12/Sep/2012:11:19:57 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=%C2%8Crodowisko+Developerskie+VirtualBox&aplimit=500 HTTP/1.1" 200 445 "-" "Jakarta Commons-HttpClient/3.1"
192.168.1.1 - - [12/Sep/2012:11:19:57 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=&aplimit=500 HTTP/1.1" 200 29174 "-" "Jakarta Commons-HttpClient/3.1"
192.168.1.1 - - [12/Sep/2012:11:19:59 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=Korzystanie+z+komputerow&aplimit=500 HTTP/1.1" 200 28682 "-" "Jakarta Commons-HttpClient/3.1"
and again...
192.168.1.1 - - [12/Sep/2012:11:20:00 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=Tutorial+Install+%28CRM+%2FIntranet%29&aplimit=500 HTTP/1.1" 200 5944 "-" "Jakarta Commons-HttpClient/3.1"
192.168.1.1 - - [12/Sep/2012:11:20:00 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=%C2%8Crodowisko+Developerskie+VirtualBox&aplimit=500 HTTP/1.1" 200 445 "-" "Jakarta Commons-HttpClient/3.1"
192.168.1.1 - - [12/Sep/2012:11:20:00 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=&aplimit=500 HTTP/1.1" 200 29174 "-" "Jakarta Commons-HttpClient/3.1"
192.168.1.1 - - [12/Sep/2012:11:20:02 +0200] "GET /api.php?format=xml&action=query&list=allpages&apfrom=Korzystanie+z+komputerow&aplimit=500 HTTP/1.1" 200 28682 "-" "Jakarta Commons-HttpClient/3.1"
ans so on... the job in manifold panel shows 'Starting up', and Documents and Active counts reach proper number and does not change. So it seems that logging in and fetching list works but the process does not advance to index documents.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451863#comment-13451863 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
I've had a look at the login functionality. Since you are keeping hold of an HttpClient instance for the entire lifetime of a session, cookies should stick to that instance. This should be OK. However, each individual HttpClient instance will need to log in itself, so you will have a lot of logins - but perhaps that is OK too.
I am puzzled, however, why you have the following:
{code}
connectionManager.getParams().setMaxTotalConnections(20);
{code}
Since this is local to the connection instance, there should need to be one connection per instance, so it seems to me that you can leave this at 1 and be fine. If you set up a global pool in Commons HttpClient 3.x, it segregates cookies only by server name, and there may be different credentials depending on which connection was in play, so we cannot use a global pool even if we wanted to.
So I think the only other thing we need to fix is explicit support for session timeout. Am I correct in assuming that you get back a 401 error when session has timed out on a document fetch? Is there anything in the payload which would indicate session timeout instead of general inaccessibility?
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448722#comment-13448722 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Hi Maciej,
The Web Connector avoids this problem by keeping its session cookies in a database table. You could use the same approach for the wiki connector, if you need session management, so I'd definitely have a look at that code. The alternative is a global in-memory structure, as you have currently implemented, but you have to be careful that there is no possibility of cross-pollution of session cookies between different connections, and I am not sure there is a good mechanism in HttpClient to achieve that.
You will obviously also need to handle the case of session expiration properly, when a content fetch fails because a login session has expired, and relogin is necessary.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-518) Support for API authorization and
security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright updated CONNECTORS-518:
-----------------------------------
Issue Type: Improvement (was: Bug)
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-518) Support for API authorization and
security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maciej Lizewski updated CONNECTORS-518:
---------------------------------------
Attachment: wiki.zip
Attached file with classes to handle security tokens and API authentication.
Unfortunatelly files were formatted by my IDE, so merging will require to re-format them (or use them as they are)...
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450530#comment-13450530 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Ok, I hand merged the code from your contribution and checked it into the branch. If you have time please make sure it still works.
I can see a number of issues that will need resolution before I'd be happy to see the branch merged to trunk, but we can work on that over a bit more time. Having the session expiration involve forcing a retry because of an exception will show up in the history, for one thing, and will cause the job to stall for minutes at a time. I am sure we can do better.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450495#comment-13450495 ]
Maciej Lizewski commented on CONNECTORS-518:
--------------------------------------------
unfortunatelly it is NetBeans. I know that it makes things harder, but when I realised it happens it was too late to roll back because I would loose all my changes or have to put it line-by-line again... I have tried to format according to your hints but the previous WikiConnector.java does not follow them neither so it is impossible to diff those files anyway. Maybe if you could reformat both: previous version withous any changes and compare it to reformatted my class you would have clear diff...
as for cookies and session - every HttpClient has its own container (I am sure, because when there was separate HttpClient for every request session and cookies were not maintained at all).
there is of course unresolved problem with session expiration, but when it expires 'processDocuments' should throw an error because returned xml will not match expected result. Then reconnection should take place and session will be recreated (but I haven't tested it yet).
Unfortunately our priority now is to achieve running system in our environment ASAP so we make quick changes that adopt connectors to our needs. If the community can use those changes - it is ok, but we can polish those modules only after everything will work for us.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453927#comment-13453927 ]
Maciej Lizewski commented on CONNECTORS-518:
--------------------------------------------
During further tests I did notice such thing:
Connector uses 'canonical' attribute from namespaces list to identify namespace when fetching pages list. I do not know it worked for someone, but for custom namespaces it does not work (at least for me). According to mediawiki API docs there should be used namespace 'id' (https://www.mediawiki.org/wiki/API:Allpages). So I prepared patch for connector to use 'id' instead of 'canonical'.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453932#comment-13453932 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Checked the patch into the CONNECTORS-518 branch: r1383901.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452019#comment-13452019 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Ah, I see why two connections are needed. The code initiates a second request while parsing the response from the first request. This should probably be reworked so that the first request completes (and returns the token) before the second request is kicked off.
I'll look into cleaning this up in the branch over the next day or two.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright reassigned CONNECTORS-518:
--------------------------------------
Assignee: Karl Wright
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright resolved CONNECTORS-518.
------------------------------------
Resolution: Fixed
Fix Version/s: (was: ManifoldCF next)
ManifoldCF 0.7
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF 0.7
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454202#comment-13454202 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Merged the branch into trunk, after confirming that all tests now pass: r1384073
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maciej Lizewski updated CONNECTORS-518:
---------------------------------------
Attachment: WikiConnector.java.namespace.id.patch
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448712#comment-13448712 ]
Maciej Lizewski commented on CONNECTORS-518:
--------------------------------------------
ok... I just did some changes in Wiki connector.
Moved 'security tag' code snippets from WebCrawler to Wiki and added configuration for login, password, domain to API interface.
I will try to provide patches for those changes but I am not sure if they are fully thread-safe, as I had to keep global HttpClient object to properly handle session cookies. Anyway - extensions worked for me in first tests I just did.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453260#comment-13453260 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Added code that limits the number of login attempts to one per API request.
So I think the rework of this code is complete. If you care to try it on a Wiki where you have protected access, I'd be curious to know if I broke anything. I'd also be happy to merge this back to trunk once it is confirmed that everything still works for both the protected access case and the unprotected access situation.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright updated CONNECTORS-518:
-----------------------------------
Summary: Wiki Connector support for API authorization and security tokens (was: Support for API authorization and security tokens)
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451931#comment-13451931 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Per dev@ conversation with Maciej:
(1) The login process needs an extra connection, which is why the pool has more than 1. It probably really only needs 2 though.
(2) The response on session timeout is a 200 code, plus this:
{code}
<?xml version="1.0"?>
<api><error info="You need read permission to use this module"
code="readapidenied"/></api>
{code}
We'd have to look for this on every API response, which should not be a problem, and conditionally relogin if we see that.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454045#comment-13454045 ]
Maciej Lizewski commented on CONNECTORS-518:
--------------------------------------------
Here comes call stack.
"Startup thread"
java.lang.Object.wait(Object.java)
java.lang.Thread.join(Thread.java:1186)
java.lang.Thread.join(Thread.java:1239)
org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:673)
org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:709)
org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1394)
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:186)
org.apache.manifoldcf.core.database.DBInterfaceDerby.performModification(DBInterfaceDerby.java:833)
org.apache.manifoldcf.core.database.DBInterfaceDerby.performUpdate(DBInterfaceDerby.java:279)
org.apache.manifoldcf.core.database.BaseTable.performUpdate(BaseTable.java:80)
org.apache.manifoldcf.crawler.jobs.JobQueue.updateExistingRecordInitial(JobQueue.java:1078)
org.apache.manifoldcf.crawler.jobs.JobManager.addDocumentsInitial(JobManager.java:3502)
org.apache.manifoldcf.crawler.system.SeedingActivity.writeSeedDocuments(SeedingActivity.java:232)
org.apache.manifoldcf.crawler.system.SeedingActivity.addSeedDocument(SeedingActivity.java:94)
org.apache.manifoldcf.crawler.system.SeedingActivity.addSeedDocument(SeedingActivity.java:122)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.executeListPagesViaThread(WikiConnector.java:2003)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.listAllPages(WikiConnector.java:1971)
org.apache.manifoldcf.crawler.connectors.wiki.WikiConnector.addSeedDocuments(WikiConnector.java:705)
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.addSeedDocuments(BaseRepositoryConnector.java:150)
org.apache.manifoldcf.crawler.system.StartupThread.run(StartupThread.java:179)
it seems that in listAllPages method:
String newLastTitle = executeListPagesViaThread(lastTitle,namespace,prefix,activities);
if (newLastTitle == null)
break;
newLastTitle is never null...
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CONNECTORS-518) Support for API
authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450495#comment-13450495 ]
Maciej Lizewski edited comment on CONNECTORS-518 at 9/7/12 8:54 PM:
--------------------------------------------------------------------
unfortunatelly it is NetBeans. I know that it makes things harder, but when I realised it happened it was too late to roll back because I would loose all my changes or have to put it line-by-line again... I have tried to format according to your hints but the previous WikiConnector.java does not follow them neither so it is impossible to diff those files anyway. Maybe if you could reformat both: previous version withous any changes and compare it to reformatted my class you would have clear diff...
as for cookies and session - every HttpClient has its own container (I am sure, because when there was separate HttpClient for every request session and cookies were not maintained at all).
there is of course unresolved problem with session expiration, but when it expires 'processDocuments' should throw an error because returned xml will not match expected result. Then reconnection should take place and session will be recreated (but I haven't tested it yet).
Unfortunately our priority now is to achieve running system in our environment ASAP so we make quick changes that adopt connectors to our needs. If the community can use those changes - it is ok, but we can polish those modules only after everything will work for us.
was (Author: redguy666):
unfortunatelly it is NetBeans. I know that it makes things harder, but when I realised it happens it was too late to roll back because I would loose all my changes or have to put it line-by-line again... I have tried to format according to your hints but the previous WikiConnector.java does not follow them neither so it is impossible to diff those files anyway. Maybe if you could reformat both: previous version withous any changes and compare it to reformatted my class you would have clear diff...
as for cookies and session - every HttpClient has its own container (I am sure, because when there was separate HttpClient for every request session and cookies were not maintained at all).
there is of course unresolved problem with session expiration, but when it expires 'processDocuments' should throw an error because returned xml will not match expected result. Then reconnection should take place and session will be recreated (but I haven't tested it yet).
Unfortunately our priority now is to achieve running system in our environment ASAP so we make quick changes that adopt connectors to our needs. If the community can use those changes - it is ok, but we can polish those modules only after everything will work for us.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-518) Support for API authorization and
security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wright updated CONNECTORS-518:
-----------------------------------
Affects Version/s: ManifoldCF 0.6
Fix Version/s: ManifoldCF next
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Support for API authorization
and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449642#comment-13449642 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Ok - but I can't even read the diffs for the reason you state, so I can't know if it is OK to commit.
Apache formatting rules are as follows:
- column width is 2 characters
- spaces, not tabs
The formatting of { and } in the Apache world are as you have it, although in the ManifoldCF project we also occasionally put the { on a line by itself.
If you can reprocess the changed files with your IDE set to the Apache standards, that should get us a long way towards a readable diff. If this is Eclipse (I bet it is) there is an Apache configuration file available somewhere online even. But in my view, ideally your IDE should be configured for Apache work to not reformat anything automatically - it makes it very hard to collaborate with others who may be using different tools.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-518) Wiki Connector support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454047#comment-13454047 ]
Karl Wright commented on CONNECTORS-518:
----------------------------------------
Ah, that makes sense - now I know how I broke this. I'll fix it.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CONNECTORS-518) Wiki Connector support for
API authorization and security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453927#comment-13453927 ]
Maciej Lizewski edited comment on CONNECTORS-518 at 9/12/12 11:08 PM:
----------------------------------------------------------------------
During further tests I did notice such thing:
Connector uses 'canonical' attribute from namespaces list to identify namespace when fetching pages list. I do not know it worked for someone, but for custom namespaces it does not work (at least for me). According to mediawiki API docs there should be used namespace 'id' (https://www.mediawiki.org/wiki/API:Allpages). So I prepared patch for connector to use 'id' instead of 'canonical'.
Using 'canonical' for list pages query causes:
<api>
<error code="apunknown_apnamespace" info="Unrecognized value for parameter 'apnamespace': CanonicalNamespace"/>
</api>
was (Author: redguy666):
During further tests I did notice such thing:
Connector uses 'canonical' attribute from namespaces list to identify namespace when fetching pages list. I do not know it worked for someone, but for custom namespaces it does not work (at least for me). According to mediawiki API docs there should be used namespace 'id' (https://www.mediawiki.org/wiki/API:Allpages). So I prepared patch for connector to use 'id' instead of 'canonical'.
> Wiki Connector support for API authorization and security tokens
> ----------------------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, WikiConnector.java.namespace.id.patch, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CONNECTORS-518) Support for API
authorization and security tokens
Posted by "Karl Wright (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448722#comment-13448722 ]
Karl Wright edited comment on CONNECTORS-518 at 9/6/12 12:37 AM:
-----------------------------------------------------------------
Hi Maciej,
The Web Connector avoids this problem by keeping its session cookies in a database table. You could use the same approach for the wiki connector, if you need session management, so I'd definitely have a look at that code. The alternative is a global in-memory structure, as you have currently implemented, but you have to be careful that there is no possibility of cross-pollution of session cookies between different connections, and I am not sure there is a good mechanism in HttpClient to achieve that. So you may want to create and maintain your own in-memory global object that has the right characteristics for session management within ManifoldCF.
You will obviously also need to handle the case of session expiration properly, when a content fetch fails because a login session has expired, and relogin is necessary.
was (Author: kwright@metacarta.com):
Hi Maciej,
The Web Connector avoids this problem by keeping its session cookies in a database table. You could use the same approach for the wiki connector, if you need session management, so I'd definitely have a look at that code. The alternative is a global in-memory structure, as you have currently implemented, but you have to be careful that there is no possibility of cross-pollution of session cookies between different connections, and I am not sure there is a good mechanism in HttpClient to achieve that.
You will obviously also need to handle the case of session expiration properly, when a content fetch fails because a login session has expired, and relogin is necessary.
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-518) Support for API authorization and
security tokens
Posted by "Maciej Lizewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CONNECTORS-518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maciej Lizewski updated CONNECTORS-518:
---------------------------------------
Attachment: WikiConnector.java
> Support for API authorization and security tokens
> -------------------------------------------------
>
> Key: CONNECTORS-518
> URL: https://issues.apache.org/jira/browse/CONNECTORS-518
> Project: ManifoldCF
> Issue Type: Bug
> Components: Wiki connector
> Affects Versions: ManifoldCF 0.6
> Reporter: Maciej Lizewski
> Assignee: Karl Wright
> Fix For: ManifoldCF next
>
> Attachments: WikiConnector.java, wiki.zip
>
>
> Wiki connector does not support API with restricted access (there is "login" method in API: https://www.mediawiki.org/wiki/API:Login)
> There is no "security" tab for forced authorization tokens or any other security implemented. There should be at least forced tokens tab or tokens assigned to wiki namespaces (second would be better)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira