You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@syncope.apache.org by Martin van Es <mr...@gmail.com> on 2017/01/23 12:30:14 UTC
CSVDir pull connector challenge
Hi,
Finally, I've taken the time and went ahead (re)installing Syncope to
try and play with 2.0.
First: it's a nice improvement (on the admin interface). Well done!
I've (re) created my test LDAP connector and am able to
provision/activate/enable/disable users and groups/groupMembership
from admin console.
Now I'd like to emulate an authoritative source connector (e.g. HR)
from CSVDir connector. I supply five columns in this file called
id,email,sn,status and delete. I inserted a header line designating
these columns and exactly one test account as 2nd line. Values are
separated by comma's.
I created the connector and resource to follow the columnames/order in
my file, but when I try to setup user provision rules, two thing
surprise me:
I can't select target columns that are designated for key, status and
delete by the connector. Is this by-design?
Second, when I finish the provisioning rules (mapping surname to sn
and email to email, because that's all that's available on target) by
clicking "Save" in the last dialog, Syncope fails with error: "Unable
to find property: 'connObjectKeyValidation'. Locale: null, style:
null"
This is the full error in console.log:
13:23:26.249 ERROR
org.apache.syncope.client.console.panels.AbstractModalPanel - While
creating or updating
org.apache.syncope.common.lib.to.ResourceTO@4e48ade6[
key=CSV local
connector=3e972c1b-d06f-45dc-972c-1bd06f35dc0e
connectorDisplayName=CSV import
provisions=[org.apache.syncope.common.lib.to.ProvisionTO@375fcfeb[
key=<null>
anyType=USER
objectClass=__ACCOUNT__
auxClasses=[]
syncToken=<null>
mapping=org.apache.syncope.common.lib.to.MappingTO@16fab764[
connObjectLink=<null>
items=[org.apache.syncope.common.lib.to.MappingItemTO@5b25b01e[
key=<null>
intAttrName=surname
extAttrName=sn
connObjectKey=false
password=false
mandatoryCondition=false
purpose=PULL
propagationJEXLTransformer=<null>
pullJEXLTransformer=<null>
mappingItemTransformerClassNames=[]
], org.apache.syncope.common.lib.to.MappingItemTO@2dfaeed6[
key=<null>
intAttrName=email
extAttrName=email
connObjectKey=false
password=false
mandatoryCondition=false
purpose=PULL
propagationJEXLTransformer=<null>
pullJEXLTransformer=<null>
mappingItemTransformerClassNames=[]
]]
linkingItems=[]
]
virSchemas=[]
]]
orgUnit=<null>
propagationPriority=0
randomPwdIfNotProvided=true
enforceMandatoryCondition=false
createTraceLevel=ALL
updateTraceLevel=ALL
deleteTraceLevel=ALL
provisioningTraceLevel=ALL
passwordPolicy=<null>
accountPolicy=<null>
pullPolicy=<null>
confOverride=[]
overrideCapabilities=false
capabilitiesOverride=[SYNC]
propagationActionsClassNames=[]
]
java.util.MissingResourceException: Unable to find property:
'connObjectKeyValidation'. Locale: null, style: null
at org.apache.wicket.Localizer.getString(Localizer.java:268)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.model.StringResourceModel.getString(StringResourceModel.java:439)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.model.StringResourceModel.getString(StringResourceModel.java:424)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.syncope.client.console.wizards.resources.ResourceProvisionPanel.onSubmit(ResourceProvisionPanel.java:323)
~[syncope-client-console-2.0.1.jar:2.0.1]
at org.apache.syncope.client.console.wicket.markup.html.bootstrap.dialog.BaseModal$2.onSubmit(BaseModal.java:203)
~[syncope-client-console-2.0.1.jar:2.0.1]
at org.apache.wicket.ajax.markup.html.form.AjaxSubmitLink$1.onSubmit(AjaxSubmitLink.java:111)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.ajax.form.AjaxFormSubmitBehavior$AjaxFormSubmitter.onSubmit(AjaxFormSubmitBehavior.java:215)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.markup.html.form.Form.delegateSubmit(Form.java:1307)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.markup.html.form.Form.process(Form.java:974)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.markup.html.form.Form.onFormSubmitted(Form.java:795)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.ajax.form.AjaxFormSubmitBehavior.onEvent(AjaxFormSubmitBehavior.java:171)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.ajax.AjaxEventBehavior.respond(AjaxEventBehavior.java:155)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.ajax.AbstractDefaultAjaxBehavior.onRequest(AbstractDefaultAjaxBehavior.java:601)
~[wicket-core-7.4.0.jar:7.4.0]
at sun.reflect.GeneratedMethodAccessor471.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_111]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_111]
at org.apache.wicket.RequestListenerInterface.internalInvoke(RequestListenerInterface.java:258)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.RequestListenerInterface.invoke(RequestListenerInterface.java:241)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.core.request.handler.ListenerInterfaceRequestHandler.invokeListener(ListenerInterfaceRequestHandler.java:248)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.core.request.handler.ListenerInterfaceRequestHandler.respond(ListenerInterfaceRequestHandler.java:234)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:895)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.request.RequestHandlerStack.execute(RequestHandlerStack.java:64)
~[wicket-request-7.4.0.jar:7.4.0]
at org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:265)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:222)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:293)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.protocol.ws.AbstractUpgradeFilter.processRequestCycle(AbstractUpgradeFilter.java:70)
~[wicket-native-websocket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:203)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:284)
~[wicket-core-7.4.0.jar:7.4.0]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
~[tomcat8-catalina-8.0.32.jar:8.0.32]
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
~[tomcat8-coyote-8.0.32.jar:8.0.32]
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
~[tomcat8-coyote-8.0.32.jar:8.0.32]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
~[tomcat8-coyote-8.0.32.jar:8.0.32]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
~[tomcat8-coyote-8.0.32.jar:8.0.32]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_111]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
~[tomcat8-util-8.0.32.jar:8.0.32]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
--
If 'but' was any useful, it would be a logic operator
Re: CSVDir pull connector challenge
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 24/01/2017 10:56, Martin van Es wrote:
> On Tue, Jan 24, 2017 at 10:03 AM, Francesco Chicchiricc�<il...@apache.org> wrote:
>>> So, you suggest I turn to Connid now for my functional issues with CSVDir?
>> I would first clarify if there is something wrong ongoing (as suggested
>> above), then possibly report to ConnId.
> I was referring to the required explicit __NAME_ or __UID__ remote key
> mapping to make CSVDir actually work in syncope and/or the absence of
> a selectable key attribute when configuring the mapping.
Ah ok, sure, why not.
Regards.
--
Francesco Chicchiricc�
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: CSVDir pull connector challenge
Posted by Martin van Es <mr...@gmail.com>.
On Tue, Jan 24, 2017 at 10:03 AM, Francesco Chicchiriccò
<il...@apache.org> wrote:
>> So, you suggest I turn to Connid now for my functional issues with CSVDir?
>
>
> I would first clarify if there is something wrong ongoing (as suggested
> above), then possibly report to ConnId.
I was referring to the required explicit __NAME_ or __UID__ remote key
mapping to make CSVDir actually work in syncope and/or the absence of
a selectable key attribute when configuring the mapping.
Best regards,
Martin
Re: CSVDir pull connector challenge
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 23/01/2017 17:46, Martin van Es wrote:
> On Mon, Jan 23, 2017 at 4:36 PM, Francesco Chicchiricc�
> <il...@apache.org> wrote:
>> but essentially, the "mandatory condition" can be specified both at Schema
>> level (hence value(s) must be provided globally) or at mapping level (hence
>> value(s) must be provided when provisioning to / from that external
>> resource).
> Ok, that's clear.
> But that doesn't explain why email wouldn't propagate from my CSVDir
> source into Syncope when the mandatory flag was false?
You need to look at core-connid.log and the propagation task(s)
generated for the given user(s) in order to have a better view of what
is actually happening.
>> Anyway, as commented there, the real issue in only about the failure to
>> report the error message to Admin UI; the rest is about the way how the
>> ConnId CSVDir bundle works, so not any Syncope issue.
> So, you suggest I turn to Connid now for my functional issues with CSVDir?
I would first clarify if there is something wrong ongoing (as suggested
above), then possibly report to ConnId.
Regards.
--
Francesco Chicchiricc�
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: CSVDir pull connector challenge
Posted by Martin van Es <mr...@gmail.com>.
On Mon, Jan 23, 2017 at 4:36 PM, Francesco Chicchiriccò
<il...@apache.org> wrote:
> but essentially, the "mandatory condition" can be specified both at Schema
> level (hence value(s) must be provided globally) or at mapping level (hence
> value(s) must be provided when provisioning to / from that external
> resource).
Ok, that's clear.
But that doesn't explain why email wouldn't propagate from my CSVDir
source into Syncope when the mandatory flag was false?
> Anyway, as commented there, the real issue in only about the failure to
> report the error message to Admin UI; the rest is about the way how the
> ConnId CSVDir bundle works, so not any Syncope issue.
So, you suggest I turn to Connid now for my functional issues with CSVDir?
Regards,
Martin
Re: CSVDir pull connector challenge
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 23/01/2017 15:35, Martin van Es wrote:
> On Mon, Jan 23, 2017 at 1:47 PM, Francesco Chicchiriccò
> <il...@apache.org> wrote:
>>> I can't select target columns that are designated for key, status and
>>> delete by the connector. Is this by-design?
>> I think it is somewhat by design, but I am not sure it is for good; for the
>> moment, please use:
>>
>> * __NAME__ as value for key column
>> * __ENABLE__ as value for status column (you should not need to provide a
>> mapping for this, though, as it is done automatically)
> Well that's contradictory to the error I reported (Unable to find
> property: 'connObjectKeyValidation') but using your hint I am now able
> to harvest accounts from the csv file, thx.
>
> Another thing I noticed: I need to make email attribute mandatory for
> it to appear in the provisioned user, while my assumption was that it
> would provision email when available, but not break on absence? The
> status attribute behaves differently (status false is correctly
> updated to suspended and vice versa) while status -> __ENABLE_
> mandotory field is set to false.
I invite you to read the details from
https://syncope.apache.org/docs/reference-guide.html#mapping
but essentially, the "mandatory condition" can be specified both at
Schema level (hence value(s) must be provided globally) or at mapping
level (hence value(s) must be provided when provisioning to / from that
external resource).
As an example, this simply means that Syncope refuses to send out
propagations to the CSVDir connector if email is not provided and
mapping mandatory condition is set to 'true'.
When the mapping mandatory condition is set to 'false', instead, Syncope
won't raise any error before propagating to the CSVDir connector if
email is not provided.
What happens into the connector, in such case, depends on the connector
bundle implementation.
>> I am able to replicate your error, please file an issue for this.
> https://issues.apache.org/jira/browse/SYNCOPE-1000
>
> (HA! 1000 is mine ;)
Nice catch :-)
Anyway, as commented there, the real issue in only about the failure to
report the error message to Admin UI; the rest is about the way how the
ConnId CSVDir bundle works, so not any Syncope issue.
Regards.
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: CSVDir pull connector challenge
Posted by Martin van Es <mr...@gmail.com>.
On Mon, Jan 23, 2017 at 1:47 PM, Francesco Chicchiriccò
<il...@apache.org> wrote:
>> I can't select target columns that are designated for key, status and
>> delete by the connector. Is this by-design?
>
> I think it is somewhat by design, but I am not sure it is for good; for the
> moment, please use:
>
> * __NAME__ as value for key column
> * __ENABLE__ as value for status column (you should not need to provide a
> mapping for this, though, as it is done automatically)
Well that's contradictory to the error I reported (Unable to find
property: 'connObjectKeyValidation') but using your hint I am now able
to harvest accounts from the csv file, thx.
Another thing I noticed: I need to make email attribute mandatory for
it to appear in the provisioned user, while my assumption was that it
would provision email when available, but not break on absence? The
status attribute behaves differently (status false is correctly
updated to suspended and vice versa) while status -> __ENABLE_
mandotory field is set to false.
> I am able to replicate your error, please file an issue for this.
https://issues.apache.org/jira/browse/SYNCOPE-1000
(HA! 1000 is mine ;)
Best regards,
Martin
Re: CSVDir pull connector challenge
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 23/01/2017 13:30, Martin van Es wrote:
> Hi,
>
> Finally, I've taken the time and went ahead (re)installing Syncope to
> try and play with 2.0.
> First: it's a nice improvement (on the admin interface). Well done!
Thanks! :-)
Also glad for your enduring interest in Apache Syncope.
> I've (re) created my test LDAP connector and am able to
> provision/activate/enable/disable users and groups/groupMembership
> from admin console.
>
> Now I'd like to emulate an authoritative source connector (e.g. HR)
> from CSVDir connector. I supply five columns in this file called
> id,email,sn,status and delete. I inserted a header line designating
> these columns and exactly one test account as 2nd line. Values are
> separated by comma's.
>
> I created the connector and resource to follow the columnames/order in
> my file, but when I try to setup user provision rules, two thing
> surprise me:
>
> I can't select target columns that are designated for key, status and
> delete by the connector. Is this by-design?
As far as I can read from the class implementing the ConnId SCHEMA
operation (e.g. the one that it is used to populate that Admin UI
autocomplete text fields):
https://github.com/Tirasa/ConnIdCSVDirBundle/blob/master/src/main/java/net/tirasa/connid/bundles/csvdir/methods/CSVDirSchema.java#L65
I think it is somewhat by design, but I am not sure it is for good; for
the moment, please use:
* __NAME__ as value for key column
* __ENABLE__ as value for status column (you should not need to provide
a mapping for this, though, as it is done automatically)
The delete column seems to be reserved for internal usage.
> Second, when I finish the provisioning rules (mapping surname to sn
> and email to email, because that's all that's available on target) by
> clicking "Save" in the last dialog, Syncope fails with error: "Unable
> to find property: 'connObjectKeyValidation'. Locale: null, style:
> null"
The message you should get is "There must be exactly one AccountId",
which is anyway bad as 'AccountId' (up to 1_2_X) is now (from 2_0_X)
ConnObjectKey instead.
It complains that there must be exactly one mapping flagged as 'Remote key'.
I am able to replicate your error, please file an issue for this.
Regards.
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/