You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@syncope.apache.org by Rainer Herbst <ra...@aip.de> on 2019/06/28 08:14:09 UTC
CSVDir
Dear all,
for the last 2 weeks I tried to get Syncope running using maven, and at
least I have an instance that works somehow.
But I stumbled over a lot of problems with CSVDir not working. I created a
connector and a ressource, but everytime I try to pull the content of a csv
file, the task fails with error
org.quartz.JobExecutionException: While pulling from connector [See nested
exception:
org.identityconnectors.framework.common.exceptions.ConnectorException:
org.identityconnectors.framework.common.exceptions.ConnectorIOException:
java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException: user
lacks privilege or object not found: USER_EX88887 in statement [SELECT *
FROM USER_EX88887]]
There are some files created in the CSV directory:
ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l /srv/tomcat/data/
total 24
-rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
-rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
-rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
-rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
-rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
What is missing there?
I like the openess and the concepts of Syncope, but, to be honest, I did not
expect to spend 2 days trying to read a simple CSV data file... :-(
Help is very much appreciated!
Rainer Herbst
IT-Service
Phone: +49 331 7499-257
e-mail: rainer.herbst@aip.de
https://www.aip.de
-----------------------------------------------------------------------------------------------
Leibniz-Institut für Astrophysik Potsdam (AIP)
An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
Stiftung bürgerlichen Rechts
Stiftungsverzeichnis Brandenburg: 26 742-00/7026
-----------------------------------------------------------------------------------------------
Re: CSVDir
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 01/07/19 10:50, Rainer Herbst wrote:
> Thanks a lot, that solved the problem!
>
> Or should I say, it is a goot workaround? A solution would be to
> escape the field names, so that the SQL statements will work no matter
> what field names are used? ;-)
Hi Rainer,
glad that Davide was able to help with your troubles.
Feel free to reach up the ConnId mailing list [1] to discuss this, or to
submit a PR [2].
Regards.
[1] https://groups.google.com/forum/#!forum/connid-users
[2] https://github.com/Tirasa/ConnIdCSVDirBundle
> On Fri, 28 Jun 2019 17:08:23 +0200
> Davide Cortellucci <da...@tirasa.net> wrote:
>> I managed to reproduce your error.
>>
>> It looks like the problem is with the "group" column name: since it
>> is one of the SQL reserved words, you should change it in your csv
>> and update the connector accordingly.
>>
>> Best regards
>>
>> Davide
>>
>> On 6/28/19 11:14 AM, Rainer Herbst wrote:
>>> Hi Davide,
>>>
>>>
>>> thank you for the quick response.
>>> Here is the current connector configuration. The ressource
>>> configuration follows below.
>>>
>>> {
>>> "key" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>>> "adminRealm" : "/AIPTest",
>>> "location" : "file:/opt/syncope/bundles/",
>>> "connectorName" : "net.tirasa.connid.bundles.csvdir.CSVDirConnector",
>>> "bundleName" : "net.tirasa.connid.bundles.csvdir",
>>> "version" : "0.8.8",
>>> "displayName" : "SystemDB",
>>> "connRequestTimeout" : 10,
>>> "poolConf" : {
>>> "maxObjects" : 2000,
>>> "minIdle" : 20,
>>> "maxIdle" : 200,
>>> "maxWait" : 3000,
>>> "minEvictableIdleTimeMillis" : 2000
>>> },
>>> "conf" : [ {
>>> "schema" : {
>>> "name" : "sourcePath",
>>> "displayName" : "Source path",
>>> "helpMessage" : "Absolute path of a directory where the CSV files
>>> to be processed are located",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 1,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "/srv/tomcat/data/" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fileMask",
>>> "displayName" : "File mask",
>>> "helpMessage" : "Regular expression describing files to be
>>> processed",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 2,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ".csv" ]
>>> }, {
>>> "schema" : {
>>> "name" : "encoding",
>>> "displayName" : "File encoding",
>>> "helpMessage" : "Basic encoding of the file",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 3,
>>> "confidential" : false,
>>> "defaultValues" : [ "UTF-8" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "UTF-8" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fieldDelimiter",
>>> "displayName" : "Field delimiter",
>>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>> Default is \",\".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 4,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "," ]
>>> }, {
>>> "schema" : {
>>> "name" : "textQualifier",
>>> "displayName" : "Text qualifier",
>>> "helpMessage" : "Delimiter to determine beginning and end of text
>>> in value. Default is \".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 5,
>>> "confidential" : false,
>>> "defaultValues" : [ "\"" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "'" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyColumnNames",
>>> "displayName" : "Key column name",
>>> "helpMessage" : "Name of the column used to identify user uniquely",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 6,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "uidNumber" ]
>>> }, {
>>> "schema" : {
>>> "name" : "passwordColumnName",
>>> "displayName" : "Password column name",
>>> "helpMessage" : "Name of the column used to specify user password",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 7,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : false,
>>> "values" : [ ]
>>> }, {
>>> "schema" : {
>>> "name" : "deleteColumnName",
>>> "displayName" : "Delete column name",
>>> "helpMessage" : "Name of the column used to specify users to be
>>> deleted",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 8,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : false,
>>> "values" : [ ]
>>> }, {
>>> "schema" : {
>>> "name" : "quotationRequired",
>>> "displayName" : "Value quotation required",
>>> "helpMessage" : "Specify if value quotation is required. Default
>>> is true.",
>>> "type" : "java.lang.Boolean",
>>> "required" : false,
>>> "order" : 9,
>>> "confidential" : false,
>>> "defaultValues" : [ true ]
>>> },
>>> "overridable" : false,
>>> "values" : [ true ]
>>> }, {
>>> "schema" : {
>>> "name" : "fields",
>>> "displayName" : "Column names",
>>> "helpMessage" : "Column names",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 10,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "uidNumber" ]
>>> }, {
>>> "schema" : {
>>> "name" : "ignoreHeader",
>>> "displayName" : "Ignore header",
>>> "helpMessage" : "Specify if first line of the file must be
>>> ignored. Default is true.",
>>> "type" : "java.lang.Boolean",
>>> "required" : false,
>>> "order" : 11,
>>> "confidential" : false,
>>> "defaultValues" : [ true ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyseparator",
>>> "displayName" : "Key separator",
>>> "helpMessage" : "Character used to separate keys in a multi-key
>>> scenario. Default is \",\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 12,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ";" ]
>>> }, {
>>> "schema" : {
>>> "name" : "multivalueSeparator",
>>> "displayName" : "Multi value separator",
>>> "helpMessage" : "Character used to separate values in a
>>> multi-value scenario. Multivalue unsupported if not provided.",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 13,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : false,
>>> "values" : [ ]
>>> }, {
>>> "schema" : {
>>> "name" : "defaultStatusValue",
>>> "displayName" : "Default Status Value",
>>> "helpMessage" : "Enter the value for status in case of status not
>>> specified. Default is \"true\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 14,
>>> "confidential" : false,
>>> "defaultValues" : [ "true" ]
>>> },
>>> "overridable" : false,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "disabledStatusValue",
>>> "displayName" : "Disabled Status Value",
>>> "helpMessage" : "Specify a value for disabled status. Default is
>>> \"false\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 15,
>>> "confidential" : false,
>>> "defaultValues" : [ "false" ]
>>> },
>>> "overridable" : false,
>>> "values" : [ "false" ]
>>> }, {
>>> "schema" : {
>>> "name" : "enabledStatusValue",
>>> "displayName" : "Enable Status Value",
>>> "helpMessage" : "Specify a value for enabled status. Default is
>>> \"true\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 16,
>>> "confidential" : false,
>>> "defaultValues" : [ "true" ]
>>> },
>>> "overridable" : false,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "statusColumn",
>>> "displayName" : "Status Column name",
>>> "helpMessage" : "Status column name.",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 17,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ]
>>> } ],
>>> "capabilities" : [ "SEARCH", "CREATE", "UPDATE" ]
>>> }
>>>
>>> Ressource config:
>>> {
>>> "key" : "Users",
>>> "connector" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>>> "connectorDisplayName" : "SystemDB",
>>> "orgUnit" : null,
>>> "propagationPriority" : 2,
>>> "randomPwdIfNotProvided" : false,
>>> "enforceMandatoryCondition" : false,
>>> "createTraceLevel" : "ALL",
>>> "updateTraceLevel" : "ALL",
>>> "deleteTraceLevel" : "ALL",
>>> "provisioningTraceLevel" : "ALL",
>>> "passwordPolicy" : "0b17832c-cdb2-4e92-9783-2ccdb23e92d1",
>>> "accountPolicy" : "319350c1-6809-49ed-9350-c1680929eda1",
>>> "pullPolicy" : "ec43ef48-4ba9-4ef0-83ef-484ba9fef083",
>>> "pushPolicy" : "a66afe6e-7e23-44d4-aafe-6e7e2374d4dc",
>>> "overrideCapabilities" : false,
>>> "provisions" : [ {
>>> "key" : "71e72b11-a3a7-4090-a72b-11a3a780902a",
>>> "anyType" : "USER",
>>> "objectClass" : "__ACCOUNT__",
>>> "syncToken" : null,
>>> "ignoreCaseMatch" : false,
>>> "uidOnCreate" : null,
>>> "mapping" : {
>>> "connObjectLink" : null,
>>> "connObjectKeyItem" : {
>>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>>> "intAttrName" : "numericalUID",
>>> "extAttrName" : "uidNumber",
>>> "connObjectKey" : true,
>>> "password" : false,
>>> "mandatoryCondition" : "true",
>>> "purpose" : "PROPAGATION",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> },
>>> "items" : [ {
>>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>>> "intAttrName" : "numericalUID",
>>> "extAttrName" : "uidNumber",
>>> "connObjectKey" : true,
>>> "password" : false,
>>> "mandatoryCondition" : "true",
>>> "purpose" : "PROPAGATION",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "3ba6be21-82db-4702-a6be-2182dba702b0",
>>> "intAttrName" : "organizationalUnit",
>>> "extAttrName" : "group",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "69557f34-b7cd-4e11-957f-34b7cd1e1167",
>>> "intAttrName" : "creationDate",
>>> "extAttrName" : "createdDatum",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "d55ae9a2-10ca-45c3-9ae9-a210cae5c31f",
>>> "intAttrName" : "displayName",
>>> "extAttrName" : "fullName",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "9ccf8d00-14cb-4085-8f8d-0014cbb08527",
>>> "intAttrName" : "uid",
>>> "extAttrName" : "uid",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "true",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> } ],
>>> "linkingItems" : [ {
>>> "key" : null,
>>> "intAttrName" : "AllUsers",
>>> "extAttrName" : "uid",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> } ]
>>> },
>>> "auxClasses" : [ ],
>>> "virSchemas" : [ "AllUsers" ]
>>> } ],
>>> "confOverride" : [ {
>>> "schema" : {
>>> "name" : "sourcePath",
>>> "displayName" : "Source path",
>>> "helpMessage" : "Absolute path of a directory where the CSV files
>>> to be processed are located",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 1,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "/srv/tomcat/data/" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fileMask",
>>> "displayName" : "File mask",
>>> "helpMessage" : "Regular expression describing files to be
>>> processed",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 2,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "aip-system_users_test.csv" ]
>>> }, {
>>> "schema" : {
>>> "name" : "encoding",
>>> "displayName" : "File encoding",
>>> "helpMessage" : "Basic encoding of the file",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 3,
>>> "confidential" : false,
>>> "defaultValues" : [ "UTF-8" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "UTF-8" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fieldDelimiter",
>>> "displayName" : "Field delimiter",
>>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>> Default is \",\".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 4,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "," ]
>>> }, {
>>> "schema" : {
>>> "name" : "textQualifier",
>>> "displayName" : "Text qualifier",
>>> "helpMessage" : "Delimiter to determine beginning and end of text
>>> in value. Default is \".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 5,
>>> "confidential" : false,
>>> "defaultValues" : [ "\"" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "'" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyColumnNames",
>>> "displayName" : "Key column name",
>>> "helpMessage" : "Name of the column used to identify user uniquely",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 6,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "uidNumber" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fields",
>>> "displayName" : "Column names",
>>> "helpMessage" : "Column names",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 10,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "createdDatum", "createdTime", "uid", "uidNumber",
>>> "gidNumber", "fullName", "office", "phone", "fax", "email",
>>> "homeDir", "host", "group", "shell", "www", "passwd", "termination",
>>> "tutor", "remarks" ]
>>> }, {
>>> "schema" : {
>>> "name" : "ignoreHeader",
>>> "displayName" : "Ignore header",
>>> "helpMessage" : "Specify if first line of the file must be
>>> ignored. Default is true.",
>>> "type" : "java.lang.Boolean",
>>> "required" : false,
>>> "order" : 11,
>>> "confidential" : false,
>>> "defaultValues" : [ true ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyseparator",
>>> "displayName" : "Key separator",
>>> "helpMessage" : "Character used to separate keys in a multi-key
>>> scenario. Default is \",\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 12,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ";" ]
>>> }, {
>>> "schema" : {
>>> "name" : "statusColumn",
>>> "displayName" : "Status Column name",
>>> "helpMessage" : "Status column name.",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 17,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "passwd" ]
>>> } ],
>>> "capabilitiesOverride" : [ "SEARCH", "CREATE", "UPDATE" ],
>>> "propagationActions" : [ ]
>>> }
>>>
>>> Best regards
>>>
>>> On Fri, 28 Jun 2019 10:40:58 +0200
>>> Davide Cortellucci <da...@tirasa.net> wrote:
>>>> Hi Rainer,
>>>>
>>>> can you please share the CSVDir configuration?
>>>>
>>>> Regards,
>>>> Davide
>>>>
>>>> On 6/28/19 10:14 AM, Rainer Herbst wrote:
>>>>> Dear all,
>>>>>
>>>>>
>>>>> for the last 2 weeks I tried to get Syncope running using maven,
>>>>> and at least I have an instance that works somehow.
>>>>>
>>>>> But I stumbled over a lot of problems with CSVDir not working. I
>>>>> created a connector and a ressource, but everytime I try to pull
>>>>> the content of a csv file, the task fails with error
>>>>> org.quartz.JobExecutionException: While pulling from connector
>>>>> [See nested exception:
>>>>> org.identityconnectors.framework.common.exceptions.ConnectorException:
>>>>> org.identityconnectors.framework.common.exceptions.ConnectorIOException:
>>>>> java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
>>>>> user lacks privilege or object not found: USER_EX88887 in
>>>>> statement [SELECT * FROM USER_EX88887]]
>>>>>
>>>>> There are some files created in the CSV directory:
>>>>> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
>>>>> /srv/tomcat/data/
>>>>> total 24
>>>>> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
>>>>> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
>>>>> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
>>>>> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
>>>>> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
>>>>> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>>>>>
>>>>> What is missing there?
>>>>>
>>>>> I like the openess and the concepts of Syncope, but, to be honest,
>>>>> I did not expect to spend 2 days trying to read a simple CSV data
>>>>> file... :-(
>>>>>
>>>>> Help is very much appreciated!
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: CSVDir
Posted by Rainer Herbst <ra...@aip.de>.
Thanks a lot, that solved the problem!
Or should I say, it is a goot workaround? A solution would be to escape the
field names, so that the SQL statements will work no matter what field names
are used? ;-)
On Fri, 28 Jun 2019 17:08:23 +0200
Davide Cortellucci <da...@tirasa.net> wrote:
> I managed to reproduce your error.
>
> It looks like the problem is with the "group" column name: since it
>is one of the SQL reserved words, you should change it in your csv
>and update the connector accordingly.
>
> Best regards
>
> Davide
>
> On 6/28/19 11:14 AM, Rainer Herbst wrote:
>> Hi Davide,
>>
>>
>> thank you for the quick response.
>> Here is the current connector configuration. The ressource
>>configuration follows below.
>>
>> {
>> "key" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>> "adminRealm" : "/AIPTest",
>> "location" : "file:/opt/syncope/bundles/",
>> "connectorName" :
>>"net.tirasa.connid.bundles.csvdir.CSVDirConnector",
>> "bundleName" : "net.tirasa.connid.bundles.csvdir",
>> "version" : "0.8.8",
>> "displayName" : "SystemDB",
>> "connRequestTimeout" : 10,
>> "poolConf" : {
>> "maxObjects" : 2000,
>> "minIdle" : 20,
>> "maxIdle" : 200,
>> "maxWait" : 3000,
>> "minEvictableIdleTimeMillis" : 2000
>> },
>> "conf" : [ {
>> "schema" : {
>> "name" : "sourcePath",
>> "displayName" : "Source path",
>> "helpMessage" : "Absolute path of a directory where the CSV files
>>to be processed are located",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 1,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "/srv/tomcat/data/" ]
>> }, {
>> "schema" : {
>> "name" : "fileMask",
>> "displayName" : "File mask",
>> "helpMessage" : "Regular expression describing files to be
>>processed",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 2,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ ".csv" ]
>> }, {
>> "schema" : {
>> "name" : "encoding",
>> "displayName" : "File encoding",
>> "helpMessage" : "Basic encoding of the file",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 3,
>> "confidential" : false,
>> "defaultValues" : [ "UTF-8" ]
>> },
>> "overridable" : true,
>> "values" : [ "UTF-8" ]
>> }, {
>> "schema" : {
>> "name" : "fieldDelimiter",
>> "displayName" : "Field delimiter",
>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>Default is \",\".",
>> "type" : "char",
>> "required" : false,
>> "order" : 4,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ "," ]
>> }, {
>> "schema" : {
>> "name" : "textQualifier",
>> "displayName" : "Text qualifier",
>> "helpMessage" : "Delimiter to determine beginning and end of text
>>in value. Default is \".",
>> "type" : "char",
>> "required" : false,
>> "order" : 5,
>> "confidential" : false,
>> "defaultValues" : [ "\"" ]
>> },
>> "overridable" : true,
>> "values" : [ "'" ]
>> }, {
>> "schema" : {
>> "name" : "keyColumnNames",
>> "displayName" : "Key column name",
>> "helpMessage" : "Name of the column used to identify user
>>uniquely",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 6,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "uidNumber" ]
>> }, {
>> "schema" : {
>> "name" : "passwordColumnName",
>> "displayName" : "Password column name",
>> "helpMessage" : "Name of the column used to specify user
>>password",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 7,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : false,
>> "values" : [ ]
>> }, {
>> "schema" : {
>> "name" : "deleteColumnName",
>> "displayName" : "Delete column name",
>> "helpMessage" : "Name of the column used to specify users to be
>>deleted",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 8,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : false,
>> "values" : [ ]
>> }, {
>> "schema" : {
>> "name" : "quotationRequired",
>> "displayName" : "Value quotation required",
>> "helpMessage" : "Specify if value quotation is required. Default
>>is true.",
>> "type" : "java.lang.Boolean",
>> "required" : false,
>> "order" : 9,
>> "confidential" : false,
>> "defaultValues" : [ true ]
>> },
>> "overridable" : false,
>> "values" : [ true ]
>> }, {
>> "schema" : {
>> "name" : "fields",
>> "displayName" : "Column names",
>> "helpMessage" : "Column names",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 10,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "uidNumber" ]
>> }, {
>> "schema" : {
>> "name" : "ignoreHeader",
>> "displayName" : "Ignore header",
>> "helpMessage" : "Specify if first line of the file must be
>>ignored. Default is true.",
>> "type" : "java.lang.Boolean",
>> "required" : false,
>> "order" : 11,
>> "confidential" : false,
>> "defaultValues" : [ true ]
>> },
>> "overridable" : true,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "keyseparator",
>> "displayName" : "Key separator",
>> "helpMessage" : "Character used to separate keys in a multi-key
>>scenario. Default is \",\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 12,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ ";" ]
>> }, {
>> "schema" : {
>> "name" : "multivalueSeparator",
>> "displayName" : "Multi value separator",
>> "helpMessage" : "Character used to separate values in a
>>multi-value scenario. Multivalue unsupported if not provided.",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 13,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : false,
>> "values" : [ ]
>> }, {
>> "schema" : {
>> "name" : "defaultStatusValue",
>> "displayName" : "Default Status Value",
>> "helpMessage" : "Enter the value for status in case of status not
>>specified. Default is \"true\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 14,
>> "confidential" : false,
>> "defaultValues" : [ "true" ]
>> },
>> "overridable" : false,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "disabledStatusValue",
>> "displayName" : "Disabled Status Value",
>> "helpMessage" : "Specify a value for disabled status. Default is
>>\"false\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 15,
>> "confidential" : false,
>> "defaultValues" : [ "false" ]
>> },
>> "overridable" : false,
>> "values" : [ "false" ]
>> }, {
>> "schema" : {
>> "name" : "enabledStatusValue",
>> "displayName" : "Enable Status Value",
>> "helpMessage" : "Specify a value for enabled status. Default is
>>\"true\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 16,
>> "confidential" : false,
>> "defaultValues" : [ "true" ]
>> },
>> "overridable" : false,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "statusColumn",
>> "displayName" : "Status Column name",
>> "helpMessage" : "Status column name.",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 17,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ ]
>> } ],
>> "capabilities" : [ "SEARCH", "CREATE", "UPDATE" ]
>> }
>>
>> Ressource config:
>> {
>> "key" : "Users",
>> "connector" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>> "connectorDisplayName" : "SystemDB",
>> "orgUnit" : null,
>> "propagationPriority" : 2,
>> "randomPwdIfNotProvided" : false,
>> "enforceMandatoryCondition" : false,
>> "createTraceLevel" : "ALL",
>> "updateTraceLevel" : "ALL",
>> "deleteTraceLevel" : "ALL",
>> "provisioningTraceLevel" : "ALL",
>> "passwordPolicy" : "0b17832c-cdb2-4e92-9783-2ccdb23e92d1",
>> "accountPolicy" : "319350c1-6809-49ed-9350-c1680929eda1",
>> "pullPolicy" : "ec43ef48-4ba9-4ef0-83ef-484ba9fef083",
>> "pushPolicy" : "a66afe6e-7e23-44d4-aafe-6e7e2374d4dc",
>> "overrideCapabilities" : false,
>> "provisions" : [ {
>> "key" : "71e72b11-a3a7-4090-a72b-11a3a780902a",
>> "anyType" : "USER",
>> "objectClass" : "__ACCOUNT__",
>> "syncToken" : null,
>> "ignoreCaseMatch" : false,
>> "uidOnCreate" : null,
>> "mapping" : {
>> "connObjectLink" : null,
>> "connObjectKeyItem" : {
>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>> "intAttrName" : "numericalUID",
>> "extAttrName" : "uidNumber",
>> "connObjectKey" : true,
>> "password" : false,
>> "mandatoryCondition" : "true",
>> "purpose" : "PROPAGATION",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> },
>> "items" : [ {
>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>> "intAttrName" : "numericalUID",
>> "extAttrName" : "uidNumber",
>> "connObjectKey" : true,
>> "password" : false,
>> "mandatoryCondition" : "true",
>> "purpose" : "PROPAGATION",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "3ba6be21-82db-4702-a6be-2182dba702b0",
>> "intAttrName" : "organizationalUnit",
>> "extAttrName" : "group",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "69557f34-b7cd-4e11-957f-34b7cd1e1167",
>> "intAttrName" : "creationDate",
>> "extAttrName" : "createdDatum",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "d55ae9a2-10ca-45c3-9ae9-a210cae5c31f",
>> "intAttrName" : "displayName",
>> "extAttrName" : "fullName",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "9ccf8d00-14cb-4085-8f8d-0014cbb08527",
>> "intAttrName" : "uid",
>> "extAttrName" : "uid",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "true",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> } ],
>> "linkingItems" : [ {
>> "key" : null,
>> "intAttrName" : "AllUsers",
>> "extAttrName" : "uid",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> } ]
>> },
>> "auxClasses" : [ ],
>> "virSchemas" : [ "AllUsers" ]
>> } ],
>> "confOverride" : [ {
>> "schema" : {
>> "name" : "sourcePath",
>> "displayName" : "Source path",
>> "helpMessage" : "Absolute path of a directory where the CSV files
>>to be processed are located",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 1,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "/srv/tomcat/data/" ]
>> }, {
>> "schema" : {
>> "name" : "fileMask",
>> "displayName" : "File mask",
>> "helpMessage" : "Regular expression describing files to be
>>processed",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 2,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "aip-system_users_test.csv" ]
>> }, {
>> "schema" : {
>> "name" : "encoding",
>> "displayName" : "File encoding",
>> "helpMessage" : "Basic encoding of the file",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 3,
>> "confidential" : false,
>> "defaultValues" : [ "UTF-8" ]
>> },
>> "overridable" : true,
>> "values" : [ "UTF-8" ]
>> }, {
>> "schema" : {
>> "name" : "fieldDelimiter",
>> "displayName" : "Field delimiter",
>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>Default is \",\".",
>> "type" : "char",
>> "required" : false,
>> "order" : 4,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ "," ]
>> }, {
>> "schema" : {
>> "name" : "textQualifier",
>> "displayName" : "Text qualifier",
>> "helpMessage" : "Delimiter to determine beginning and end of text
>>in value. Default is \".",
>> "type" : "char",
>> "required" : false,
>> "order" : 5,
>> "confidential" : false,
>> "defaultValues" : [ "\"" ]
>> },
>> "overridable" : true,
>> "values" : [ "'" ]
>> }, {
>> "schema" : {
>> "name" : "keyColumnNames",
>> "displayName" : "Key column name",
>> "helpMessage" : "Name of the column used to identify user
>>uniquely",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 6,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "uidNumber" ]
>> }, {
>> "schema" : {
>> "name" : "fields",
>> "displayName" : "Column names",
>> "helpMessage" : "Column names",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 10,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "createdDatum", "createdTime", "uid", "uidNumber",
>>"gidNumber", "fullName", "office", "phone", "fax", "email",
>>"homeDir", "host", "group", "shell", "www", "passwd", "termination",
>>"tutor", "remarks" ]
>> }, {
>> "schema" : {
>> "name" : "ignoreHeader",
>> "displayName" : "Ignore header",
>> "helpMessage" : "Specify if first line of the file must be
>>ignored. Default is true.",
>> "type" : "java.lang.Boolean",
>> "required" : false,
>> "order" : 11,
>> "confidential" : false,
>> "defaultValues" : [ true ]
>> },
>> "overridable" : true,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "keyseparator",
>> "displayName" : "Key separator",
>> "helpMessage" : "Character used to separate keys in a multi-key
>>scenario. Default is \",\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 12,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ ";" ]
>> }, {
>> "schema" : {
>> "name" : "statusColumn",
>> "displayName" : "Status Column name",
>> "helpMessage" : "Status column name.",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 17,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "passwd" ]
>> } ],
>> "capabilitiesOverride" : [ "SEARCH", "CREATE", "UPDATE" ],
>> "propagationActions" : [ ]
>> }
>>
>> Best regards
>>
>> On Fri, 28 Jun 2019 10:40:58 +0200
>> Davide Cortellucci <da...@tirasa.net> wrote:
>>> Hi Rainer,
>>>
>>> can you please share the CSVDir configuration?
>>>
>>> Regards,
>>> Davide
>>>
>>> On 6/28/19 10:14 AM, Rainer Herbst wrote:
>>>> Dear all,
>>>>
>>>>
>>>> for the last 2 weeks I tried to get Syncope running using maven, and
>>>>at least I have an instance that works somehow.
>>>>
>>>> But I stumbled over a lot of problems with CSVDir not working. I
>>>>created a connector and a ressource, but everytime I try to pull the
>>>>content of a csv file, the task fails with error
>>>> org.quartz.JobExecutionException: While pulling from connector [See
>>>>nested exception:
>>>>org.identityconnectors.framework.common.exceptions.ConnectorException:
>>>>org.identityconnectors.framework.common.exceptions.ConnectorIOException:
>>>>java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
>>>>user lacks privilege or object not found: USER_EX88887 in statement
>>>>[SELECT * FROM USER_EX88887]]
>>>>
>>>> There are some files created in the CSV directory:
>>>> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
>>>>/srv/tomcat/data/
>>>> total 24
>>>> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
>>>> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
>>>> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
>>>> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
>>>> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
>>>> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>>>>
>>>> What is missing there?
>>>>
>>>> I like the openess and the concepts of Syncope, but, to be honest, I
>>>>did not expect to spend 2 days trying to read a simple CSV data
>>>>file... :-(
>>>>
>>>> Help is very much appreciated!
>>>>
>>>>
>>>> Rainer Herbst
>>>> IT-Service
>>>> Phone: +49 331 7499-257
>>>> e-mail: rainer.herbst@aip.de
>>>> https://www.aip.de
>>>>
>>>> -----------------------------------------------------------------------------------------------
>>>>Leibniz-Institut für Astrophysik Potsdam (AIP)
>>>> An der Sternwarte 16, 14482 Potsdam
>>>>
>>>> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
>>>> Stiftung bürgerlichen Rechts
>>>> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
>>>> -----------------------------------------------------------------------------------------------
>>>
>>>
>> Rainer Herbst
>> IT-Service
>> Phone: +49 331 7499-257
>> e-mail: rainer.herbst@aip.de
>> https://www.aip.de
>>
>> -----------------------------------------------------------------------------------------------
>> Leibniz-Institut für Astrophysik Potsdam (AIP)
>> An der Sternwarte 16, 14482 Potsdam
>>
>> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
>> Stiftung bürgerlichen Rechts
>> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
>> -----------------------------------------------------------------------------------------------
>>
>
Rainer Herbst
IT-Service
Phone: +49 331 7499-257
e-mail: rainer.herbst@aip.de
https://www.aip.de
-----------------------------------------------------------------------------------------------
Leibniz-Institut für Astrophysik Potsdam (AIP)
An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
Stiftung bürgerlichen Rechts
Stiftungsverzeichnis Brandenburg: 26 742-00/7026
-----------------------------------------------------------------------------------------------
Re: CSVDir
Posted by Davide Cortellucci <da...@tirasa.net>.
I managed to reproduce your error.
It looks like the problem is with the "group" column name: since it is
one of the SQL reserved words, you should change it in your csv and
update the connector accordingly.
Best regards
Davide
On 6/28/19 11:14 AM, Rainer Herbst wrote:
> Hi Davide,
>
>
> thank you for the quick response.
> Here is the current connector configuration. The ressource
> configuration follows below.
>
> {
> "key" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
> "adminRealm" : "/AIPTest",
> "location" : "file:/opt/syncope/bundles/",
> "connectorName" : "net.tirasa.connid.bundles.csvdir.CSVDirConnector",
> "bundleName" : "net.tirasa.connid.bundles.csvdir",
> "version" : "0.8.8",
> "displayName" : "SystemDB",
> "connRequestTimeout" : 10,
> "poolConf" : {
> "maxObjects" : 2000,
> "minIdle" : 20,
> "maxIdle" : 200,
> "maxWait" : 3000,
> "minEvictableIdleTimeMillis" : 2000
> },
> "conf" : [ {
> "schema" : {
> "name" : "sourcePath",
> "displayName" : "Source path",
> "helpMessage" : "Absolute path of a directory where the CSV files to
> be processed are located",
> "type" : "java.lang.String",
> "required" : true,
> "order" : 1,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "/srv/tomcat/data/" ]
> }, {
> "schema" : {
> "name" : "fileMask",
> "displayName" : "File mask",
> "helpMessage" : "Regular expression describing files to be processed",
> "type" : "java.lang.String",
> "required" : true,
> "order" : 2,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ ".csv" ]
> }, {
> "schema" : {
> "name" : "encoding",
> "displayName" : "File encoding",
> "helpMessage" : "Basic encoding of the file",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 3,
> "confidential" : false,
> "defaultValues" : [ "UTF-8" ]
> },
> "overridable" : true,
> "values" : [ "UTF-8" ]
> }, {
> "schema" : {
> "name" : "fieldDelimiter",
> "displayName" : "Field delimiter",
> "helpMessage" : "Delimiter used to separate fields in CSV files.
> Default is \",\".",
> "type" : "char",
> "required" : false,
> "order" : 4,
> "confidential" : false,
> "defaultValues" : [ "," ]
> },
> "overridable" : true,
> "values" : [ "," ]
> }, {
> "schema" : {
> "name" : "textQualifier",
> "displayName" : "Text qualifier",
> "helpMessage" : "Delimiter to determine beginning and end of text in
> value. Default is \".",
> "type" : "char",
> "required" : false,
> "order" : 5,
> "confidential" : false,
> "defaultValues" : [ "\"" ]
> },
> "overridable" : true,
> "values" : [ "'" ]
> }, {
> "schema" : {
> "name" : "keyColumnNames",
> "displayName" : "Key column name",
> "helpMessage" : "Name of the column used to identify user uniquely",
> "type" : "[Ljava.lang.String;",
> "required" : true,
> "order" : 6,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "uidNumber" ]
> }, {
> "schema" : {
> "name" : "passwordColumnName",
> "displayName" : "Password column name",
> "helpMessage" : "Name of the column used to specify user password",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 7,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : false,
> "values" : [ ]
> }, {
> "schema" : {
> "name" : "deleteColumnName",
> "displayName" : "Delete column name",
> "helpMessage" : "Name of the column used to specify users to be
> deleted",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 8,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : false,
> "values" : [ ]
> }, {
> "schema" : {
> "name" : "quotationRequired",
> "displayName" : "Value quotation required",
> "helpMessage" : "Specify if value quotation is required. Default is
> true.",
> "type" : "java.lang.Boolean",
> "required" : false,
> "order" : 9,
> "confidential" : false,
> "defaultValues" : [ true ]
> },
> "overridable" : false,
> "values" : [ true ]
> }, {
> "schema" : {
> "name" : "fields",
> "displayName" : "Column names",
> "helpMessage" : "Column names",
> "type" : "[Ljava.lang.String;",
> "required" : true,
> "order" : 10,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "uidNumber" ]
> }, {
> "schema" : {
> "name" : "ignoreHeader",
> "displayName" : "Ignore header",
> "helpMessage" : "Specify if first line of the file must be ignored.
> Default is true.",
> "type" : "java.lang.Boolean",
> "required" : false,
> "order" : 11,
> "confidential" : false,
> "defaultValues" : [ true ]
> },
> "overridable" : true,
> "values" : [ "true" ]
> }, {
> "schema" : {
> "name" : "keyseparator",
> "displayName" : "Key separator",
> "helpMessage" : "Character used to separate keys in a multi-key
> scenario. Default is \",\".",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 12,
> "confidential" : false,
> "defaultValues" : [ "," ]
> },
> "overridable" : true,
> "values" : [ ";" ]
> }, {
> "schema" : {
> "name" : "multivalueSeparator",
> "displayName" : "Multi value separator",
> "helpMessage" : "Character used to separate values in a multi-value
> scenario. Multivalue unsupported if not provided.",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 13,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : false,
> "values" : [ ]
> }, {
> "schema" : {
> "name" : "defaultStatusValue",
> "displayName" : "Default Status Value",
> "helpMessage" : "Enter the value for status in case of status not
> specified. Default is \"true\".",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 14,
> "confidential" : false,
> "defaultValues" : [ "true" ]
> },
> "overridable" : false,
> "values" : [ "true" ]
> }, {
> "schema" : {
> "name" : "disabledStatusValue",
> "displayName" : "Disabled Status Value",
> "helpMessage" : "Specify a value for disabled status. Default is
> \"false\".",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 15,
> "confidential" : false,
> "defaultValues" : [ "false" ]
> },
> "overridable" : false,
> "values" : [ "false" ]
> }, {
> "schema" : {
> "name" : "enabledStatusValue",
> "displayName" : "Enable Status Value",
> "helpMessage" : "Specify a value for enabled status. Default is
> \"true\".",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 16,
> "confidential" : false,
> "defaultValues" : [ "true" ]
> },
> "overridable" : false,
> "values" : [ "true" ]
> }, {
> "schema" : {
> "name" : "statusColumn",
> "displayName" : "Status Column name",
> "helpMessage" : "Status column name.",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 17,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ ]
> } ],
> "capabilities" : [ "SEARCH", "CREATE", "UPDATE" ]
> }
>
> Ressource config:
> {
> "key" : "Users",
> "connector" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
> "connectorDisplayName" : "SystemDB",
> "orgUnit" : null,
> "propagationPriority" : 2,
> "randomPwdIfNotProvided" : false,
> "enforceMandatoryCondition" : false,
> "createTraceLevel" : "ALL",
> "updateTraceLevel" : "ALL",
> "deleteTraceLevel" : "ALL",
> "provisioningTraceLevel" : "ALL",
> "passwordPolicy" : "0b17832c-cdb2-4e92-9783-2ccdb23e92d1",
> "accountPolicy" : "319350c1-6809-49ed-9350-c1680929eda1",
> "pullPolicy" : "ec43ef48-4ba9-4ef0-83ef-484ba9fef083",
> "pushPolicy" : "a66afe6e-7e23-44d4-aafe-6e7e2374d4dc",
> "overrideCapabilities" : false,
> "provisions" : [ {
> "key" : "71e72b11-a3a7-4090-a72b-11a3a780902a",
> "anyType" : "USER",
> "objectClass" : "__ACCOUNT__",
> "syncToken" : null,
> "ignoreCaseMatch" : false,
> "uidOnCreate" : null,
> "mapping" : {
> "connObjectLink" : null,
> "connObjectKeyItem" : {
> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
> "intAttrName" : "numericalUID",
> "extAttrName" : "uidNumber",
> "connObjectKey" : true,
> "password" : false,
> "mandatoryCondition" : "true",
> "purpose" : "PROPAGATION",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> },
> "items" : [ {
> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
> "intAttrName" : "numericalUID",
> "extAttrName" : "uidNumber",
> "connObjectKey" : true,
> "password" : false,
> "mandatoryCondition" : "true",
> "purpose" : "PROPAGATION",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> }, {
> "key" : "3ba6be21-82db-4702-a6be-2182dba702b0",
> "intAttrName" : "organizationalUnit",
> "extAttrName" : "group",
> "connObjectKey" : false,
> "password" : false,
> "mandatoryCondition" : "false",
> "purpose" : "BOTH",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> }, {
> "key" : "69557f34-b7cd-4e11-957f-34b7cd1e1167",
> "intAttrName" : "creationDate",
> "extAttrName" : "createdDatum",
> "connObjectKey" : false,
> "password" : false,
> "mandatoryCondition" : "false",
> "purpose" : "BOTH",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> }, {
> "key" : "d55ae9a2-10ca-45c3-9ae9-a210cae5c31f",
> "intAttrName" : "displayName",
> "extAttrName" : "fullName",
> "connObjectKey" : false,
> "password" : false,
> "mandatoryCondition" : "false",
> "purpose" : "BOTH",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> }, {
> "key" : "9ccf8d00-14cb-4085-8f8d-0014cbb08527",
> "intAttrName" : "uid",
> "extAttrName" : "uid",
> "connObjectKey" : false,
> "password" : false,
> "mandatoryCondition" : "true",
> "purpose" : "BOTH",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> } ],
> "linkingItems" : [ {
> "key" : null,
> "intAttrName" : "AllUsers",
> "extAttrName" : "uid",
> "connObjectKey" : false,
> "password" : false,
> "mandatoryCondition" : "false",
> "purpose" : "BOTH",
> "propagationJEXLTransformer" : null,
> "pullJEXLTransformer" : null,
> "transformers" : [ ]
> } ]
> },
> "auxClasses" : [ ],
> "virSchemas" : [ "AllUsers" ]
> } ],
> "confOverride" : [ {
> "schema" : {
> "name" : "sourcePath",
> "displayName" : "Source path",
> "helpMessage" : "Absolute path of a directory where the CSV files to
> be processed are located",
> "type" : "java.lang.String",
> "required" : true,
> "order" : 1,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "/srv/tomcat/data/" ]
> }, {
> "schema" : {
> "name" : "fileMask",
> "displayName" : "File mask",
> "helpMessage" : "Regular expression describing files to be processed",
> "type" : "java.lang.String",
> "required" : true,
> "order" : 2,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "aip-system_users_test.csv" ]
> }, {
> "schema" : {
> "name" : "encoding",
> "displayName" : "File encoding",
> "helpMessage" : "Basic encoding of the file",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 3,
> "confidential" : false,
> "defaultValues" : [ "UTF-8" ]
> },
> "overridable" : true,
> "values" : [ "UTF-8" ]
> }, {
> "schema" : {
> "name" : "fieldDelimiter",
> "displayName" : "Field delimiter",
> "helpMessage" : "Delimiter used to separate fields in CSV files.
> Default is \",\".",
> "type" : "char",
> "required" : false,
> "order" : 4,
> "confidential" : false,
> "defaultValues" : [ "," ]
> },
> "overridable" : true,
> "values" : [ "," ]
> }, {
> "schema" : {
> "name" : "textQualifier",
> "displayName" : "Text qualifier",
> "helpMessage" : "Delimiter to determine beginning and end of text in
> value. Default is \".",
> "type" : "char",
> "required" : false,
> "order" : 5,
> "confidential" : false,
> "defaultValues" : [ "\"" ]
> },
> "overridable" : true,
> "values" : [ "'" ]
> }, {
> "schema" : {
> "name" : "keyColumnNames",
> "displayName" : "Key column name",
> "helpMessage" : "Name of the column used to identify user uniquely",
> "type" : "[Ljava.lang.String;",
> "required" : true,
> "order" : 6,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "uidNumber" ]
> }, {
> "schema" : {
> "name" : "fields",
> "displayName" : "Column names",
> "helpMessage" : "Column names",
> "type" : "[Ljava.lang.String;",
> "required" : true,
> "order" : 10,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "createdDatum", "createdTime", "uid", "uidNumber",
> "gidNumber", "fullName", "office", "phone", "fax", "email", "homeDir",
> "host", "group", "shell", "www", "passwd", "termination", "tutor",
> "remarks" ]
> }, {
> "schema" : {
> "name" : "ignoreHeader",
> "displayName" : "Ignore header",
> "helpMessage" : "Specify if first line of the file must be ignored.
> Default is true.",
> "type" : "java.lang.Boolean",
> "required" : false,
> "order" : 11,
> "confidential" : false,
> "defaultValues" : [ true ]
> },
> "overridable" : true,
> "values" : [ "true" ]
> }, {
> "schema" : {
> "name" : "keyseparator",
> "displayName" : "Key separator",
> "helpMessage" : "Character used to separate keys in a multi-key
> scenario. Default is \",\".",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 12,
> "confidential" : false,
> "defaultValues" : [ "," ]
> },
> "overridable" : true,
> "values" : [ ";" ]
> }, {
> "schema" : {
> "name" : "statusColumn",
> "displayName" : "Status Column name",
> "helpMessage" : "Status column name.",
> "type" : "java.lang.String",
> "required" : false,
> "order" : 17,
> "confidential" : false,
> "defaultValues" : [ ]
> },
> "overridable" : true,
> "values" : [ "passwd" ]
> } ],
> "capabilitiesOverride" : [ "SEARCH", "CREATE", "UPDATE" ],
> "propagationActions" : [ ]
> }
>
> Best regards
>
> On Fri, 28 Jun 2019 10:40:58 +0200
> Davide Cortellucci <da...@tirasa.net> wrote:
>> Hi Rainer,
>>
>> can you please share the CSVDir configuration?
>>
>> Regards,
>> Davide
>>
>> On 6/28/19 10:14 AM, Rainer Herbst wrote:
>>> Dear all,
>>>
>>>
>>> for the last 2 weeks I tried to get Syncope running using maven, and
>>> at least I have an instance that works somehow.
>>>
>>> But I stumbled over a lot of problems with CSVDir not working. I
>>> created a connector and a ressource, but everytime I try to pull the
>>> content of a csv file, the task fails with error
>>> org.quartz.JobExecutionException: While pulling from connector [See
>>> nested exception:
>>> org.identityconnectors.framework.common.exceptions.ConnectorException:
>>> org.identityconnectors.framework.common.exceptions.ConnectorIOException:
>>> java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
>>> user lacks privilege or object not found: USER_EX88887 in statement
>>> [SELECT * FROM USER_EX88887]]
>>>
>>> There are some files created in the CSV directory:
>>> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
>>> /srv/tomcat/data/
>>> total 24
>>> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
>>> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
>>> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
>>> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
>>> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
>>> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>>>
>>> What is missing there?
>>>
>>> I like the openess and the concepts of Syncope, but, to be honest, I
>>> did not expect to spend 2 days trying to read a simple CSV data
>>> file... :-(
>>>
>>> Help is very much appreciated!
>>>
>>>
>>> Rainer Herbst
>>> IT-Service
>>> Phone: +49 331 7499-257
>>> e-mail: rainer.herbst@aip.de
>>> https://www.aip.de
>>>
>>> -----------------------------------------------------------------------------------------------
>>> Leibniz-Institut für Astrophysik Potsdam (AIP)
>>> An der Sternwarte 16, 14482 Potsdam
>>>
>>> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
>>> Stiftung bürgerlichen Rechts
>>> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
>>> -----------------------------------------------------------------------------------------------
>>
>>
>
> Rainer Herbst
> IT-Service
> Phone: +49 331 7499-257
> e-mail: rainer.herbst@aip.de
> https://www.aip.de
>
> -----------------------------------------------------------------------------------------------
>
> Leibniz-Institut für Astrophysik Potsdam (AIP)
> An der Sternwarte 16, 14482 Potsdam
>
> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
> Stiftung bürgerlichen Rechts
> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
> -----------------------------------------------------------------------------------------------
>
>
Re: CSVDir
Posted by Rainer Herbst <ra...@aip.de>.
Hi Davide,
thank you for the quick response.
Here is the current connector configuration. The ressource configuration
follows below.
{
"key" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
"adminRealm" : "/AIPTest",
"location" : "file:/opt/syncope/bundles/",
"connectorName" : "net.tirasa.connid.bundles.csvdir.CSVDirConnector",
"bundleName" : "net.tirasa.connid.bundles.csvdir",
"version" : "0.8.8",
"displayName" : "SystemDB",
"connRequestTimeout" : 10,
"poolConf" : {
"maxObjects" : 2000,
"minIdle" : 20,
"maxIdle" : 200,
"maxWait" : 3000,
"minEvictableIdleTimeMillis" : 2000
},
"conf" : [ {
"schema" : {
"name" : "sourcePath",
"displayName" : "Source path",
"helpMessage" : "Absolute path of a directory where the CSV files to be
processed are located",
"type" : "java.lang.String",
"required" : true,
"order" : 1,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "/srv/tomcat/data/" ]
}, {
"schema" : {
"name" : "fileMask",
"displayName" : "File mask",
"helpMessage" : "Regular expression describing files to be processed",
"type" : "java.lang.String",
"required" : true,
"order" : 2,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ ".csv" ]
}, {
"schema" : {
"name" : "encoding",
"displayName" : "File encoding",
"helpMessage" : "Basic encoding of the file",
"type" : "java.lang.String",
"required" : false,
"order" : 3,
"confidential" : false,
"defaultValues" : [ "UTF-8" ]
},
"overridable" : true,
"values" : [ "UTF-8" ]
}, {
"schema" : {
"name" : "fieldDelimiter",
"displayName" : "Field delimiter",
"helpMessage" : "Delimiter used to separate fields in CSV files. Default
is \",\".",
"type" : "char",
"required" : false,
"order" : 4,
"confidential" : false,
"defaultValues" : [ "," ]
},
"overridable" : true,
"values" : [ "," ]
}, {
"schema" : {
"name" : "textQualifier",
"displayName" : "Text qualifier",
"helpMessage" : "Delimiter to determine beginning and end of text in
value. Default is \".",
"type" : "char",
"required" : false,
"order" : 5,
"confidential" : false,
"defaultValues" : [ "\"" ]
},
"overridable" : true,
"values" : [ "'" ]
}, {
"schema" : {
"name" : "keyColumnNames",
"displayName" : "Key column name",
"helpMessage" : "Name of the column used to identify user uniquely",
"type" : "[Ljava.lang.String;",
"required" : true,
"order" : 6,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "uidNumber" ]
}, {
"schema" : {
"name" : "passwordColumnName",
"displayName" : "Password column name",
"helpMessage" : "Name of the column used to specify user password",
"type" : "java.lang.String",
"required" : false,
"order" : 7,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : false,
"values" : [ ]
}, {
"schema" : {
"name" : "deleteColumnName",
"displayName" : "Delete column name",
"helpMessage" : "Name of the column used to specify users to be deleted",
"type" : "java.lang.String",
"required" : false,
"order" : 8,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : false,
"values" : [ ]
}, {
"schema" : {
"name" : "quotationRequired",
"displayName" : "Value quotation required",
"helpMessage" : "Specify if value quotation is required. Default is
true.",
"type" : "java.lang.Boolean",
"required" : false,
"order" : 9,
"confidential" : false,
"defaultValues" : [ true ]
},
"overridable" : false,
"values" : [ true ]
}, {
"schema" : {
"name" : "fields",
"displayName" : "Column names",
"helpMessage" : "Column names",
"type" : "[Ljava.lang.String;",
"required" : true,
"order" : 10,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "uidNumber" ]
}, {
"schema" : {
"name" : "ignoreHeader",
"displayName" : "Ignore header",
"helpMessage" : "Specify if first line of the file must be ignored.
Default is true.",
"type" : "java.lang.Boolean",
"required" : false,
"order" : 11,
"confidential" : false,
"defaultValues" : [ true ]
},
"overridable" : true,
"values" : [ "true" ]
}, {
"schema" : {
"name" : "keyseparator",
"displayName" : "Key separator",
"helpMessage" : "Character used to separate keys in a multi-key scenario.
Default is \",\".",
"type" : "java.lang.String",
"required" : false,
"order" : 12,
"confidential" : false,
"defaultValues" : [ "," ]
},
"overridable" : true,
"values" : [ ";" ]
}, {
"schema" : {
"name" : "multivalueSeparator",
"displayName" : "Multi value separator",
"helpMessage" : "Character used to separate values in a multi-value
scenario. Multivalue unsupported if not provided.",
"type" : "java.lang.String",
"required" : false,
"order" : 13,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : false,
"values" : [ ]
}, {
"schema" : {
"name" : "defaultStatusValue",
"displayName" : "Default Status Value",
"helpMessage" : "Enter the value for status in case of status not
specified. Default is \"true\".",
"type" : "java.lang.String",
"required" : false,
"order" : 14,
"confidential" : false,
"defaultValues" : [ "true" ]
},
"overridable" : false,
"values" : [ "true" ]
}, {
"schema" : {
"name" : "disabledStatusValue",
"displayName" : "Disabled Status Value",
"helpMessage" : "Specify a value for disabled status. Default is
\"false\".",
"type" : "java.lang.String",
"required" : false,
"order" : 15,
"confidential" : false,
"defaultValues" : [ "false" ]
},
"overridable" : false,
"values" : [ "false" ]
}, {
"schema" : {
"name" : "enabledStatusValue",
"displayName" : "Enable Status Value",
"helpMessage" : "Specify a value for enabled status. Default is
\"true\".",
"type" : "java.lang.String",
"required" : false,
"order" : 16,
"confidential" : false,
"defaultValues" : [ "true" ]
},
"overridable" : false,
"values" : [ "true" ]
}, {
"schema" : {
"name" : "statusColumn",
"displayName" : "Status Column name",
"helpMessage" : "Status column name.",
"type" : "java.lang.String",
"required" : false,
"order" : 17,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ ]
} ],
"capabilities" : [ "SEARCH", "CREATE", "UPDATE" ]
}
Ressource config:
{
"key" : "Users",
"connector" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
"connectorDisplayName" : "SystemDB",
"orgUnit" : null,
"propagationPriority" : 2,
"randomPwdIfNotProvided" : false,
"enforceMandatoryCondition" : false,
"createTraceLevel" : "ALL",
"updateTraceLevel" : "ALL",
"deleteTraceLevel" : "ALL",
"provisioningTraceLevel" : "ALL",
"passwordPolicy" : "0b17832c-cdb2-4e92-9783-2ccdb23e92d1",
"accountPolicy" : "319350c1-6809-49ed-9350-c1680929eda1",
"pullPolicy" : "ec43ef48-4ba9-4ef0-83ef-484ba9fef083",
"pushPolicy" : "a66afe6e-7e23-44d4-aafe-6e7e2374d4dc",
"overrideCapabilities" : false,
"provisions" : [ {
"key" : "71e72b11-a3a7-4090-a72b-11a3a780902a",
"anyType" : "USER",
"objectClass" : "__ACCOUNT__",
"syncToken" : null,
"ignoreCaseMatch" : false,
"uidOnCreate" : null,
"mapping" : {
"connObjectLink" : null,
"connObjectKeyItem" : {
"key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
"intAttrName" : "numericalUID",
"extAttrName" : "uidNumber",
"connObjectKey" : true,
"password" : false,
"mandatoryCondition" : "true",
"purpose" : "PROPAGATION",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
},
"items" : [ {
"key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
"intAttrName" : "numericalUID",
"extAttrName" : "uidNumber",
"connObjectKey" : true,
"password" : false,
"mandatoryCondition" : "true",
"purpose" : "PROPAGATION",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
}, {
"key" : "3ba6be21-82db-4702-a6be-2182dba702b0",
"intAttrName" : "organizationalUnit",
"extAttrName" : "group",
"connObjectKey" : false,
"password" : false,
"mandatoryCondition" : "false",
"purpose" : "BOTH",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
}, {
"key" : "69557f34-b7cd-4e11-957f-34b7cd1e1167",
"intAttrName" : "creationDate",
"extAttrName" : "createdDatum",
"connObjectKey" : false,
"password" : false,
"mandatoryCondition" : "false",
"purpose" : "BOTH",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
}, {
"key" : "d55ae9a2-10ca-45c3-9ae9-a210cae5c31f",
"intAttrName" : "displayName",
"extAttrName" : "fullName",
"connObjectKey" : false,
"password" : false,
"mandatoryCondition" : "false",
"purpose" : "BOTH",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
}, {
"key" : "9ccf8d00-14cb-4085-8f8d-0014cbb08527",
"intAttrName" : "uid",
"extAttrName" : "uid",
"connObjectKey" : false,
"password" : false,
"mandatoryCondition" : "true",
"purpose" : "BOTH",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
} ],
"linkingItems" : [ {
"key" : null,
"intAttrName" : "AllUsers",
"extAttrName" : "uid",
"connObjectKey" : false,
"password" : false,
"mandatoryCondition" : "false",
"purpose" : "BOTH",
"propagationJEXLTransformer" : null,
"pullJEXLTransformer" : null,
"transformers" : [ ]
} ]
},
"auxClasses" : [ ],
"virSchemas" : [ "AllUsers" ]
} ],
"confOverride" : [ {
"schema" : {
"name" : "sourcePath",
"displayName" : "Source path",
"helpMessage" : "Absolute path of a directory where the CSV files to be
processed are located",
"type" : "java.lang.String",
"required" : true,
"order" : 1,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "/srv/tomcat/data/" ]
}, {
"schema" : {
"name" : "fileMask",
"displayName" : "File mask",
"helpMessage" : "Regular expression describing files to be processed",
"type" : "java.lang.String",
"required" : true,
"order" : 2,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "aip-system_users_test.csv" ]
}, {
"schema" : {
"name" : "encoding",
"displayName" : "File encoding",
"helpMessage" : "Basic encoding of the file",
"type" : "java.lang.String",
"required" : false,
"order" : 3,
"confidential" : false,
"defaultValues" : [ "UTF-8" ]
},
"overridable" : true,
"values" : [ "UTF-8" ]
}, {
"schema" : {
"name" : "fieldDelimiter",
"displayName" : "Field delimiter",
"helpMessage" : "Delimiter used to separate fields in CSV files. Default
is \",\".",
"type" : "char",
"required" : false,
"order" : 4,
"confidential" : false,
"defaultValues" : [ "," ]
},
"overridable" : true,
"values" : [ "," ]
}, {
"schema" : {
"name" : "textQualifier",
"displayName" : "Text qualifier",
"helpMessage" : "Delimiter to determine beginning and end of text in
value. Default is \".",
"type" : "char",
"required" : false,
"order" : 5,
"confidential" : false,
"defaultValues" : [ "\"" ]
},
"overridable" : true,
"values" : [ "'" ]
}, {
"schema" : {
"name" : "keyColumnNames",
"displayName" : "Key column name",
"helpMessage" : "Name of the column used to identify user uniquely",
"type" : "[Ljava.lang.String;",
"required" : true,
"order" : 6,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "uidNumber" ]
}, {
"schema" : {
"name" : "fields",
"displayName" : "Column names",
"helpMessage" : "Column names",
"type" : "[Ljava.lang.String;",
"required" : true,
"order" : 10,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "createdDatum", "createdTime", "uid", "uidNumber",
"gidNumber", "fullName", "office", "phone", "fax", "email", "homeDir",
"host", "group", "shell", "www", "passwd", "termination", "tutor", "remarks"
]
}, {
"schema" : {
"name" : "ignoreHeader",
"displayName" : "Ignore header",
"helpMessage" : "Specify if first line of the file must be ignored.
Default is true.",
"type" : "java.lang.Boolean",
"required" : false,
"order" : 11,
"confidential" : false,
"defaultValues" : [ true ]
},
"overridable" : true,
"values" : [ "true" ]
}, {
"schema" : {
"name" : "keyseparator",
"displayName" : "Key separator",
"helpMessage" : "Character used to separate keys in a multi-key scenario.
Default is \",\".",
"type" : "java.lang.String",
"required" : false,
"order" : 12,
"confidential" : false,
"defaultValues" : [ "," ]
},
"overridable" : true,
"values" : [ ";" ]
}, {
"schema" : {
"name" : "statusColumn",
"displayName" : "Status Column name",
"helpMessage" : "Status column name.",
"type" : "java.lang.String",
"required" : false,
"order" : 17,
"confidential" : false,
"defaultValues" : [ ]
},
"overridable" : true,
"values" : [ "passwd" ]
} ],
"capabilitiesOverride" : [ "SEARCH", "CREATE", "UPDATE" ],
"propagationActions" : [ ]
}
Best regards
On Fri, 28 Jun 2019 10:40:58 +0200
Davide Cortellucci <da...@tirasa.net> wrote:
> Hi Rainer,
>
> can you please share the CSVDir configuration?
>
> Regards,
> Davide
>
> On 6/28/19 10:14 AM, Rainer Herbst wrote:
>> Dear all,
>>
>>
>> for the last 2 weeks I tried to get Syncope running using maven, and
>>at least I have an instance that works somehow.
>>
>> But I stumbled over a lot of problems with CSVDir not working. I
>>created a connector and a ressource, but everytime I try to pull the
>>content of a csv file, the task fails with error
>> org.quartz.JobExecutionException: While pulling from connector [See
>>nested exception:
>>org.identityconnectors.framework.common.exceptions.ConnectorException:
>>org.identityconnectors.framework.common.exceptions.ConnectorIOException:
>>java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
>>user lacks privilege or object not found: USER_EX88887 in statement
>>[SELECT * FROM USER_EX88887]]
>>
>> There are some files created in the CSV directory:
>> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
>>/srv/tomcat/data/
>> total 24
>> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
>> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
>> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
>> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
>> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
>> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>>
>> What is missing there?
>>
>> I like the openess and the concepts of Syncope, but, to be honest, I
>>did not expect to spend 2 days trying to read a simple CSV data
>>file... :-(
>>
>> Help is very much appreciated!
>>
>>
>> Rainer Herbst
>> IT-Service
>> Phone: +49 331 7499-257
>> e-mail: rainer.herbst@aip.de
>> https://www.aip.de
>>
>> -----------------------------------------------------------------------------------------------
>> Leibniz-Institut für Astrophysik Potsdam (AIP)
>> An der Sternwarte 16, 14482 Potsdam
>>
>> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
>> Stiftung bürgerlichen Rechts
>> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
>> -----------------------------------------------------------------------------------------------
>
Rainer Herbst
IT-Service
Phone: +49 331 7499-257
e-mail: rainer.herbst@aip.de
https://www.aip.de
-----------------------------------------------------------------------------------------------
Leibniz-Institut für Astrophysik Potsdam (AIP)
An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
Stiftung bürgerlichen Rechts
Stiftungsverzeichnis Brandenburg: 26 742-00/7026
-----------------------------------------------------------------------------------------------
Re: CSVDir
Posted by Davide Cortellucci <da...@tirasa.net>.
Hi Rainer,
can you please share the CSVDir configuration?
Regards,
Davide
On 6/28/19 10:14 AM, Rainer Herbst wrote:
> Dear all,
>
>
> for the last 2 weeks I tried to get Syncope running using maven, and
> at least I have an instance that works somehow.
>
> But I stumbled over a lot of problems with CSVDir not working. I
> created a connector and a ressource, but everytime I try to pull the
> content of a csv file, the task fails with error
> org.quartz.JobExecutionException: While pulling from connector [See
> nested exception:
> org.identityconnectors.framework.common.exceptions.ConnectorException:
> org.identityconnectors.framework.common.exceptions.ConnectorIOException:
> java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
> user lacks privilege or object not found: USER_EX88887 in statement
> [SELECT * FROM USER_EX88887]]
>
> There are some files created in the CSV directory:
> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
> /srv/tomcat/data/
> total 24
> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>
> What is missing there?
>
> I like the openess and the concepts of Syncope, but, to be honest, I
> did not expect to spend 2 days trying to read a simple CSV data
> file... :-(
>
> Help is very much appreciated!
>
>
> Rainer Herbst
> IT-Service
> Phone: +49 331 7499-257
> e-mail: rainer.herbst@aip.de
> https://www.aip.de
>
> -----------------------------------------------------------------------------------------------
>
> Leibniz-Institut für Astrophysik Potsdam (AIP)
> An der Sternwarte 16, 14482 Potsdam
>
> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
> Stiftung bürgerlichen Rechts
> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
> -----------------------------------------------------------------------------------------------
>