You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@syncope.apache.org by Rainer Herbst <ra...@aip.de> on 2019/07/01 08:50:20 UTC
Re: CSVDir
Thanks a lot, that solved the problem!
Or should I say, it is a goot workaround? A solution would be to escape the
field names, so that the SQL statements will work no matter what field names
are used? ;-)
On Fri, 28 Jun 2019 17:08:23 +0200
Davide Cortellucci <da...@tirasa.net> wrote:
> I managed to reproduce your error.
>
> It looks like the problem is with the "group" column name: since it
>is one of the SQL reserved words, you should change it in your csv
>and update the connector accordingly.
>
> Best regards
>
> Davide
>
> On 6/28/19 11:14 AM, Rainer Herbst wrote:
>> Hi Davide,
>>
>>
>> thank you for the quick response.
>> Here is the current connector configuration. The ressource
>>configuration follows below.
>>
>> {
>> "key" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>> "adminRealm" : "/AIPTest",
>> "location" : "file:/opt/syncope/bundles/",
>> "connectorName" :
>>"net.tirasa.connid.bundles.csvdir.CSVDirConnector",
>> "bundleName" : "net.tirasa.connid.bundles.csvdir",
>> "version" : "0.8.8",
>> "displayName" : "SystemDB",
>> "connRequestTimeout" : 10,
>> "poolConf" : {
>> "maxObjects" : 2000,
>> "minIdle" : 20,
>> "maxIdle" : 200,
>> "maxWait" : 3000,
>> "minEvictableIdleTimeMillis" : 2000
>> },
>> "conf" : [ {
>> "schema" : {
>> "name" : "sourcePath",
>> "displayName" : "Source path",
>> "helpMessage" : "Absolute path of a directory where the CSV files
>>to be processed are located",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 1,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "/srv/tomcat/data/" ]
>> }, {
>> "schema" : {
>> "name" : "fileMask",
>> "displayName" : "File mask",
>> "helpMessage" : "Regular expression describing files to be
>>processed",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 2,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ ".csv" ]
>> }, {
>> "schema" : {
>> "name" : "encoding",
>> "displayName" : "File encoding",
>> "helpMessage" : "Basic encoding of the file",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 3,
>> "confidential" : false,
>> "defaultValues" : [ "UTF-8" ]
>> },
>> "overridable" : true,
>> "values" : [ "UTF-8" ]
>> }, {
>> "schema" : {
>> "name" : "fieldDelimiter",
>> "displayName" : "Field delimiter",
>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>Default is \",\".",
>> "type" : "char",
>> "required" : false,
>> "order" : 4,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ "," ]
>> }, {
>> "schema" : {
>> "name" : "textQualifier",
>> "displayName" : "Text qualifier",
>> "helpMessage" : "Delimiter to determine beginning and end of text
>>in value. Default is \".",
>> "type" : "char",
>> "required" : false,
>> "order" : 5,
>> "confidential" : false,
>> "defaultValues" : [ "\"" ]
>> },
>> "overridable" : true,
>> "values" : [ "'" ]
>> }, {
>> "schema" : {
>> "name" : "keyColumnNames",
>> "displayName" : "Key column name",
>> "helpMessage" : "Name of the column used to identify user
>>uniquely",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 6,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "uidNumber" ]
>> }, {
>> "schema" : {
>> "name" : "passwordColumnName",
>> "displayName" : "Password column name",
>> "helpMessage" : "Name of the column used to specify user
>>password",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 7,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : false,
>> "values" : [ ]
>> }, {
>> "schema" : {
>> "name" : "deleteColumnName",
>> "displayName" : "Delete column name",
>> "helpMessage" : "Name of the column used to specify users to be
>>deleted",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 8,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : false,
>> "values" : [ ]
>> }, {
>> "schema" : {
>> "name" : "quotationRequired",
>> "displayName" : "Value quotation required",
>> "helpMessage" : "Specify if value quotation is required. Default
>>is true.",
>> "type" : "java.lang.Boolean",
>> "required" : false,
>> "order" : 9,
>> "confidential" : false,
>> "defaultValues" : [ true ]
>> },
>> "overridable" : false,
>> "values" : [ true ]
>> }, {
>> "schema" : {
>> "name" : "fields",
>> "displayName" : "Column names",
>> "helpMessage" : "Column names",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 10,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "uidNumber" ]
>> }, {
>> "schema" : {
>> "name" : "ignoreHeader",
>> "displayName" : "Ignore header",
>> "helpMessage" : "Specify if first line of the file must be
>>ignored. Default is true.",
>> "type" : "java.lang.Boolean",
>> "required" : false,
>> "order" : 11,
>> "confidential" : false,
>> "defaultValues" : [ true ]
>> },
>> "overridable" : true,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "keyseparator",
>> "displayName" : "Key separator",
>> "helpMessage" : "Character used to separate keys in a multi-key
>>scenario. Default is \",\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 12,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ ";" ]
>> }, {
>> "schema" : {
>> "name" : "multivalueSeparator",
>> "displayName" : "Multi value separator",
>> "helpMessage" : "Character used to separate values in a
>>multi-value scenario. Multivalue unsupported if not provided.",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 13,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : false,
>> "values" : [ ]
>> }, {
>> "schema" : {
>> "name" : "defaultStatusValue",
>> "displayName" : "Default Status Value",
>> "helpMessage" : "Enter the value for status in case of status not
>>specified. Default is \"true\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 14,
>> "confidential" : false,
>> "defaultValues" : [ "true" ]
>> },
>> "overridable" : false,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "disabledStatusValue",
>> "displayName" : "Disabled Status Value",
>> "helpMessage" : "Specify a value for disabled status. Default is
>>\"false\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 15,
>> "confidential" : false,
>> "defaultValues" : [ "false" ]
>> },
>> "overridable" : false,
>> "values" : [ "false" ]
>> }, {
>> "schema" : {
>> "name" : "enabledStatusValue",
>> "displayName" : "Enable Status Value",
>> "helpMessage" : "Specify a value for enabled status. Default is
>>\"true\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 16,
>> "confidential" : false,
>> "defaultValues" : [ "true" ]
>> },
>> "overridable" : false,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "statusColumn",
>> "displayName" : "Status Column name",
>> "helpMessage" : "Status column name.",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 17,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ ]
>> } ],
>> "capabilities" : [ "SEARCH", "CREATE", "UPDATE" ]
>> }
>>
>> Ressource config:
>> {
>> "key" : "Users",
>> "connector" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>> "connectorDisplayName" : "SystemDB",
>> "orgUnit" : null,
>> "propagationPriority" : 2,
>> "randomPwdIfNotProvided" : false,
>> "enforceMandatoryCondition" : false,
>> "createTraceLevel" : "ALL",
>> "updateTraceLevel" : "ALL",
>> "deleteTraceLevel" : "ALL",
>> "provisioningTraceLevel" : "ALL",
>> "passwordPolicy" : "0b17832c-cdb2-4e92-9783-2ccdb23e92d1",
>> "accountPolicy" : "319350c1-6809-49ed-9350-c1680929eda1",
>> "pullPolicy" : "ec43ef48-4ba9-4ef0-83ef-484ba9fef083",
>> "pushPolicy" : "a66afe6e-7e23-44d4-aafe-6e7e2374d4dc",
>> "overrideCapabilities" : false,
>> "provisions" : [ {
>> "key" : "71e72b11-a3a7-4090-a72b-11a3a780902a",
>> "anyType" : "USER",
>> "objectClass" : "__ACCOUNT__",
>> "syncToken" : null,
>> "ignoreCaseMatch" : false,
>> "uidOnCreate" : null,
>> "mapping" : {
>> "connObjectLink" : null,
>> "connObjectKeyItem" : {
>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>> "intAttrName" : "numericalUID",
>> "extAttrName" : "uidNumber",
>> "connObjectKey" : true,
>> "password" : false,
>> "mandatoryCondition" : "true",
>> "purpose" : "PROPAGATION",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> },
>> "items" : [ {
>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>> "intAttrName" : "numericalUID",
>> "extAttrName" : "uidNumber",
>> "connObjectKey" : true,
>> "password" : false,
>> "mandatoryCondition" : "true",
>> "purpose" : "PROPAGATION",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "3ba6be21-82db-4702-a6be-2182dba702b0",
>> "intAttrName" : "organizationalUnit",
>> "extAttrName" : "group",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "69557f34-b7cd-4e11-957f-34b7cd1e1167",
>> "intAttrName" : "creationDate",
>> "extAttrName" : "createdDatum",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "d55ae9a2-10ca-45c3-9ae9-a210cae5c31f",
>> "intAttrName" : "displayName",
>> "extAttrName" : "fullName",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> }, {
>> "key" : "9ccf8d00-14cb-4085-8f8d-0014cbb08527",
>> "intAttrName" : "uid",
>> "extAttrName" : "uid",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "true",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> } ],
>> "linkingItems" : [ {
>> "key" : null,
>> "intAttrName" : "AllUsers",
>> "extAttrName" : "uid",
>> "connObjectKey" : false,
>> "password" : false,
>> "mandatoryCondition" : "false",
>> "purpose" : "BOTH",
>> "propagationJEXLTransformer" : null,
>> "pullJEXLTransformer" : null,
>> "transformers" : [ ]
>> } ]
>> },
>> "auxClasses" : [ ],
>> "virSchemas" : [ "AllUsers" ]
>> } ],
>> "confOverride" : [ {
>> "schema" : {
>> "name" : "sourcePath",
>> "displayName" : "Source path",
>> "helpMessage" : "Absolute path of a directory where the CSV files
>>to be processed are located",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 1,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "/srv/tomcat/data/" ]
>> }, {
>> "schema" : {
>> "name" : "fileMask",
>> "displayName" : "File mask",
>> "helpMessage" : "Regular expression describing files to be
>>processed",
>> "type" : "java.lang.String",
>> "required" : true,
>> "order" : 2,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "aip-system_users_test.csv" ]
>> }, {
>> "schema" : {
>> "name" : "encoding",
>> "displayName" : "File encoding",
>> "helpMessage" : "Basic encoding of the file",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 3,
>> "confidential" : false,
>> "defaultValues" : [ "UTF-8" ]
>> },
>> "overridable" : true,
>> "values" : [ "UTF-8" ]
>> }, {
>> "schema" : {
>> "name" : "fieldDelimiter",
>> "displayName" : "Field delimiter",
>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>Default is \",\".",
>> "type" : "char",
>> "required" : false,
>> "order" : 4,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ "," ]
>> }, {
>> "schema" : {
>> "name" : "textQualifier",
>> "displayName" : "Text qualifier",
>> "helpMessage" : "Delimiter to determine beginning and end of text
>>in value. Default is \".",
>> "type" : "char",
>> "required" : false,
>> "order" : 5,
>> "confidential" : false,
>> "defaultValues" : [ "\"" ]
>> },
>> "overridable" : true,
>> "values" : [ "'" ]
>> }, {
>> "schema" : {
>> "name" : "keyColumnNames",
>> "displayName" : "Key column name",
>> "helpMessage" : "Name of the column used to identify user
>>uniquely",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 6,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "uidNumber" ]
>> }, {
>> "schema" : {
>> "name" : "fields",
>> "displayName" : "Column names",
>> "helpMessage" : "Column names",
>> "type" : "[Ljava.lang.String;",
>> "required" : true,
>> "order" : 10,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "createdDatum", "createdTime", "uid", "uidNumber",
>>"gidNumber", "fullName", "office", "phone", "fax", "email",
>>"homeDir", "host", "group", "shell", "www", "passwd", "termination",
>>"tutor", "remarks" ]
>> }, {
>> "schema" : {
>> "name" : "ignoreHeader",
>> "displayName" : "Ignore header",
>> "helpMessage" : "Specify if first line of the file must be
>>ignored. Default is true.",
>> "type" : "java.lang.Boolean",
>> "required" : false,
>> "order" : 11,
>> "confidential" : false,
>> "defaultValues" : [ true ]
>> },
>> "overridable" : true,
>> "values" : [ "true" ]
>> }, {
>> "schema" : {
>> "name" : "keyseparator",
>> "displayName" : "Key separator",
>> "helpMessage" : "Character used to separate keys in a multi-key
>>scenario. Default is \",\".",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 12,
>> "confidential" : false,
>> "defaultValues" : [ "," ]
>> },
>> "overridable" : true,
>> "values" : [ ";" ]
>> }, {
>> "schema" : {
>> "name" : "statusColumn",
>> "displayName" : "Status Column name",
>> "helpMessage" : "Status column name.",
>> "type" : "java.lang.String",
>> "required" : false,
>> "order" : 17,
>> "confidential" : false,
>> "defaultValues" : [ ]
>> },
>> "overridable" : true,
>> "values" : [ "passwd" ]
>> } ],
>> "capabilitiesOverride" : [ "SEARCH", "CREATE", "UPDATE" ],
>> "propagationActions" : [ ]
>> }
>>
>> Best regards
>>
>> On Fri, 28 Jun 2019 10:40:58 +0200
>> Davide Cortellucci <da...@tirasa.net> wrote:
>>> Hi Rainer,
>>>
>>> can you please share the CSVDir configuration?
>>>
>>> Regards,
>>> Davide
>>>
>>> On 6/28/19 10:14 AM, Rainer Herbst wrote:
>>>> Dear all,
>>>>
>>>>
>>>> for the last 2 weeks I tried to get Syncope running using maven, and
>>>>at least I have an instance that works somehow.
>>>>
>>>> But I stumbled over a lot of problems with CSVDir not working. I
>>>>created a connector and a ressource, but everytime I try to pull the
>>>>content of a csv file, the task fails with error
>>>> org.quartz.JobExecutionException: While pulling from connector [See
>>>>nested exception:
>>>>org.identityconnectors.framework.common.exceptions.ConnectorException:
>>>>org.identityconnectors.framework.common.exceptions.ConnectorIOException:
>>>>java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
>>>>user lacks privilege or object not found: USER_EX88887 in statement
>>>>[SELECT * FROM USER_EX88887]]
>>>>
>>>> There are some files created in the CSV directory:
>>>> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
>>>>/srv/tomcat/data/
>>>> total 24
>>>> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
>>>> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
>>>> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
>>>> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
>>>> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
>>>> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>>>>
>>>> What is missing there?
>>>>
>>>> I like the openess and the concepts of Syncope, but, to be honest, I
>>>>did not expect to spend 2 days trying to read a simple CSV data
>>>>file... :-(
>>>>
>>>> Help is very much appreciated!
>>>>
>>>>
>>>> Rainer Herbst
>>>> IT-Service
>>>> Phone: +49 331 7499-257
>>>> e-mail: rainer.herbst@aip.de
>>>> https://www.aip.de
>>>>
>>>> -----------------------------------------------------------------------------------------------
>>>>Leibniz-Institut für Astrophysik Potsdam (AIP)
>>>> An der Sternwarte 16, 14482 Potsdam
>>>>
>>>> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
>>>> Stiftung bürgerlichen Rechts
>>>> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
>>>> -----------------------------------------------------------------------------------------------
>>>
>>>
>> Rainer Herbst
>> IT-Service
>> Phone: +49 331 7499-257
>> e-mail: rainer.herbst@aip.de
>> https://www.aip.de
>>
>> -----------------------------------------------------------------------------------------------
>> Leibniz-Institut für Astrophysik Potsdam (AIP)
>> An der Sternwarte 16, 14482 Potsdam
>>
>> Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
>> Stiftung bürgerlichen Rechts
>> Stiftungsverzeichnis Brandenburg: 26 742-00/7026
>> -----------------------------------------------------------------------------------------------
>>
>
Rainer Herbst
IT-Service
Phone: +49 331 7499-257
e-mail: rainer.herbst@aip.de
https://www.aip.de
-----------------------------------------------------------------------------------------------
Leibniz-Institut für Astrophysik Potsdam (AIP)
An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Matthias Winker
Stiftung bürgerlichen Rechts
Stiftungsverzeichnis Brandenburg: 26 742-00/7026
-----------------------------------------------------------------------------------------------
Re: CSVDir
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 01/07/19 10:50, Rainer Herbst wrote:
> Thanks a lot, that solved the problem!
>
> Or should I say, it is a goot workaround? A solution would be to
> escape the field names, so that the SQL statements will work no matter
> what field names are used? ;-)
Hi Rainer,
glad that Davide was able to help with your troubles.
Feel free to reach up the ConnId mailing list [1] to discuss this, or to
submit a PR [2].
Regards.
[1] https://groups.google.com/forum/#!forum/connid-users
[2] https://github.com/Tirasa/ConnIdCSVDirBundle
> On Fri, 28 Jun 2019 17:08:23 +0200
> Davide Cortellucci <da...@tirasa.net> wrote:
>> I managed to reproduce your error.
>>
>> It looks like the problem is with the "group" column name: since it
>> is one of the SQL reserved words, you should change it in your csv
>> and update the connector accordingly.
>>
>> Best regards
>>
>> Davide
>>
>> On 6/28/19 11:14 AM, Rainer Herbst wrote:
>>> Hi Davide,
>>>
>>>
>>> thank you for the quick response.
>>> Here is the current connector configuration. The ressource
>>> configuration follows below.
>>>
>>> {
>>> "key" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>>> "adminRealm" : "/AIPTest",
>>> "location" : "file:/opt/syncope/bundles/",
>>> "connectorName" : "net.tirasa.connid.bundles.csvdir.CSVDirConnector",
>>> "bundleName" : "net.tirasa.connid.bundles.csvdir",
>>> "version" : "0.8.8",
>>> "displayName" : "SystemDB",
>>> "connRequestTimeout" : 10,
>>> "poolConf" : {
>>> "maxObjects" : 2000,
>>> "minIdle" : 20,
>>> "maxIdle" : 200,
>>> "maxWait" : 3000,
>>> "minEvictableIdleTimeMillis" : 2000
>>> },
>>> "conf" : [ {
>>> "schema" : {
>>> "name" : "sourcePath",
>>> "displayName" : "Source path",
>>> "helpMessage" : "Absolute path of a directory where the CSV files
>>> to be processed are located",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 1,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "/srv/tomcat/data/" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fileMask",
>>> "displayName" : "File mask",
>>> "helpMessage" : "Regular expression describing files to be
>>> processed",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 2,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ".csv" ]
>>> }, {
>>> "schema" : {
>>> "name" : "encoding",
>>> "displayName" : "File encoding",
>>> "helpMessage" : "Basic encoding of the file",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 3,
>>> "confidential" : false,
>>> "defaultValues" : [ "UTF-8" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "UTF-8" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fieldDelimiter",
>>> "displayName" : "Field delimiter",
>>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>> Default is \",\".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 4,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "," ]
>>> }, {
>>> "schema" : {
>>> "name" : "textQualifier",
>>> "displayName" : "Text qualifier",
>>> "helpMessage" : "Delimiter to determine beginning and end of text
>>> in value. Default is \".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 5,
>>> "confidential" : false,
>>> "defaultValues" : [ "\"" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "'" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyColumnNames",
>>> "displayName" : "Key column name",
>>> "helpMessage" : "Name of the column used to identify user uniquely",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 6,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "uidNumber" ]
>>> }, {
>>> "schema" : {
>>> "name" : "passwordColumnName",
>>> "displayName" : "Password column name",
>>> "helpMessage" : "Name of the column used to specify user password",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 7,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : false,
>>> "values" : [ ]
>>> }, {
>>> "schema" : {
>>> "name" : "deleteColumnName",
>>> "displayName" : "Delete column name",
>>> "helpMessage" : "Name of the column used to specify users to be
>>> deleted",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 8,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : false,
>>> "values" : [ ]
>>> }, {
>>> "schema" : {
>>> "name" : "quotationRequired",
>>> "displayName" : "Value quotation required",
>>> "helpMessage" : "Specify if value quotation is required. Default
>>> is true.",
>>> "type" : "java.lang.Boolean",
>>> "required" : false,
>>> "order" : 9,
>>> "confidential" : false,
>>> "defaultValues" : [ true ]
>>> },
>>> "overridable" : false,
>>> "values" : [ true ]
>>> }, {
>>> "schema" : {
>>> "name" : "fields",
>>> "displayName" : "Column names",
>>> "helpMessage" : "Column names",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 10,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "uidNumber" ]
>>> }, {
>>> "schema" : {
>>> "name" : "ignoreHeader",
>>> "displayName" : "Ignore header",
>>> "helpMessage" : "Specify if first line of the file must be
>>> ignored. Default is true.",
>>> "type" : "java.lang.Boolean",
>>> "required" : false,
>>> "order" : 11,
>>> "confidential" : false,
>>> "defaultValues" : [ true ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyseparator",
>>> "displayName" : "Key separator",
>>> "helpMessage" : "Character used to separate keys in a multi-key
>>> scenario. Default is \",\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 12,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ";" ]
>>> }, {
>>> "schema" : {
>>> "name" : "multivalueSeparator",
>>> "displayName" : "Multi value separator",
>>> "helpMessage" : "Character used to separate values in a
>>> multi-value scenario. Multivalue unsupported if not provided.",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 13,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : false,
>>> "values" : [ ]
>>> }, {
>>> "schema" : {
>>> "name" : "defaultStatusValue",
>>> "displayName" : "Default Status Value",
>>> "helpMessage" : "Enter the value for status in case of status not
>>> specified. Default is \"true\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 14,
>>> "confidential" : false,
>>> "defaultValues" : [ "true" ]
>>> },
>>> "overridable" : false,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "disabledStatusValue",
>>> "displayName" : "Disabled Status Value",
>>> "helpMessage" : "Specify a value for disabled status. Default is
>>> \"false\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 15,
>>> "confidential" : false,
>>> "defaultValues" : [ "false" ]
>>> },
>>> "overridable" : false,
>>> "values" : [ "false" ]
>>> }, {
>>> "schema" : {
>>> "name" : "enabledStatusValue",
>>> "displayName" : "Enable Status Value",
>>> "helpMessage" : "Specify a value for enabled status. Default is
>>> \"true\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 16,
>>> "confidential" : false,
>>> "defaultValues" : [ "true" ]
>>> },
>>> "overridable" : false,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "statusColumn",
>>> "displayName" : "Status Column name",
>>> "helpMessage" : "Status column name.",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 17,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ]
>>> } ],
>>> "capabilities" : [ "SEARCH", "CREATE", "UPDATE" ]
>>> }
>>>
>>> Ressource config:
>>> {
>>> "key" : "Users",
>>> "connector" : "566c864b-9b85-4cdf-ac86-4b9b850cdfe1",
>>> "connectorDisplayName" : "SystemDB",
>>> "orgUnit" : null,
>>> "propagationPriority" : 2,
>>> "randomPwdIfNotProvided" : false,
>>> "enforceMandatoryCondition" : false,
>>> "createTraceLevel" : "ALL",
>>> "updateTraceLevel" : "ALL",
>>> "deleteTraceLevel" : "ALL",
>>> "provisioningTraceLevel" : "ALL",
>>> "passwordPolicy" : "0b17832c-cdb2-4e92-9783-2ccdb23e92d1",
>>> "accountPolicy" : "319350c1-6809-49ed-9350-c1680929eda1",
>>> "pullPolicy" : "ec43ef48-4ba9-4ef0-83ef-484ba9fef083",
>>> "pushPolicy" : "a66afe6e-7e23-44d4-aafe-6e7e2374d4dc",
>>> "overrideCapabilities" : false,
>>> "provisions" : [ {
>>> "key" : "71e72b11-a3a7-4090-a72b-11a3a780902a",
>>> "anyType" : "USER",
>>> "objectClass" : "__ACCOUNT__",
>>> "syncToken" : null,
>>> "ignoreCaseMatch" : false,
>>> "uidOnCreate" : null,
>>> "mapping" : {
>>> "connObjectLink" : null,
>>> "connObjectKeyItem" : {
>>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>>> "intAttrName" : "numericalUID",
>>> "extAttrName" : "uidNumber",
>>> "connObjectKey" : true,
>>> "password" : false,
>>> "mandatoryCondition" : "true",
>>> "purpose" : "PROPAGATION",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> },
>>> "items" : [ {
>>> "key" : "34938fac-168c-4e66-938f-ac168c0e66bd",
>>> "intAttrName" : "numericalUID",
>>> "extAttrName" : "uidNumber",
>>> "connObjectKey" : true,
>>> "password" : false,
>>> "mandatoryCondition" : "true",
>>> "purpose" : "PROPAGATION",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "3ba6be21-82db-4702-a6be-2182dba702b0",
>>> "intAttrName" : "organizationalUnit",
>>> "extAttrName" : "group",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "69557f34-b7cd-4e11-957f-34b7cd1e1167",
>>> "intAttrName" : "creationDate",
>>> "extAttrName" : "createdDatum",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "d55ae9a2-10ca-45c3-9ae9-a210cae5c31f",
>>> "intAttrName" : "displayName",
>>> "extAttrName" : "fullName",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> }, {
>>> "key" : "9ccf8d00-14cb-4085-8f8d-0014cbb08527",
>>> "intAttrName" : "uid",
>>> "extAttrName" : "uid",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "true",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> } ],
>>> "linkingItems" : [ {
>>> "key" : null,
>>> "intAttrName" : "AllUsers",
>>> "extAttrName" : "uid",
>>> "connObjectKey" : false,
>>> "password" : false,
>>> "mandatoryCondition" : "false",
>>> "purpose" : "BOTH",
>>> "propagationJEXLTransformer" : null,
>>> "pullJEXLTransformer" : null,
>>> "transformers" : [ ]
>>> } ]
>>> },
>>> "auxClasses" : [ ],
>>> "virSchemas" : [ "AllUsers" ]
>>> } ],
>>> "confOverride" : [ {
>>> "schema" : {
>>> "name" : "sourcePath",
>>> "displayName" : "Source path",
>>> "helpMessage" : "Absolute path of a directory where the CSV files
>>> to be processed are located",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 1,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "/srv/tomcat/data/" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fileMask",
>>> "displayName" : "File mask",
>>> "helpMessage" : "Regular expression describing files to be
>>> processed",
>>> "type" : "java.lang.String",
>>> "required" : true,
>>> "order" : 2,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "aip-system_users_test.csv" ]
>>> }, {
>>> "schema" : {
>>> "name" : "encoding",
>>> "displayName" : "File encoding",
>>> "helpMessage" : "Basic encoding of the file",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 3,
>>> "confidential" : false,
>>> "defaultValues" : [ "UTF-8" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "UTF-8" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fieldDelimiter",
>>> "displayName" : "Field delimiter",
>>> "helpMessage" : "Delimiter used to separate fields in CSV files.
>>> Default is \",\".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 4,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "," ]
>>> }, {
>>> "schema" : {
>>> "name" : "textQualifier",
>>> "displayName" : "Text qualifier",
>>> "helpMessage" : "Delimiter to determine beginning and end of text
>>> in value. Default is \".",
>>> "type" : "char",
>>> "required" : false,
>>> "order" : 5,
>>> "confidential" : false,
>>> "defaultValues" : [ "\"" ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "'" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyColumnNames",
>>> "displayName" : "Key column name",
>>> "helpMessage" : "Name of the column used to identify user uniquely",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 6,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "uidNumber" ]
>>> }, {
>>> "schema" : {
>>> "name" : "fields",
>>> "displayName" : "Column names",
>>> "helpMessage" : "Column names",
>>> "type" : "[Ljava.lang.String;",
>>> "required" : true,
>>> "order" : 10,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "createdDatum", "createdTime", "uid", "uidNumber",
>>> "gidNumber", "fullName", "office", "phone", "fax", "email",
>>> "homeDir", "host", "group", "shell", "www", "passwd", "termination",
>>> "tutor", "remarks" ]
>>> }, {
>>> "schema" : {
>>> "name" : "ignoreHeader",
>>> "displayName" : "Ignore header",
>>> "helpMessage" : "Specify if first line of the file must be
>>> ignored. Default is true.",
>>> "type" : "java.lang.Boolean",
>>> "required" : false,
>>> "order" : 11,
>>> "confidential" : false,
>>> "defaultValues" : [ true ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "true" ]
>>> }, {
>>> "schema" : {
>>> "name" : "keyseparator",
>>> "displayName" : "Key separator",
>>> "helpMessage" : "Character used to separate keys in a multi-key
>>> scenario. Default is \",\".",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 12,
>>> "confidential" : false,
>>> "defaultValues" : [ "," ]
>>> },
>>> "overridable" : true,
>>> "values" : [ ";" ]
>>> }, {
>>> "schema" : {
>>> "name" : "statusColumn",
>>> "displayName" : "Status Column name",
>>> "helpMessage" : "Status column name.",
>>> "type" : "java.lang.String",
>>> "required" : false,
>>> "order" : 17,
>>> "confidential" : false,
>>> "defaultValues" : [ ]
>>> },
>>> "overridable" : true,
>>> "values" : [ "passwd" ]
>>> } ],
>>> "capabilitiesOverride" : [ "SEARCH", "CREATE", "UPDATE" ],
>>> "propagationActions" : [ ]
>>> }
>>>
>>> Best regards
>>>
>>> On Fri, 28 Jun 2019 10:40:58 +0200
>>> Davide Cortellucci <da...@tirasa.net> wrote:
>>>> Hi Rainer,
>>>>
>>>> can you please share the CSVDir configuration?
>>>>
>>>> Regards,
>>>> Davide
>>>>
>>>> On 6/28/19 10:14 AM, Rainer Herbst wrote:
>>>>> Dear all,
>>>>>
>>>>>
>>>>> for the last 2 weeks I tried to get Syncope running using maven,
>>>>> and at least I have an instance that works somehow.
>>>>>
>>>>> But I stumbled over a lot of problems with CSVDir not working. I
>>>>> created a connector and a ressource, but everytime I try to pull
>>>>> the content of a csv file, the task fails with error
>>>>> org.quartz.JobExecutionException: While pulling from connector
>>>>> [See nested exception:
>>>>> org.identityconnectors.framework.common.exceptions.ConnectorException:
>>>>> org.identityconnectors.framework.common.exceptions.ConnectorIOException:
>>>>> java.lang.IllegalStateException: java.sql.SQLSyntaxErrorException:
>>>>> user lacks privilege or object not found: USER_EX88887 in
>>>>> statement [SELECT * FROM USER_EX88887]]
>>>>>
>>>>> There are some files created in the CSV directory:
>>>>> ldap1:/srv/tomcat/webapps/syncope/WEB-INF/classes # ls -l
>>>>> /srv/tomcat/data/
>>>>> total 24
>>>>> -rw-r--r-- 1 root root 777 Jun 28 09:15 my_users_test.csv
>>>>> -rw-r--r-- 1 tomcat tomcat 16 Jun 28 10:05 csvdir_db.lck
>>>>> -rw-r--r-- 1 tomcat tomcat 1608 Jun 28 09:45 csvdir_db.log
>>>>> -rw-r--r-- 1 tomcat tomcat 101 Jun 28 09:42 csvdir_db.properties
>>>>> -rw-r--r-- 1 tomcat tomcat 1559 Jun 28 09:42 csvdir_db.script
>>>>> drwxr-xr-x 2 tomcat tomcat 4096 Jun 28 09:09 csvdir_db.tmp
>>>>>
>>>>> What is missing there?
>>>>>
>>>>> I like the openess and the concepts of Syncope, but, to be honest,
>>>>> I did not expect to spend 2 days trying to read a simple CSV data
>>>>> file... :-(
>>>>>
>>>>> Help is very much appreciated!
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/