[jira] [Commented] (GRIFFIN-338) Custom Connector for mysql not working

["Wan Kun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GRIFFIN-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173046#comment-17173046 ] 

Wan Kun commented on GRIFFIN-338:
---------------------------------

Hi, [~ishanverma]

Could you provide your POC error log messages for troubleshooting?

I recommend that you compile Griffin from the master branch and try again.

Here is an example of Griffin measure for  mysql source connector that works.
{code:java}
{
  "name": "accu_batch",
  "process.type": "batch",
  "data.sources": [
    {
      "name": "source",
      "baseline": true,
      "connector": {
        "type": "jdbc",
        "config": {
          "user": "xxx",
          "password": "xxx",
          "tablename":  "stu",
          "where": "id < 3",
          "url":"jdbc:mysql://localhost:3306/test",
          "database": "test",
          "driver": "com.mysql.jdbc.Driver"
        }
      }
    },
    {
      "name": "target",
      "connector": {
        "type": "jdbc",
        "config": {
          "user": "xxx",
          "password": "xxx",
          "tablename":  "stu2",
          "where": "id < 3",
          "url":"jdbc:mysql://localhost:3306/test",
          "database": "test",
          "driver": "com.mysql.jdbc.Driver"
        }
      }
    }
  ],
  "evaluate.rule": {
    "rules": [
      {
        "dsl.type": "griffin-dsl",
        "dq.type": "accuracy",
        "out.dataframe.name": "accu",
        "rule": "source.id = target.id AND upper(source.name) = upper(target.name) ",
        "details": {
          "source": "source",
          "target": "target",
          "miss": "miss_count",
          "total": "total_count",
          "matched": "matched_count"
        },
        "out": [
          {
            "type": "record",
            "name": "missRecords"
          }
        ]
      }
    ]
  },
  "sinks": [
    "consoleSink"
  ]
}
{code}

> Custom Connector for mysql not working
> --------------------------------------
>
>                 Key: GRIFFIN-338
>                 URL: https://issues.apache.org/jira/browse/GRIFFIN-338
>             Project: Griffin
>          Issue Type: Bug
>         Environment: griffin version= 0.5, mysql on EC2 instance  using griffin docker image
>            Reporter: ishan verma
>            Priority: Critical
>         Attachments: image-2020-07-29-20-23-40-838.png
>
>
> Hi Team,
> I am currently working on data quality POC using griffin.
> So far till now everything is working fine using HIVE  as a data source, but there is new requirement to add mysql as source.
>  
> I have tried every possible way to have *mysql* as custom data connector but its not working. Measure is getting created and job is going to successful but griffin showing *NO CONTENT* on ui . below is my code for that:-
> "data.sources": [
>     {
>       "name": "source",
>       "connectors": [
>         {
>         "name": "source1595488803031",  
>         "type": "CUSTOM",
>         "data.unit": "1day",
>         "data.time.zone": "",
>         "config": {
>           "class": "org.apache.griffin.measure.datasource.connector.batch.MySqlDataConnector",
>           "database": "griffin_poc",
>           "tablename": "person_src",
>           "url": "jdbc:mysql://griffin:3306/griffin_poc",
>           "user": "test_u",
>           "password": "test_p",
>           "driver": "com.mysql.jdbc.Driver"
>         }
>         }
> can anyone please provide some valuable suggestions on this , how to use mysql as my datasource as it is very critical for my POC. Its an urgent issue.
> Anything i am missing  here to link with  mysql, please guide me through this.
>  
> Also i have setup mysql on ec2 instance using griffin docker image.
>  
>  
> Any leads will be appreciated.
> Thanks:)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)