[GitHub] [incubator-seatunnel] laglangyue opened a new issue, #2521: [Bug] [Core] ArrayType convert for spark2.4.0 and improve the convert method name

[GitBox <gi...@apache.org>]

laglangyue opened a new issue, #2521:
URL: https://github.com/apache/incubator-seatunnel/issues/2521

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
   
   
   ### What happened
   
   I am testing the type Mapper，sparkExample-v2，the ArrayType will error
   ![Uploading image.png…]()
   
   
   ### SeaTunnel Version
   
   dev
   
   ### SeaTunnel Config
   
   ```conf
   seatunnel-examples/seatunnel-spark-connector-v2-example/src/main/resources/examples/spark.batch.conf
   
   
   
   env {
     # You can set spark configuration here
     # see available properties defined by spark: https://spark.apache.org/docs/latest/configuration.html#available-properties
     #job.mode = BATCH
     spark.app.name = "SeaTunnel"
     spark.executor.instances = 2
     spark.executor.cores = 1
     spark.executor.memory = "1g"
     spark.master = local
   }
   
   source {
     # This is a example input plugin **only for test and demonstrate the feature input plugin**
     FakeSource {
       schema = {
         fields {
           c_map = "map<string, string>"
           c_array = "array<tinyint>"
           c_string = string
           c_boolean = boolean
           c_tinyint = tinyint
           c_smallint = smallint
           c_int = int
           c_bigint = bigint
           c_float = float
           c_double = double
           c_decimal = "decimal(30, 8)"
           c_null = "null"
           c_bytes = bytes
           c_date = date
           c_time = time
           c_timestamp = timestamp
         }
       }
       result_table_name = "fake"
     }
   
     # You can also use other input plugins, such as hdfs
     # hdfs {
     #   result_table_name = "accesslog"
     #   path = "hdfs://hadoop-cluster-01/nginx/accesslog"
     #   format = "json"
     # }
   
     # If you would like to get more information about how to configure seatunnel and see full list of input plugins,
     # please go to https://seatunnel.apache.org/docs/category/source-v2
   }
   
   transform {
     # split data by specific delimiter
   
     # you can also use other transform plugins, such as sql
     sql {
       sql = "select c_array,c_string,c_boolean,c_tinyint,c_smallint,c_int,c_bigint,c_float,c_double,c_null,c_bytes from fake"
       result_table_name = "sql"
     }
   
   
   }
   
   sink {
     # choose stdout output plugin to output data to console
     Console {}
   
     # you can also you other output plugins, such as sql
     # hdfs {
     #   path = "hdfs://hadoop-cluster-01/nginx/accesslog_processed"
     #   save_mode = "append"
     # }
   
   }
   
   ```
   ```
   
   
   ### Running Command
   
   ```shell
   idea run
   ```
   
   
   ### Error Exception
   
   ```log
   can't cast to ArrayData
   ```
   
   
   ### Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   _No response_
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] laglangyue closed issue #2521: [Bug] [Core] ArrayType convert for spark2.4.0 and improve the convert method name

[GitBox <gi...@apache.org>]

laglangyue closed issue #2521: [Bug] [Core] ArrayType convert for spark2.4.0 and improve the convert method name
URL: https://github.com/apache/incubator-seatunnel/issues/2521


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-seatunnel] laglangyue commented on issue #2521: [Bug] [Core] ArrayType convert for spark2.4.0 and improve the convert method name

[GitBox <gi...@apache.org>]

laglangyue commented on issue #2521:
URL: https://github.com/apache/incubator-seatunnel/issues/2521#issuecomment-1226671260

   please see #2518


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org