[GitHub] [incubator-seatunnel] dik111 commented on issue #1929: [Feature]Use hive-sink to save table as ORCFileFormat

[GitBox <gi...@apache.org>]

dik111 commented on issue #1929:
URL: https://github.com/apache/incubator-seatunnel/issues/1929#issuecomment-1138072612

   I had tried to change the source code about spark hive sink, but it doesn't work
   ``` scala
   # Hive.scala
   ...
   val frameWriter: DataFrameWriter[Row] = if (config.hasPath("save_mode")) {
           sinkFrame.write.format("orc").mode(config.getString("save_mode"))
         } else {
           sinkFrame.format("orc").write
         }
   ...
   ```
   And then I found use hive sql can help me solve this problem - -!
   ```
   env {
   	    spark.streaming.batchDuration = 5
   	    spark.app.name = "seatunnel-app-mysql_hive_test.user_info"
   	    spark.sql.catalogImplementation = "hive"
   	    
   	
   	}
   	source {
   	    jdbc {
   	        driver = "com.mysql.jdbc.Driver"
   	        url = "jdbc:mysql://xxx:3306/test?characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8&allowMultiQueries=true"
   	        table = "  (select id AS id , name AS name , sex AS sex , update_time AS update_time , create_time AS create_time , update_time244 AS update_time244 , day1234 AS day1234 , ts AS ts from test.user_info ) tmp"
   	        result_table_name = "source_table"
   	        user = "bigdata"
   	        password = "***"
   	            }
   	}
   	sink {
   	    Hive {
   	        sql = "insert into test.user_info select * from source_table"
   	    }
   	}
   	transform{}
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org