You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "CheneyYin (via GitHub)" <gi...@apache.org> on 2023/07/12 00:21:58 UTC

[GitHub] [seatunnel] CheneyYin commented on issue #4599: Seatunnel with Spark: Caused by: java.lang.OutOfMemoryError: Java heap space

CheneyYin commented on issue #4599:
URL: https://github.com/apache/seatunnel/issues/4599#issuecomment-1631673938

   > I am having a similar failure with ver.2.3.2, under java 17.0.7 2023-04-18 LTS.
   > 
   > ```
   > 2023-07-11 15:47:14,211 WARN  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-495998] [5.1] Exception in org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask@67644eb2
   > java.lang.OutOfMemoryError: Java heap space
   > ...
   > 2023-07-11 15:47:12,347 ERROR org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (731233925082382337), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-JDBC-default-identifier]-SourceTask (1/2)] end with state FAILED and Exception: java.lang.OutOfMemoryError: Java heap space
   > ```
   > 
   > My config is:
   > ```
   > env {
   >   # You can set SeaTunnel environment configuration here
   >   execution.parallelism = 2
   >   job.mode = "BATCH"
   >   checkpoint.interval = 10000
   >   #execution.checkpoint.interval = 10000
   >   #execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
   > }
   > 
   > source {
   > 
   >   JDBC {
   >     url = "jdbc:mysql:<>"
   >     driver = "com.mysql.cj.jdbc.Driver"
   >     user = <>
   >     password = <>
   >     query = "SELECT from <>"
   >     parallelism = 2
   >     fetch_size = 500
   >     }
   > }
   > sink {
   >   localFile {
   >     path="test_log"
   >     file_format_type="parquet"
   >     }
   > }
   > ```
   > The expected parquet folder is not being created.
   > 
   
   Do you have the same problem when using the spark engine?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org