You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hudi.apache.org by "mayu1@bonc.com.cn" <ma...@bonc.com.cn> on 2019/12/26 07:08:31 UTC
insert too slow
Hello!
What is the throughput of Hudi? I currently use spark to insert 10,000 records (300 bytes each), which takes one minute. Is it too slow?
my program:
import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkConf, SparkContext}
object HudiDataGen {
def main(args: Array[String]): Unit = {
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
//初始化
val conf = new SparkConf().setAppName("HudiTest")
.setMaster("local[*]")
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
val sc = new SparkContext(conf)
val spark = new SQLContext(sc)
//设置表名、基本路径和数据生成器来为本指南生成记录。
val tableName = Constant.tableName
// val basePath = Constant.hdfsPath
val basePath = args(0)
// val basePath = "file:///e:/hudi_cow_table"
val count = args(1)
for (i <- 1 to count.toInt) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
//生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
val inserts = convertToStringList(dataGen.generateInserts(10000))
// println("insert:"+System.currentTimeMillis())
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
// option(STORAGE_TYPE_OPT_KEY, "MERGE_ON_READ").
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
}
}
mayu1@bonc.com.cn
Re: Re: insert too slow
Posted by "mayu1@bonc.com.cn" <ma...@bonc.com.cn>.
Hello
I found that both spark-shell and spark-submit are slow. It takes 1 minute to insert 10,000 rows into a 10 million row dataset. Is this normal?
mayu1@bonc.com.cn
From: lamberken
Date: 2019-12-26 16:18
To: dev@hudi.apache.org
Subject: Re: insert too slow
Hi @mayu1,
Here is Tuning Guide[1], it may help you to improve performance.
[1]https://cwiki.apache.org/confluence/display/HUDI/Tuning+Guide
best,
lamber-kem
On 12/26/2019 15:46，mayu1@bonc.com.cn<ma...@bonc.com.cn> wrote：
Thank you for your reply, it really works. And how to insert 100 million records without OOM?
mayu1@bonc.com.cn
From: lamberken
Date: 2019-12-26 15:33
To: dev@hudi.apache.org
Subject: Re:insert too slow
Hi @mayu1,
Can you run the below program in cosole? looking forward to your feedback.
========================================================================
${SPARK_HOME}/bin/spark-shell \
--packages org.apache.hudi:hudi-spark-bundle:0.5.0-incubating \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
val tableName = "tableName"
val basePath = "file:///tmp/data"
for (i <- 1 to 1) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
val inserts = convertToStringList(dataGen.generateInserts(10000))
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
========================================================================
best,
lamber-ken
On 12/26/2019 15:08，mayu1@bonc.com.cn<ma...@bonc.com.cn> wrote：
Hello!
What is the throughput of Hudi? I currently use spark to insert 10,000 records (300 bytes each), which takes one minute. Is it too slow?
my program:
import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkConf, SparkContext}
object HudiDataGen {
def main(args: Array[String]): Unit = {
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
//初始化
val conf = new SparkConf().setAppName("HudiTest")
.setMaster("local[*]")
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
val sc = new SparkContext(conf)
val spark = new SQLContext(sc)
//设置表名、基本路径和数据生成器来为本指南生成记录。
val tableName = Constant.tableName
// val basePath = Constant.hdfsPath
val basePath = args(0)
// val basePath = "file:///e:/hudi_cow_table"
val count = args(1)
for (i <- 1 to count.toInt) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
//生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
val inserts = convertToStringList(dataGen.generateInserts(10000))
// println("insert:"+System.currentTimeMillis())
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
// option(STORAGE_TYPE_OPT_KEY, "MERGE_ON_READ").
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
}
}
mayu1@bonc.com.cn
Re: insert too slow
Posted by lamberken <la...@163.com>.
Hi @mayu1,
Here is Tuning Guide[1], it may help you to improve performance.
[1]https://cwiki.apache.org/confluence/display/HUDI/Tuning+Guide
best,
lamber-kem
On 12/26/2019 15:46，mayu1@bonc.com.cn<ma...@bonc.com.cn> wrote：
Thank you for your reply, it really works. And how to insert 100 million records without OOM?
mayu1@bonc.com.cn
From: lamberken
Date: 2019-12-26 15:33
To: dev@hudi.apache.org
Subject: Re:insert too slow
Hi @mayu1,
Can you run the below program in cosole? looking forward to your feedback.
========================================================================
${SPARK_HOME}/bin/spark-shell \
--packages org.apache.hudi:hudi-spark-bundle:0.5.0-incubating \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
val tableName = "tableName"
val basePath = "file:///tmp/data"
for (i <- 1 to 1) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
val inserts = convertToStringList(dataGen.generateInserts(10000))
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
========================================================================
best,
lamber-ken
On 12/26/2019 15:08，mayu1@bonc.com.cn<ma...@bonc.com.cn> wrote：
Hello!
What is the throughput of Hudi? I currently use spark to insert 10,000 records (300 bytes each), which takes one minute. Is it too slow?
my program:
import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkConf, SparkContext}
object HudiDataGen {
def main(args: Array[String]): Unit = {
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
//初始化
val conf = new SparkConf().setAppName("HudiTest")
.setMaster("local[*]")
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
val sc = new SparkContext(conf)
val spark = new SQLContext(sc)
//设置表名、基本路径和数据生成器来为本指南生成记录。
val tableName = Constant.tableName
// val basePath = Constant.hdfsPath
val basePath = args(0)
// val basePath = "file:///e:/hudi_cow_table"
val count = args(1)
for (i <- 1 to count.toInt) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
//生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
val inserts = convertToStringList(dataGen.generateInserts(10000))
// println("insert:"+System.currentTimeMillis())
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
// option(STORAGE_TYPE_OPT_KEY, "MERGE_ON_READ").
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
}
}
mayu1@bonc.com.cn
Re: Re:insert too slow
Posted by "mayu1@bonc.com.cn" <ma...@bonc.com.cn>.
Thank you for your reply, it really works. And how to insert 100 million records without OOM?
mayu1@bonc.com.cn
From: lamberken
Date: 2019-12-26 15:33
To: dev@hudi.apache.org
Subject: Re:insert too slow
Hi @mayu1,
Can you run the below program in cosole? looking forward to your feedback.
========================================================================
${SPARK_HOME}/bin/spark-shell \
--packages org.apache.hudi:hudi-spark-bundle:0.5.0-incubating \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
val tableName = "tableName"
val basePath = "file:///tmp/data"
for (i <- 1 to 1) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
val inserts = convertToStringList(dataGen.generateInserts(10000))
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
========================================================================
best,
lamber-ken
On 12/26/2019 15:08，mayu1@bonc.com.cn<ma...@bonc.com.cn> wrote：
Hello!
What is the throughput of Hudi? I currently use spark to insert 10,000 records (300 bytes each), which takes one minute. Is it too slow?
my program:
import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkConf, SparkContext}
object HudiDataGen {
def main(args: Array[String]): Unit = {
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
//初始化
val conf = new SparkConf().setAppName("HudiTest")
.setMaster("local[*]")
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
val sc = new SparkContext(conf)
val spark = new SQLContext(sc)
//设置表名、基本路径和数据生成器来为本指南生成记录。
val tableName = Constant.tableName
// val basePath = Constant.hdfsPath
val basePath = args(0)
// val basePath = "file:///e:/hudi_cow_table"
val count = args(1)
for (i <- 1 to count.toInt) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
//生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
val inserts = convertToStringList(dataGen.generateInserts(10000))
// println("insert:"+System.currentTimeMillis())
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
// option(STORAGE_TYPE_OPT_KEY, "MERGE_ON_READ").
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
}
}
mayu1@bonc.com.cn
Re:insert too slow
Posted by lamberken <la...@163.com>.
Hi @mayu1,
Can you run the below program in cosole? looking forward to your feedback.
========================================================================
${SPARK_HOME}/bin/spark-shell \
--packages org.apache.hudi:hudi-spark-bundle:0.5.0-incubating \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
val tableName = "tableName"
val basePath = "file:///tmp/data"
for (i <- 1 to 1) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
val inserts = convertToStringList(dataGen.generateInserts(10000))
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
========================================================================
best,
lamber-ken
On 12/26/2019 15:08，mayu1@bonc.com.cn<ma...@bonc.com.cn> wrote：
Hello!
What is the throughput of Hudi? I currently use spark to insert 10,000 records (300 bytes each), which takes one minute. Is it too slow?
my program:
import org.apache.spark.sql.SQLContext
import org.apache.spark.{SparkConf, SparkContext}
object HudiDataGen {
def main(args: Array[String]): Unit = {
import org.apache.hudi.DataSourceWriteOptions._
import org.apache.hudi.QuickstartUtils._
import org.apache.hudi.config.HoodieWriteConfig._
import org.apache.spark.sql.SaveMode._
import scala.collection.JavaConversions._
//初始化
val conf = new SparkConf().setAppName("HudiTest")
.setMaster("local[*]")
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
val sc = new SparkContext(conf)
val spark = new SQLContext(sc)
//设置表名、基本路径和数据生成器来为本指南生成记录。
val tableName = Constant.tableName
// val basePath = Constant.hdfsPath
val basePath = args(0)
// val basePath = "file:///e:/hudi_cow_table"
val count = args(1)
for (i <- 1 to count.toInt) {
println("start:" + System.currentTimeMillis())
val dataGen = new DataGenerator
//生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
val inserts = convertToStringList(dataGen.generateInserts(10000))
// println("insert:"+System.currentTimeMillis())
println("start insert:" + System.currentTimeMillis())
val df = spark.read.json(spark.sparkContext.parallelize(inserts, 32))
df.write.format("org.apache.hudi").
options(getQuickstartWriteConfigs).
option(PRECOMBINE_FIELD_OPT_KEY, "ts").
option(RECORDKEY_FIELD_OPT_KEY, "uuid").
option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
option(TABLE_NAME, tableName).
// option(STORAGE_TYPE_OPT_KEY, "MERGE_ON_READ").
mode(Append).
save(basePath);
println("finish" + i + " " + System.currentTimeMillis())
}
}
}
mayu1@bonc.com.cn
Re: insert too slow
Posted by Vinoth Chandar <vi...@apache.org>.
Hi,
If you could share your Spark UI or the stage that is taking more time, we
can further drill down.
Overall, this FAQ entry also sets expectations and provides some data
modelling tips
https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-Whatperformance/ingestlatencycanIexpectforHudiwriting
thanks
vinoth
On Thu, Dec 26, 2019 at 12:38 PM hamid pirahesh <hp...@gmail.com> wrote:
> First you need to determine where the bottleneck is. You have Spark and
> Hudi. If Spark is the bottleneck, fix that first.
> Regarding high speed processing with Hudi look at:
> https://eng.uber.com/marmaray-hadoop-ingestion-open-source/
> https://github.com/uber/marmaray
>
> They are doing 100B messages per day for Uber Eats and Uber Freight.
>
>
> Hamid Pirahesh
>
>
>
>
> On Wed, Dec 25, 2019 at 11:08 PM mayu1@bonc.com.cn <ma...@bonc.com.cn>
> wrote:
>
> > Hello!
> > What is the throughput of Hudi? I currently use spark to insert 10,000
> > records (300 bytes each), which takes one minute. Is it too slow?
> > my program:
> > import org.apache.spark.sql.SQLContext
> > import org.apache.spark.{SparkConf, SparkContext}
> >
> > object HudiDataGen {
> > def main(args: Array[String]): Unit = {
> > import org.apache.hudi.DataSourceWriteOptions._
> > import org.apache.hudi.QuickstartUtils._
> > import org.apache.hudi.config.HoodieWriteConfig._
> > import org.apache.spark.sql.SaveMode._
> >
> > import scala.collection.JavaConversions._
> >
> > //初始化
> > val conf = new SparkConf().setAppName("HudiTest")
> > .setMaster("local[*]")
> > conf.set("spark.serializer",
> > "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
> > val sc = new SparkContext(conf)
> > val spark = new SQLContext(sc)
> >
> > //设置表名、基本路径和数据生成器来为本指南生成记录。
> > val tableName = Constant.tableName
> > // val basePath = Constant.hdfsPath
> > val basePath = args(0)
> > // val basePath = "file:///e:/hudi_cow_table"
> > val count = args(1)
> > for (i <- 1 to count.toInt) {
> > println("start:" + System.currentTimeMillis())
> > val dataGen = new DataGenerator
> > //生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
> > val inserts =
> > convertToStringList(dataGen.generateInserts(10000))
> > // println("insert:"+System.currentTimeMillis())
> > println("start insert:" + System.currentTimeMillis())
> > val df =
> > spark.read.json(spark.sparkContext.parallelize(inserts, 32))
> > df.write.format("org.apache.hudi").
> > options(getQuickstartWriteConfigs).
> > option(PRECOMBINE_FIELD_OPT_KEY, "ts").
> > option(RECORDKEY_FIELD_OPT_KEY, "uuid").
> > option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
> > option(TABLE_NAME, tableName).
> > // option(STORAGE_TYPE_OPT_KEY,
> > "MERGE_ON_READ").
> > mode(Append).
> > save(basePath);
> > println("finish" + i + " " + System.currentTimeMillis())
> > }
> >
> > }
> > }
> >
> >
> > mayu1@bonc.com.cn
> >
>
Re: insert too slow
Posted by hamid pirahesh <hp...@gmail.com>.
First you need to determine where the bottleneck is. You have Spark and
Hudi. If Spark is the bottleneck, fix that first.
Regarding high speed processing with Hudi look at:
https://eng.uber.com/marmaray-hadoop-ingestion-open-source/
https://github.com/uber/marmaray
They are doing 100B messages per day for Uber Eats and Uber Freight.
Hamid Pirahesh
On Wed, Dec 25, 2019 at 11:08 PM mayu1@bonc.com.cn <ma...@bonc.com.cn>
wrote:
> Hello!
> What is the throughput of Hudi? I currently use spark to insert 10,000
> records (300 bytes each), which takes one minute. Is it too slow?
> my program:
> import org.apache.spark.sql.SQLContext
> import org.apache.spark.{SparkConf, SparkContext}
>
> object HudiDataGen {
> def main(args: Array[String]): Unit = {
> import org.apache.hudi.DataSourceWriteOptions._
> import org.apache.hudi.QuickstartUtils._
> import org.apache.hudi.config.HoodieWriteConfig._
> import org.apache.spark.sql.SaveMode._
>
> import scala.collection.JavaConversions._
>
> //初始化
> val conf = new SparkConf().setAppName("HudiTest")
> .setMaster("local[*]")
> conf.set("spark.serializer",
> "org.apache.spark.serializer.KryoSerializer") //使用Kryo序列化库
> val sc = new SparkContext(conf)
> val spark = new SQLContext(sc)
>
> //设置表名、基本路径和数据生成器来为本指南生成记录。
> val tableName = Constant.tableName
> // val basePath = Constant.hdfsPath
> val basePath = args(0)
> // val basePath = "file:///e:/hudi_cow_table"
> val count = args(1)
> for (i <- 1 to count.toInt) {
> println("start:" + System.currentTimeMillis())
> val dataGen = new DataGenerator
> //生成一些新的行程样本，将其加载到DataFrame中，然后将DataFrame写入Hudi数据集中，如下所示。
> val inserts =
> convertToStringList(dataGen.generateInserts(10000))
> // println("insert:"+System.currentTimeMillis())
> println("start insert:" + System.currentTimeMillis())
> val df =
> spark.read.json(spark.sparkContext.parallelize(inserts, 32))
> df.write.format("org.apache.hudi").
> options(getQuickstartWriteConfigs).
> option(PRECOMBINE_FIELD_OPT_KEY, "ts").
> option(RECORDKEY_FIELD_OPT_KEY, "uuid").
> option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
> option(TABLE_NAME, tableName).
> // option(STORAGE_TYPE_OPT_KEY,
> "MERGE_ON_READ").
> mode(Append).
> save(basePath);
> println("finish" + i + " " + System.currentTimeMillis())
> }
>
> }
> }
>
>
> mayu1@bonc.com.cn
>