You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemml.apache.org by Arvind Surve <ac...@yahoo.com.INVALID> on 2017/07/10 18:59:00 UTC
Performance test issues
Hi Krishna,
I am trying to run perftest using following command on Spark cluster. I am wondering if this was ever tried on HDFS cluster, or I am giving wrong syntax. I see following output, files with json files gets created in local env. Where to search for log/error message.
]$ ./run_perftest.py --family binomial --exec-type hybrid_spark --temp-dir hdfs:///user/biuser/asurve --filename perf_sys_1 --mode data-gen
binomial,data-gen,none,dense,10k_100,time_not_found
binomial,data-gen,none,sparse,10k_100,time_not_found
-Arvind
Arvind Surve | Spark Technology Center | http://www.spark.tc/
Re: Performance test issues
Posted by Krishna Kalyan <kr...@gmail.com>.
Hello Arvind,
Our performance tests have not been tested with HDFS. I am currently
working on adding this support.
Regards,
Krishna
On Tue, Jul 11, 2017 at 3:46 PM, Krishna Kalyan <kr...@gmail.com>
wrote:
> Hello Arvind,
> Thanks for testing these scripts. I will look into this as soon as
> possible.
>
> Regards,
> Krishna
>
> On Mon, Jul 10, 2017 at 2:59 PM, Arvind Surve <ac...@yahoo.com.invalid>
> wrote:
>
>> Hi Krishna,
>> I am trying to run perftest using following command on Spark cluster. I
>> am wondering if this was ever tried on HDFS cluster, or I am giving wrong
>> syntax. I see following output, files with json files gets created in local
>> env. Where to search for log/error message.
>>
>> ]$ ./run_perftest.py --family binomial --exec-type hybrid_spark
>> --temp-dir hdfs:///user/biuser/asurve --filename perf_sys_1 --mode data-gen
>> binomial,data-gen,none,dense,10k_100,time_not_found
>> binomial,data-gen,none,sparse,10k_100,time_not_found
>>
>> -Arvind
>> Arvind Surve | Spark Technology Center | http://www.spark.tc/
>
>
>
Re: Performance test issues
Posted by Krishna Kalyan <kr...@gmail.com>.
Hello Arvind,
Thanks for testing these scripts. I will look into this as soon as possible.
Regards,
Krishna
On Mon, Jul 10, 2017 at 2:59 PM, Arvind Surve <ac...@yahoo.com.invalid>
wrote:
> Hi Krishna,
> I am trying to run perftest using following command on Spark cluster. I am
> wondering if this was ever tried on HDFS cluster, or I am giving wrong
> syntax. I see following output, files with json files gets created in local
> env. Where to search for log/error message.
>
> ]$ ./run_perftest.py --family binomial --exec-type hybrid_spark --temp-dir
> hdfs:///user/biuser/asurve --filename perf_sys_1 --mode data-gen
> binomial,data-gen,none,dense,10k_100,time_not_found
> binomial,data-gen,none,sparse,10k_100,time_not_found
>
> -Arvind
> Arvind Surve | Spark Technology Center | http://www.spark.tc/