Getting RDD load progress

["cyril.ponomaryov" <cy...@gmail.com>]

I have an ImpalaRDD class: 

public class ImpalaRDD<T> extends RDD<T> { 
    ... 
    @Override 
    public Iterator<T> compute(Partition partition, TaskContext context) { 
        ImpalaPartitionIterator<T> iterator = new
ImpalaPartitionIterator<>(mapRow, (JdbcPartition) partition,
connectionProvider, partitioner, context); 
        TOperationHandle handle = getOperationHandle(iterator.statement); 
        return asScalaIterator(iterator); 
    } 
    ... 
} 

Impala query operation handle can be used to track its progress. How to
share this handle to Spark driver?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Getting-RDD-load-progress-tp14797.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org