You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@drill.apache.org by yousuf <yo...@css.org.sa> on 2017/01/25 15:14:48 UTC

Data archiving script

Dear Khalid

Please find  below the link for data archiving project which is uploaded 
on git <http://git.css.org/yousufuddin/Data-Archiving/tree/master>

Kindly make sure that you have accessible shared directory to archive 
the data from the server where you intent to run this spark job.


The parameters are as follows.
Usage: data_archiving2.py <kafka brokers list> <kafka topic> < file 
path> <DBHOST> <DBPORT> <DBUSER> <DBPASS> <DBNAME>

example :

bin/spark-submit --master local[*]  --jars 
external/kafka-assembly/target/scala-2.10/spark-streaming-kafka-assembly-1.6.1.jar 
/home/user/sparkstreaming_example/git_code/Data-Archiving/data_archiving2.py 
kafka-b01.css.org:9092,kafka-b02.css.org:9092,kafka-b03.css.org:9092,kafka-b04.css.org:9092,kafka-b05.css.org:9092,kafka-b06.css.org:9092,kafka-b05.css.org:9092 
gnip_live_stream /home/user/arc_test/ 10.62.84.30 3306 dev xxxx twitter

Kind Regards
Yousuf