Spark on teradata?

[gen tang <ge...@gmail.com>]

Hi,

I have a stupid question:
Is it possible to use spark on Teradata data warehouse, please? I read some
news on internet which say yes. However, I didn't find any example about
this issue

Thanks in advance.

Cheers
Gen

Re: Spark on teradata?

["Evan R. Sparks" <ev...@gmail.com>]

Have you taken a look at the TeradataDBInputFormat? Spark is compatible
with arbitrary hadoop input formats - so this might work for you:
http://developer.teradata.com/extensibility/articles/hadoop-mapreduce-connector-to-teradata-edw

On Thu, Jan 8, 2015 at 10:53 AM, gen tang <ge...@gmail.com> wrote:

> Thanks a lot for your reply.
> In fact, I need to work on almost all the data in teradata (~100T). So, I
> don't think that jdbcRDD is a good choice.
>
> Cheers
> Gen
>
>
> On Thu, Jan 8, 2015 at 7:39 PM, Reynold Xin <rx...@databricks.com> wrote:
>
>> Depending on your use cases. If the use case is to extract small amount
>> of data out of teradata, then you can use the JdbcRDD and soon a jdbc input
>> source based on the new Spark SQL external data source API.
>>
>>
>>
>> On Wed, Jan 7, 2015 at 7:14 AM, gen tang <ge...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have a stupid question:
>>> Is it possible to use spark on Teradata data warehouse, please? I read
>>> some news on internet which say yes. However, I didn't find any example
>>> about this issue
>>>
>>> Thanks in advance.
>>>
>>> Cheers
>>> Gen
>>>
>>>
>>
>

Re: Spark on teradata?

[gen tang <ge...@gmail.com>]

Thanks a lot for your reply.
In fact, I need to work on almost all the data in teradata (~100T). So, I
don't think that jdbcRDD is a good choice.

Cheers
Gen


On Thu, Jan 8, 2015 at 7:39 PM, Reynold Xin <rx...@databricks.com> wrote:

> Depending on your use cases. If the use case is to extract small amount of
> data out of teradata, then you can use the JdbcRDD and soon a jdbc input
> source based on the new Spark SQL external data source API.
>
>
>
> On Wed, Jan 7, 2015 at 7:14 AM, gen tang <ge...@gmail.com> wrote:
>
>> Hi,
>>
>> I have a stupid question:
>> Is it possible to use spark on Teradata data warehouse, please? I read
>> some news on internet which say yes. However, I didn't find any example
>> about this issue
>>
>> Thanks in advance.
>>
>> Cheers
>> Gen
>>
>>
>

Re: Spark on teradata?

[Reynold Xin <rx...@databricks.com>]

Depending on your use cases. If the use case is to extract small amount of
data out of teradata, then you can use the JdbcRDD and soon a jdbc input
source based on the new Spark SQL external data source API.



On Wed, Jan 7, 2015 at 7:14 AM, gen tang <ge...@gmail.com> wrote:

> Hi,
>
> I have a stupid question:
> Is it possible to use spark on Teradata data warehouse, please? I read
> some news on internet which say yes. However, I didn't find any example
> about this issue
>
> Thanks in advance.
>
> Cheers
> Gen
>
>

Re: Spark on teradata?

[Reynold Xin <rx...@databricks.com>]

Depending on your use cases. If the use case is to extract small amount of
data out of teradata, then you can use the JdbcRDD and soon a jdbc input
source based on the new Spark SQL external data source API.



On Wed, Jan 7, 2015 at 7:14 AM, gen tang <ge...@gmail.com> wrote:

> Hi,
>
> I have a stupid question:
> Is it possible to use spark on Teradata data warehouse, please? I read
> some news on internet which say yes. However, I didn't find any example
> about this issue
>
> Thanks in advance.
>
> Cheers
> Gen
>
>

Re: Spark on teradata?

[xhudik <xh...@gmail.com>]

I don't think this makes sense. TD database is standard RDBMS (even parallel)
while Spark is used for non-relational issues. 
What could make sense is to deploy Spark on Teradata Aster. Aster is a
database cluster that might call external programs via STREAM operator. 
That said Spark/Scala app can be can be called and process some data. The
deployment itself should be easy the potential benefit - hard to say...


hope this helps, Tomas



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-on-teradata-tp10025p10042.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org