You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Kayode Odeyemi <dr...@gmail.com> on 2015/10/22 21:31:13 UTC

Fwd: sqlContext load by offset

Hi,

I've trying to load a postgres table using the following expression:

val cachedIndex = cache.get("latest_legacy_group_index")
val mappingsDF = sqlContext.load("jdbc", Map(
  "url" -> Config.dataSourceUrl(mode, Some("mappings")),
  "dbtable" -> s"(select userid, yid, username from legacyusers offset
$cachedIndex ) as legacyusers")
)

I'll like to know if this expression is correct:

"dbtable" -> s"(select userid, yid, username from legacyusers offset
$cachedIndex ) as legacyusers")

As you can see. I'm trying to load the table records by offset

I appreciate your help.

Re: sqlContext load by offset

Posted by Kayode Odeyemi <dr...@gmail.com>.

When I use that I get a "Caused by: org.postgresql.util.PSQLException:
ERROR: column "none" does not exist"

On Thu, Oct 22, 2015 at 9:31 PM, Kayode Odeyemi <dr...@gmail.com> wrote:

> Hi,
>
> I've trying to load a postgres table using the following expression:
>
> val cachedIndex = cache.get("latest_legacy_group_index")
> val mappingsDF = sqlContext.load("jdbc", Map(
>   "url" -> Config.dataSourceUrl(mode, Some("mappings")),
>   "dbtable" -> s"(select userid, yid, username from legacyusers offset
> $cachedIndex ) as legacyusers")
> )
>
> I'll like to know if this expression is correct:
>
> "dbtable" -> s"(select userid, yid, username from legacyusers offset
> $cachedIndex ) as legacyusers")
>
> As you can see. I'm trying to load the table records by offset
>
> I appreciate your help.
>
>


-- 
Odeyemi 'Kayode O.
http://ng.linkedin.com/in/kayodeodeyemi. t: @charyorde