You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by randerzander <gi...@git.apache.org> on 2015/08/23 20:20:06 UTC
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
GitHub user randerzander opened a pull request:
https://github.com/apache/phoenix/pull/114
PHOENIX-2196 - auto capitalize field names in DF->Phoenix table save method
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/randerzander/phoenix master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/phoenix/pull/114.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #114
----
commit c32d65c8bf6b2cff95271716c85fc77ccf035740
Author: = <rg...@gmail.com>
Date: 2015-08-23T18:18:55Z
auto capitalize DF field names in DataFrameFunctions save method
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by taoshide <gi...@git.apache.org>.
Github user taoshide commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-188012880
Spark DataFrame to save the data, how to use SEQUENCE to call the saveToPhoenix method?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-133910608
Yes, I think that would work. We'd need unit tests around this too, please. The function we go through to normalize identifier/column references is SchemaUtil.normalizeIdentifier(String identifier). Here's a SQL example (which would work fine):
{code}
CREATE TABLE "t" (id VARCHAR PRIMARY KEY, "v" VARCHAR);
UPSERT INTO "t" (ID, "v") VALUES ('a','b');
SELECT ID, "v" FROM "t";
SELECT id, "v" FROM "t";
{code}
Would you have some cycles to review this pull, @jmahonin?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by jmahonin <gi...@git.apache.org>.
Github user jmahonin commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-133939400
Sure thing, on a quick glance on mobile this looks good, but I'll try spend some time with it tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-133897978
Thanks for the pull. One feature that Phoenix (and other RDBMS) has is that if a column identifier is double quoted, then it's treated as case sensitive. Otherwise, it's upper cased as you've mentioned. Would be good if we could accommodate this too in the phoenix-spark integration. If we can't do this, then we may need to keep it as case sensitive as otherwise we can't support the case sensitive use case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by randerzander <gi...@git.apache.org>.
Github user randerzander commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-133893731
As an example, reading a CSV file where headers aren't capitalized, the below is necessary before calling df.save:
var df = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("DROPMALFORMED", "true").load(input)
val columns = df.columns.map(x => x.toUpperCase)
df = df.toDF(columns:_*)
It would be nice for end users if the .save implementation handled this detail
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by jmahonin <gi...@git.apache.org>.
Github user jmahonin commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-134629729
@randerzander Not sure if you saw, but I've got a new version of your patch up at https://issues.apache.org/jira/browse/PHOENIX-2196
If you could take a quick look at let me know if that works for you I'll get it in ASAP.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-2196 - auto capitalize field names i...
Posted by randerzander <gi...@git.apache.org>.
Github user randerzander commented on the pull request:
https://github.com/apache/phoenix/pull/114#issuecomment-133904250
If I understand correctly, this change should work-
Any fieldnames with starting and ending quotes are untouched. If they aren't quoted, then they're auto-capitalized.
Does this work better?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---