[jira] [Created] (SPARK-21920) DataFrame Fail To Find The Column Name

["abhijit nag (JIRA)" <ji...@apache.org>]

abhijit nag created SPARK-21920:
-----------------------------------

             Summary: DataFrame Fail To Find The Column Name
                 Key: SPARK-21920
                 URL: https://issues.apache.org/jira/browse/SPARK-21920
             Project: Spark
          Issue Type: Question
          Components: Spark Core
    Affects Versions: 1.6.0
            Reporter: abhijit nag
            Priority: Critical


I am getting one issue like "sql.AnalysisException: cannot resolve column_name"
Wrote a simple query as below.
[DataFrame df= df1
  .join(df2, df1.col("MERCHANT").equalTo(df2.col("MERCHANT")))
  .select(df2.col("MERCH_ID"), df1.col("MERCHANT")));]

Exception Found : 
resolved attribute(s) MERCH_ID#738 missing from MERCHANT#737,MERCHANT#928,MERCH_ID#929,MER_LOC#930 in operator !Project [MERCH_ID#738,MERCHANT#737];

Problem Solved by following code:
DataFrame df= df1.alias("df1").
  .join(df2.alias("df2"), functions.col("df1.MERCHANT").equalTo(functions.col("df2.MERCHANT")))
  .select(functions.col("df2.MERCH_ID"), functions.col("df2.MERCHANT")));

Similar kind of issue appears rare, but I want to know the root cause of this problem. 
Is it a bug in Spark 1.6 or something else.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org