You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "LiaCastaneda (via GitHub)" <gi...@apache.org> on 2025/05/20 14:38:55 UTC
[I] Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous [datafusion]
LiaCastaneda opened a new issue, #16114:
URL: https://github.com/apache/datafusion/issues/16114
### Describe the bug
Logical planning fails with the following error:
` Error: Schema error: Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous`
This error occurs during logical planning through the Substrait consumer path, specifically when building the final project schema [here](https://github.com/apache/datafusion/blob/8d9c0f6b87d9f3a52e4d3dc642535d09cf86049f/datafusion/substrait/src/logical_plan/consumer/utils.rs#L183). That function is called after consuming the full substrait plan, as seen [here](https://github.com/apache/datafusion/blob/8d9c0f6b87d9f3a52e4d3dc642535d09cf86049f/datafusion/substrait/src/logical_plan/consumer/plan.rs#L61).
The error is triggered when the final schema has:
- One column with a qualifier such as left.concat(...)
- Another column without a qualifier but with the same name: concat('a', 'b')
The error is reproducible with the substrait plan the following query produces:
```
WITH a as (SELECT CONCAT('a', 'b'), 1 as "join"),
b as (SELECT CONCAT('a', 'b'), 1 as "join"),
c as (SELECT * FROM a LEFT JOIN b on a."join" = b."join")
SELECT * FROM c
```
### To Reproduce
```
{
"extensionUris": [{
"extensionUriAnchor": 1,
"uri": "/functions_string.yaml"
}, {
"extensionUriAnchor": 2,
"uri": "/functions_comparison.yaml"
}],
"extensions": [{
"extensionFunction": {
"extensionUriReference": 1,
"functionAnchor": 0,
"name": "concat:str"
}
}, {
"extensionFunction": {
"extensionUriReference": 2,
"functionAnchor": 1,
"name": "equal:any_any"
}
}],
"relations": [{
"root": {
"input": {
"project": {
"common": {
"emit": {
"outputMapping": [3, 4, 5, 6]
}
},
"input": {
"join": {
"common": {
"direct": {
}
},
"left": {
"project": {
"common": {
"emit": {
"outputMapping": [1, 2]
}
},
"input": {
"read": {
"common": {
"direct": {
}
},
"baseSchema": {
"names": ["dummy"],
"struct": {
"types": [{
"i64": {
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
}],
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
},
"virtualTable": {
"values": [{
"fields": [{
"i64": "0",
"nullable": false,
"typeVariationReference": 0
}]
}]
}
}
},
"expressions": [{
"scalarFunction": {
"functionReference": 0,
"args": [],
"outputType": {
"string": {
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
},
"arguments": [{
"value": {
"literal": {
"string": "a",
"nullable": false,
"typeVariationReference": 0
}
}
}, {
"value": {
"literal": {
"string": "b",
"nullable": false,
"typeVariationReference": 0
}
}
}],
"options": []
}
}, {
"literal": {
"i64": "1",
"nullable": false,
"typeVariationReference": 0
}
}]
}
},
"right": {
"project": {
"common": {
"emit": {
"outputMapping": [1]
}
},
"input": {
"read": {
"common": {
"direct": {
}
},
"baseSchema": {
"names": ["dummy"],
"struct": {
"types": [{
"i64": {
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
}],
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
},
"virtualTable": {
"values": [{
"fields": [{
"i64": "0",
"nullable": false,
"typeVariationReference": 0
}]
}]
}
}
},
"expressions": [{
"literal": {
"i64": "1",
"nullable": false,
"typeVariationReference": 0
}
}]
}
},
"expression": {
"scalarFunction": {
"functionReference": 1,
"args": [],
"outputType": {
"bool": {
"typeVariationReference": 0,
"nullability": "NULLABILITY_REQUIRED"
}
},
"arguments": [{
"value": {
"selection": {
"directReference": {
"structField": {
"field": 1
}
},
"rootReference": {
}
}
}
}, {
"value": {
"selection": {
"directReference": {
"structField": {
"field": 2
}
},
"rootReference": {
}
}
}
}],
"options": []
}
},
"type": "JOIN_TYPE_LEFT"
}
},
"expressions": [{
"selection": {
"directReference": {
"structField": {
"field": 0
}
},
"rootReference": {
}
}
}, {
"selection": {
"directReference": {
"structField": {
"field": 1
}
},
"rootReference": {
}
}
}, {
"selection": {
"directReference": {
"structField": {
"field": 0
}
},
"rootReference": {
}
}
}, {
"selection": {
"directReference": {
"structField": {
"field": 1
}
},
"rootReference": {
}
}
}]
}
},
"names": ["concat(\u0027a\u0027, \u0027b\u0027)", "join", "concat(\u0027a\u0027, \u0027b\u0027)", "join"]
}
}],
"expectedTypeUrls": []
}
```
### Expected behavior
build logical plan sucessfully.
### Additional context
I'm using version 47.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous [datafusion]
Posted by "LiaCastaneda (via GitHub)" <gi...@apache.org>.
LiaCastaneda commented on issue #16114:
URL: https://github.com/apache/datafusion/issues/16114#issuecomment-2895491802
goinf to try a solution I have in mind, it should be (hopefully) quite straightforward
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous [datafusion]
Posted by "LiaCastaneda (via GitHub)" <gi...@apache.org>.
LiaCastaneda closed issue #16114: Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous
URL: https://github.com/apache/datafusion/issues/16114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous [datafusion]
Posted by "LiaCastaneda (via GitHub)" <gi...@apache.org>.
LiaCastaneda commented on issue #16114:
URL: https://github.com/apache/datafusion/issues/16114#issuecomment-2895490377
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Schema contains qualified field name left."concat('a', 'b')" and unqualified field name "concat('a', 'b')" which would be ambiguous [datafusion]
Posted by "LiaCastaneda (via GitHub)" <gi...@apache.org>.
LiaCastaneda commented on issue #16114:
URL: https://github.com/apache/datafusion/issues/16114#issuecomment-2911495430
Closing since the substrait plan root names actually have duplicates, we will fix this on our end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org