You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mustafasrepo (via GitHub)" <gi...@apache.org> on 2023/07/20 14:32:11 UTC
[GitHub] [arrow-datafusion] mustafasrepo opened a new pull request, #7040: Utilize Primary key information, to support additional queries
mustafasrepo opened a new pull request, #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
Closes [#6190](https://github.com/apache/arrow-datafusion/issues/6190).
# Rationale for this change
Primary key information specifies that some of the columns in the table, would consist of unique values. This contract enables us to support additional queries, currently cannot be run by datafusiuon.
Consider the query below
```sql
SELECT sn, amount
FROM sales_global
GROUP BY sn
```
When `sn` is `PRIMARY KEY` (each `sn` will have unique values) we know that, all the columns the table `sales_global` can be emitted after aggregation (since for each group all rows would have same value). In terms of functionality, above query and following query
```sql
SELECT sn, amount
FROM sales_global
GROUP BY sn, amount
```
are same. The reason is that, since column `sn` already consists of unique values; both `GROUP BY sn, amount` and `GROUP BY sn` will produce groups that consist of single row.
```sql
SELECT sn, amount
FROM sales_global
GROUP BY sn
```
Above query can run in Postgre. However, datafusion can only emit `sn` after aggregation from the original table. With this PR above query also supported by Datafusion.
As part of this PR, we keep track of identifier key (where primary key is a special case) to accomplish this. To illusturate what an identifier key is
Consider table below
| a | b |
| ---- | --- |
| 1 | USD |
| 2 | USD |
| 3 | TRY |
In table above, `column a` consists of unique values, it is primary key. We store this information as
```rust
IdentifierKeyGroup{
identifier_key_indices: vec![0],
is_unique: true,
associated_indices: vec![0, 1])
}
```
this means that, when we know the value of `0th` column, we will know the values of `0th and 1st` columns deterministically. And each row will be unique.
As an another example, consider table below,
| a | b |
| ----- | ----- |
| 1 | USD |
| 1 | USD |
| 2 | USD |
| 2 | USD |
| 3 | TRY |
| 3 | TRY |
In the example above, `column a` is not primary key, however, still knowing value of `a` enables us to know value of `b` deterministically. We encode this information as
```rust
IdentifierKeyGroup{
identifier_key_indices: vec![0],
is_unique: false,
associated_indices: vec![0, 1])
}
```
All of these information are propagated from the primary key information at the source. These analysis enables us to support complex queries. As an example, while we can support query below
```sql
SELECT r.sn, SUM(l.amount), r.amount
FROM sales_global_with_pk AS l
JOIN sales_global_with_pk AS r
ON l.sn >= r.sn
GROUP BY r.sn
ORDER BY r.sn
```
we reject, following query
```sql
SELECT l.sn, r.sn, SUM(l.amount), r.amount
FROM sales_global_with_pk AS l
JOIN sales_global_with_pk AS r
ON l.sn >= r.sn
GROUP BY l.sn
ORDER BY l.sn
```
because second query, uses an unassociated column(`r.sn`) with `l.sn` after group by, whereas, first query uses only associated columns(`r.sn`, `r.amount`) with `r.sn` during projection.
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
# Are these changes tested?
Yes, new tests that show supported functionality are added to the `groupby.slt` file.
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb merged pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ozankabak commented on pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "ozankabak (via GitHub)" <gi...@apache.org>.
ozankabak commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1650174981
This is looking very good now! It provides a good foundation to build even more functional dependencies as we move forward and we already have some plans for that :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1653429028
Thanks @mustafasrepo and @ozankabak -- this is a really nice step forward
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Utilize Primary key information, to support additional queries
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1270846585
##########
datafusion/core/src/datasource/default_table_source.rs:
##########
@@ -52,6 +54,11 @@ impl TableSource for DefaultTableSource {
self.table_provider.schema()
}
+ /// Get a reference to primary key indices, if a primary key exists.
+ fn primary_keys(&self) -> Option<&[usize]> {
Review Comment:
> Would it be possible to make this API more general? Specifically, there are several kinds of keys that a table might have so special casing primary key will make supporting others harder in the future
>
> Single column primary key
>
> ```sql
> create table t (
> x int primary key,
> y int,
> z int)
> ```
>
> Unique (like primary key, but can be null)
>
> ```sql
> create table t (
> x int unique,
> y int,
> z int)
> ```
>
> Multi column primary key
>
> ```sql
> create table t (
> x int,
> y int,
> z int,
> primary key (x, y)
> )
> ```
>
> Foreign key:
>
> ```sql
> create table t (
> x int,
> y int,
> z int,
> foreign key (x) references other_table(a)
> )
> ```
>
> Perhaps we could make it this more general someting like
>
> ```rust
> pub enum Constraints {
> // columns together form a primary key (are unique and not null)
> PrimaryKey(Vec<usize>),
> // columns together form a unique key
> Unique(Vec<usize>),
> // More constraint types to come as we add additional support
> }
> ```
>
> fn constraints(&self) -> Vec { ... }
Thanks @alamb for your feedback. Other than this suggestion, I have implemented all of your suggestions. With, your suggestions functional dependencies are now better encapsulated. Most of functionality is moved to the methods.
`FunctionalDependecies` now has a method `new_from_primary_keys`. All of the mapping between primary key and functioanal dependency occur, with this method. Hopefully, when we change this API, changing `new_from_primary_keys` and doing necessary mapping will be enough. If it is OK with you, I can change this API in following PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1276146136
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
> I thought it is a bit weird for aggregate to receive select_expr argument.
Yes I agree that sounds weird. I still don't understand why the functional dependency can't be computed from the plan itself (e.g. from the annotations on DFSchema and then the structure of the LogicalPlan::Aggregate / LogicalPlan::Project, etc)
If the logic is in the sql planner, that implies the calculation relies on information that is not encoded in the plan itself (and thus may be lost in subsequent optimizer passes, for example)
> After this PR merges, I can file another PR, that implements this change. Then we can decide which implementation is better. In any case, I have moved groupby update logic to its own function to not clutter this method.
Sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1276223297
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
> Yes I agree that sounds weird. I still don't understand why the functional dependency can't be computed from the plan itself
Currently, if the query is
```sql
SELECT sn, amount
FROM sales_global
GROUP BY sn
```
and we know that `sn` is primary key. We rewrite query above as
```sql
SELECT sn, amount
FROM sales_global
GROUP BY sn, amount
```
so that select either contains group by expressions or aggregate expressions. This update is done here.
However, if the query were
```sql
SELECT sn
FROM sales_global
GROUP BY sn
```
we wouldn't rewrite query as
```sql
SELECT sn
FROM sales_global
GROUP BY sn, amount
```
hence, amount is not used and. We do not need to update group by here.
For this reason, we need the information of `select exprs` to decide how to extend group by expressions to support a query.
However, if we were to extend group by expressions with their functional dependencies in all cases. We won't need the `select exprs` for this decision. Such as, we would treat
```sql
SELECT sn
FROM sales_global
GROUP BY sn
```
as
```sql
SELECT sn
FROM sales_global
GROUP BY sn, amount
```
under the hood.
What do you think about extending group by expression in all cases, without checking for `select exprs`. I think this is a bit confusing.
In short, functional dependency can be calculated without external information. However, query rewrite cannot be done without resorting to the information in the `select exprs` in current design.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1650388421
Awesome -- can't wait to review this. Will do so first thing tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1275848819
##########
datafusion/common/src/dfschema.rs:
##########
@@ -20,26 +20,315 @@
use std::collections::{HashMap, HashSet};
use std::convert::TryFrom;
+use std::fmt::{Display, Formatter};
use std::hash::Hash;
use std::sync::Arc;
use crate::error::{unqualified_field_not_found, DataFusionError, Result, SchemaError};
-use crate::{field_not_found, Column, OwnedTableReference, TableReference};
+use crate::{field_not_found, Column, JoinType, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// This object defines a constraint on a table.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub enum Constraint {
+ /// Columns with the given indices form a composite primary key (they are
+ /// jointly unique and not nullable):
+ PrimaryKey(Vec<usize>),
+ /// Columns with the given indices form a composite unique key:
+ Unique(Vec<usize>),
+}
+
+/// This object defines a functional dependence in the schema. A functional
Review Comment:
Moved them to their own module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1275887927
##########
datafusion/common/src/dfschema.rs:
##########
@@ -20,26 +20,315 @@
use std::collections::{HashMap, HashSet};
use std::convert::TryFrom;
+use std::fmt::{Display, Formatter};
use std::hash::Hash;
use std::sync::Arc;
use crate::error::{unqualified_field_not_found, DataFusionError, Result, SchemaError};
-use crate::{field_not_found, Column, OwnedTableReference, TableReference};
+use crate::{field_not_found, Column, JoinType, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// This object defines a constraint on a table.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub enum Constraint {
+ /// Columns with the given indices form a composite primary key (they are
+ /// jointly unique and not nullable):
+ PrimaryKey(Vec<usize>),
+ /// Columns with the given indices form a composite unique key:
+ Unique(Vec<usize>),
+}
+
Review Comment:
Did this conversion, also doing this change enabled me to use `primary_key_from_constraints` function as a method of this function. Updated name is `new_from_table_constraints`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ozankabak commented on pull request #7040: Utilize Primary key information, to support additional queries
Posted by "ozankabak (via GitHub)" <gi...@apache.org>.
ozankabak commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1645516667
Thank you @alamb for your review. I like your suggestions, both in terms of code structure and naming. @mustafasrepo will go through your comments and improve this PR accordingly before we merge it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #7040: Utilize Primary key information, to support additional queries
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1644471898
I plan to review this PR tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1647897527
> @alamb, we decided to do a couple more refactors to support constraints involving composite keys, primary key constraints and uniqueness constraints neatly within this PR. We will only leave foreign keys as future work so that TD is minimal.
Sounds good -- thank you @ozankabak
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #7040: Utilize Primary key information, to support additional queries
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1270550382
##########
datafusion/common/src/dfschema.rs:
##########
@@ -28,18 +29,58 @@ use crate::{field_not_found, Column, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// Stores identifier keys and their associated column indices. An identifier key
Review Comment:
I think a term for this concept in the database world is "Functional Dependence" where the value of one or more columns determines the value of one or more others. There is classic way to represent them and a standard set of axioms like https://en.wikipedia.org/wiki/Armstrong%27s_axioms for reasoning about them
I recommend we change to using this the functional dependence term so that:
1. This (really cool) feature is easier to discover
2. We can rely on existing explanations of the concept more rather than having to explain it fully in DataFusion. For example, google will tell you a lot about this https://www.google.com/search?q=functional+dependence+in+database. This article seems to have a good treatment of it https://www.scaler.com/topics/dbms/functional-dependency-in-dbms/
3. We can rely on prior art for rules and representations. In your example with `sn` and `amount`, I think would be modeled like `{sn, amount} -> {amount}`
##########
datafusion/core/src/datasource/default_table_source.rs:
##########
@@ -52,6 +54,11 @@ impl TableSource for DefaultTableSource {
self.table_provider.schema()
}
+ /// Get a reference to primary key indices, if a primary key exists.
+ fn primary_keys(&self) -> Option<&[usize]> {
Review Comment:
Would it be possible to make this API more general? Specifically, there are several kinds of keys that a table might have so special casing primary key will make supporting others harder in the future
Single column primary key
```sql
create table t (
x int primary key,
y int,
z int)
```
Unique (like primary key, but can be null)
```sql
create table t (
x int unique,
y int,
z int)
```
Multi column primary key
```sql
create table t (
x int,
y int,
z int,
primary key (x, y)
)
```
Foreign key:
```sql
create table t (
x int,
y int,
z int,
foreign key (x) references other_table(a)
)
```
Perhaps we could make it this more general someting like
```rust
pub enum Constraints {
// columns together form a primary key (are unique and not null)
PrimaryKey(Vec<usize>),
// columns together form a unique key
Unique(Vec<usize>),
// More constraint types to come as we add additional support
}
```
fn constraints(&self) -> Vec<Constraints> {
...
}
```
##########
datafusion/common/src/dfschema.rs:
##########
@@ -28,18 +29,58 @@ use crate::{field_not_found, Column, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// Stores identifier keys and their associated column indices. An identifier key
+/// is a column whose value determines values of some other (dependent) columns.
+/// These dependent columns are the "associated columns" of this identifier key.
+/// If two rows have the same identifier key, associated columns in these rows
+/// are necessarily the same. If the identifier key is unique, the set of
+/// associated columns is equal to the entire schema and the identifier key can
+/// serve as a primary key. Note that a primary key may "downgrade" into an
+/// identifier key due to an operation such as a join, and this object is used to
+/// track dependence relationships in such cases.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct IdentifierKeyGroup {
Review Comment:
Another potential way to implement this might be
```rust
enum FunctionalDependence {
source_indices: vec<Usize>,
target_indices: Vec<usize>,
type: Dependency,
}
enum Dependency {
Single, // unique constraint
Multi, // non unique (e.g. multipart primary key)
}
```
But I think what you have here is basically the same, except for the the names
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +431,51 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
+ let id_key_groups = schema.identifier_key_groups();
Review Comment:
here is another good candidate to put into a method on `IdentifierKeyGroups` with a named function which I think would keep the sql planner easier to read and understand what was happening
##########
datafusion/common/src/dfschema.rs:
##########
@@ -116,14 +162,36 @@ impl DFSchema {
)
}
+ /// Assign identifier key groups
+ pub fn with_identifier_key_groups(
+ mut self,
+ identifier_key_groups: IdentifierKeyGroups,
+ ) -> Self {
+ self.identifier_key_groups = identifier_key_groups;
+ self
+ }
+
/// Create a new schema that contains the fields from this schema followed by the fields
/// from the supplied schema. An error will be returned if there are duplicate field names.
pub fn join(&self, schema: &DFSchema) -> Result<Self> {
let mut fields = self.fields.clone();
let mut metadata = self.metadata.clone();
fields.extend_from_slice(schema.fields().as_slice());
metadata.extend(schema.metadata.clone());
- Self::new_with_metadata(fields, metadata)
+ let mut id_key_groups = self.identifier_key_groups.clone();
Review Comment:
style nit is if you encapsulated this code in its own function (like Schema::join_functional_dependencies())` it might be more self-describing
##########
datafusion/expr/src/logical_plan/builder.rs:
##########
@@ -1087,9 +1105,48 @@ pub fn build_join_schema(
}
};
+ let mut right_id_key_groups = add_offset_to_identifier_key_groups(
Review Comment:
This code I think could also be put as a method on KeyIdentiferGroups which would make it easier to find
```rust
impl KeyIdentiferGroups {
/// Compute the output dependncies of self JOIN other with the specified join type
fn join(&self, other:&Self, join_type: JoinType) {..}
}
##########
datafusion/expr/src/logical_plan/builder.rs:
##########
@@ -1214,6 +1271,171 @@ pub fn union(left_plan: LogicalPlan, right_plan: LogicalPlan) -> Result<LogicalP
}))
}
+// Update entries inside the `entries` vector with their corresponding index
+// inside the `proj_indices` vector.
+fn update_elements_with_matching_indices(
+ entries: &[usize],
+ proj_indices: &[usize],
+) -> Vec<usize> {
+ entries
+ .iter()
+ .filter_map(|val| proj_indices.iter().position(|proj_idx| proj_idx == val))
+ .collect()
+}
+
+/// Update identifier key indices using the index mapping in `proj_indices`.
+/// Assume that `proj_indices` is \[2, 5, 8\] and identifier key groups is
+/// \[5\] -> \[5, 8\] in `df_schema`. Then, the return value will be \[1\] -> \[1, 2\].
+/// This means that the first index of the `proj_indices` (5) is an identifier key,
+/// and this identifier key is associated with columns at indices 1 and 2 (in
+/// the updated schema). In the updated schema, fields at indices \[2, 5, 8\] will
+/// be at \[0, 1, 2\].
+pub fn project_identifier_key_indices(
Review Comment:
this is another good example of code that would be easier to find it if were a member function of `IdentifierKeygroups`
##########
datafusion/common/src/dfschema.rs:
##########
@@ -28,18 +29,58 @@ use crate::{field_not_found, Column, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// Stores identifier keys and their associated column indices. An identifier key
+/// is a column whose value determines values of some other (dependent) columns.
+/// These dependent columns are the "associated columns" of this identifier key.
+/// If two rows have the same identifier key, associated columns in these rows
+/// are necessarily the same. If the identifier key is unique, the set of
+/// associated columns is equal to the entire schema and the identifier key can
+/// serve as a primary key. Note that a primary key may "downgrade" into an
+/// identifier key due to an operation such as a join, and this object is used to
+/// track dependence relationships in such cases.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct IdentifierKeyGroup {
+ pub identifier_key_indices: Vec<usize>,
+ /// Flag indicating whether this identifier key is unique.
+ /// If true, these indices constitute a primary key. Otherwise, the
+ /// identifier key still uniquely determines its associated columns.
+ pub is_unique: bool,
+ pub associated_indices: Vec<usize>,
+}
+
+impl IdentifierKeyGroup {
+ pub fn new(
+ identifier_key_indices: Vec<usize>,
+ associated_indices: Vec<usize>,
+ ) -> Self {
+ Self {
+ identifier_key_indices,
+ is_unique: false,
+ associated_indices,
+ }
+ }
+
+ pub fn with_is_unique(mut self, is_unique: bool) -> Self {
+ self.is_unique = is_unique;
+ self
+ }
+}
+
+pub type IdentifierKeyGroups = Vec<IdentifierKeyGroup>;
Review Comment:
Stylistically if you made this a real type, it would:
1. encapsulate the manipulation more (like making `add_offset_to_identifier_key_groups` a method)
2. Make finding the available operations on it easier in the future
Something like
```suggestion
pub struct IdentifierKeyGroups {
inner: Vec<IdentifierKeyGroup>;
}
impl IdentifierKeyGroups {
pub add_offset(&mut self, offset: usize) {..}
...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1650059482
@alamb, I have implemented your suggestions. Now we support unique and primary key constraints at the source (foreign key not supported yet). I have extended existing tests, to cover unique and composite primary key cases. It is now ready for further review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1276074747
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
In here, we update group by exprs by the expressions that are in target indices (if they are in the select exprs). With this change, we can use dependents that are among select exprs, when their determinant is inside group by expression.
We can do so inside `Aggregate::try_new_with_schema` if we pass `select_expr` argument to it. I thought it is a bit weird for aggregate to receive `select_expr` argument. After this PR merges, I can file another PR, that implements this change. Then we can decide which implementation is better. In any case, I have moved groupby update logic to its own function to not clutter this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1275325562
##########
datafusion/core/tests/sqllogictests/test_files/groupby.slt:
##########
@@ -2847,3 +2847,390 @@ SELECT country, ARRAY_AGG(amount ORDER BY amount DESC) AS amounts,
FRA [200.0, 50.0] 50 50
GRC [80.0, 30.0] 30 30
TUR [100.0, 75.0] 75 75
+
+# create a table for testing
+statement ok
+CREATE TABLE sales_global_with_pk (zip_code INT,
+ country VARCHAR(3),
+ sn INT,
+ ts TIMESTAMP,
+ currency VARCHAR(3),
+ amount FLOAT,
+ primary key(sn)
+ ) as VALUES
+ (0, 'GRC', 0, '2022-01-01 06:00:00'::timestamp, 'EUR', 30.0),
+ (1, 'FRA', 1, '2022-01-01 08:00:00'::timestamp, 'EUR', 50.0),
+ (1, 'TUR', 2, '2022-01-01 11:30:00'::timestamp, 'TRY', 75.0),
+ (1, 'FRA', 3, '2022-01-02 12:00:00'::timestamp, 'EUR', 200.0),
+ (1, 'TUR', 4, '2022-01-03 10:00:00'::timestamp, 'TRY', 100.0)
+
+# create a table for testing, where primary key is composite
+statement ok
+CREATE TABLE sales_global_with_composite_pk (zip_code INT,
+ country VARCHAR(3),
+ sn INT,
+ ts TIMESTAMP,
+ currency VARCHAR(3),
+ amount FLOAT,
+ primary key(sn, ts)
+ ) as VALUES
+ (0, 'GRC', 0, '2022-01-01 06:00:00'::timestamp, 'EUR', 30.0),
+ (1, 'FRA', 1, '2022-01-01 08:00:00'::timestamp, 'EUR', 50.0),
+ (1, 'TUR', 2, '2022-01-01 11:30:00'::timestamp, 'TRY', 75.0),
+ (1, 'FRA', 3, '2022-01-02 12:00:00'::timestamp, 'EUR', 200.0),
+ (1, 'TUR', 4, '2022-01-03 10:00:00'::timestamp, 'TRY', 100.0)
+
+# create a table for testing, where sn is unique key
+statement ok
+CREATE TABLE sales_global_with_unique (zip_code INT,
+ country VARCHAR(3),
+ sn INT,
+ ts TIMESTAMP,
+ currency VARCHAR(3),
+ amount FLOAT,
+ unique(sn)
+ ) as VALUES
+ (0, 'GRC', 0, '2022-01-01 06:00:00'::timestamp, 'EUR', 30.0),
+ (1, 'FRA', 1, '2022-01-01 08:00:00'::timestamp, 'EUR', 50.0),
+ (1, 'TUR', 2, '2022-01-01 11:30:00'::timestamp, 'TRY', 75.0),
+ (1, 'FRA', 3, '2022-01-02 12:00:00'::timestamp, 'EUR', 200.0),
+ (1, 'TUR', 4, '2022-01-03 10:00:00'::timestamp, 'TRY', 100.0),
+ (1, 'TUR', NULL, '2022-01-03 10:00:00'::timestamp, 'TRY', 100.0)
+
+# when group by contains primary key expression
+# we can use all the expressions in the table during selection
+# (not just group by expressions + aggregation result)
+query TT
+EXPLAIN SELECT s.sn, s.amount, 2*s.sn
+ FROM sales_global_with_pk AS s
+ GROUP BY sn
+ ORDER BY sn
+----
+logical_plan
+Sort: s.sn ASC NULLS LAST
+--Projection: s.sn, s.amount, Int64(2) * CAST(s.sn AS Int64)
+----Aggregate: groupBy=[[s.sn, s.amount]], aggr=[[]]
+------SubqueryAlias: s
+--------TableScan: sales_global_with_pk projection=[sn, amount]
+physical_plan
+SortPreservingMergeExec: [sn@0 ASC NULLS LAST]
+--SortExec: expr=[sn@0 ASC NULLS LAST]
+----ProjectionExec: expr=[sn@0 as sn, amount@1 as amount, 2 * CAST(sn@0 AS Int64) as Int64(2) * s.sn]
+------AggregateExec: mode=FinalPartitioned, gby=[sn@0 as sn, amount@1 as amount], aggr=[]
Review Comment:
If the plan is grouping on a unique column I think the `AggregateExec` could be avoided entirely because each group will have exactly one non null row
Maybe that would be good optimization to add in a follow on PR
##########
datafusion/expr/src/logical_plan/ddl.rs:
##########
@@ -222,8 +223,8 @@ impl Hash for CreateExternalTable {
pub struct CreateMemoryTable {
/// The table name
pub name: OwnedTableReference,
- /// The ordered list of columns in the primary key, or an empty vector if none
- pub primary_key: Vec<Column>,
+ /// The list of constraints in the schema, such as primary key, unique, etc.
+ pub constraints: Vec<Constraint>,
Review Comment:
👍
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
Could you explain why the dependency logic needs to be in the SQL planner? It seems more general than just SQL (it should apply to dataframes as well)
Also, this function calls `LogicalPlanBuilder::aggregate` which then eventally calls `Aggregate::try_new_with_schema` which also re-calculates functional dependency information
Maybe this logic could be put in `Aggregate::try_new_with_schema` if it is not redundant
##########
datafusion/common/src/dfschema.rs:
##########
@@ -20,26 +20,315 @@
use std::collections::{HashMap, HashSet};
use std::convert::TryFrom;
+use std::fmt::{Display, Formatter};
use std::hash::Hash;
use std::sync::Arc;
use crate::error::{unqualified_field_not_found, DataFusionError, Result, SchemaError};
-use crate::{field_not_found, Column, OwnedTableReference, TableReference};
+use crate::{field_not_found, Column, JoinType, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// This object defines a constraint on a table.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub enum Constraint {
+ /// Columns with the given indices form a composite primary key (they are
+ /// jointly unique and not nullable):
+ PrimaryKey(Vec<usize>),
+ /// Columns with the given indices form a composite unique key:
+ Unique(Vec<usize>),
+}
+
Review Comment:
It might make the API cleaner to have a `Contraints` that is similar to `FunctionalDependencies`, so the API can be in terms of `Option<&Constraints>` rather than `Option<[&Constraint]>`
```rust
/// This object encapsulates a list of functional constraints:
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Constraints {
inner: Vec<Constraint>,
}
##########
datafusion/sql/tests/sql_integration.rs:
##########
@@ -232,7 +232,7 @@ fn cast_to_invalid_decimal_type_precision_lt_scale() {
fn plan_create_table_with_pk() {
let sql = "create table person (id int, name string, primary key(id))";
let plan = r#"
-CreateMemoryTable: Bare { table: "person" } primary_key=[id]
+CreateMemoryTable: Bare { table: "person" } constraints=[PrimaryKey([0])]
Review Comment:
👍
##########
datafusion/common/src/dfschema.rs:
##########
@@ -20,26 +20,315 @@
use std::collections::{HashMap, HashSet};
use std::convert::TryFrom;
+use std::fmt::{Display, Formatter};
use std::hash::Hash;
use std::sync::Arc;
use crate::error::{unqualified_field_not_found, DataFusionError, Result, SchemaError};
-use crate::{field_not_found, Column, OwnedTableReference, TableReference};
+use crate::{field_not_found, Column, JoinType, OwnedTableReference, TableReference};
use arrow::compute::can_cast_types;
use arrow::datatypes::{DataType, Field, FieldRef, Fields, Schema, SchemaRef};
-use std::fmt::{Display, Formatter};
/// A reference-counted reference to a `DFSchema`.
pub type DFSchemaRef = Arc<DFSchema>;
+/// This object defines a constraint on a table.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub enum Constraint {
+ /// Columns with the given indices form a composite primary key (they are
+ /// jointly unique and not nullable):
+ PrimaryKey(Vec<usize>),
+ /// Columns with the given indices form a composite unique key:
+ Unique(Vec<usize>),
+}
+
+/// This object defines a functional dependence in the schema. A functional
Review Comment:
Both `Constraint` and `FunctionalDependencies` might be easier to find if they were in their own modules / .rs files
##########
datafusion/expr/src/logical_plan/ddl.rs:
##########
@@ -222,8 +223,8 @@ impl Hash for CreateExternalTable {
pub struct CreateMemoryTable {
/// The table name
pub name: OwnedTableReference,
- /// The ordered list of columns in the primary key, or an empty vector if none
- pub primary_key: Vec<Column>,
+ /// The list of constraints in the schema, such as primary key, unique, etc.
+ pub constraints: Vec<Constraint>,
Review Comment:
👍
##########
datafusion/sql/src/statement.rs:
##########
@@ -1163,20 +1165,40 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
fn primary_key_from_constraints(
Review Comment:
🤔 maybe this function would be more clearly if it were renamed to reflect its expanded role for Constrants not just primary keys
Like
```suggestion
fn constrants_from_constraints(
```
?
##########
datafusion/sql/tests/sql_integration.rs:
##########
@@ -232,7 +232,7 @@ fn cast_to_invalid_decimal_type_precision_lt_scale() {
fn plan_create_table_with_pk() {
let sql = "create table person (id int, name string, primary key(id))";
let plan = r#"
-CreateMemoryTable: Bare { table: "person" } primary_key=[id]
+CreateMemoryTable: Bare { table: "person" } constraints=[PrimaryKey([0])]
Review Comment:
👍
##########
datafusion/sql/src/statement.rs:
##########
@@ -1163,20 +1165,40 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
fn primary_key_from_constraints(
Review Comment:
🤔 maybe this function would be more clearly if it were renamed to reflect its expanded role for Constrants not just primary keys
Like
```suggestion
fn constrants_from_constraints(
```
?
##########
datafusion/optimizer/src/analyzer/count_wildcard_rule.rs:
##########
@@ -263,7 +263,9 @@ fn rewrite_schema(schema: &DFSchema) -> DFSchemaRef {
})
.collect::<Vec<DFField>>();
DFSchemaRef::new(
- DFSchema::new_with_metadata(new_fields, schema.metadata().clone()).unwrap(),
+ DFSchema::new_with_metadata(new_fields, schema.metadata().clone())
Review Comment:
Constructing DFSchema's is unfortunately awkward but I don't think that is the fault of this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1276232108
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
> What do you think about extending group by expression in all cases, without checking for select exprs. I think this is a bit confusing.
I agree
> In short, functional dependency can be calculated without external information. However, query rewrite cannot be done without resorting to the information in the select exprs in current design.
I think this is the key point I was missing -- that the code in the sql planner is actually doing a SQL level rewrite. Thank you for clarifying.
What would you think about pulling the rewrite code into its own function and adding a clear explanation of what it was doing (basically copy/paste your example from above)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1276304465
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
Thanks @alamb. I have opened a mini [PR](https://github.com/apache/arrow-datafusion/pull/7111) for the update of docstring.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "mustafasrepo (via GitHub)" <gi...@apache.org>.
mustafasrepo commented on code in PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#discussion_r1276074747
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
In here, we update group by exprs by their expressions that are in target indices (if they are in the select exprs). With this change, we can use depedents in the functional dependence in the select expression, when their determinant is inside group by expression. We can do so inside `Aggregate::try_new_with_schema` if we pass `select_expr` argument to it. I thought it is a bit weird for aggregate to receive `select_expr` argument. After this PR merges, I can file another PR, that implements this change. Then we can decide which implementation is better. In any case, I have moved groupby update logic to its own function to not clutter this method.
##########
datafusion/sql/src/select.rs:
##########
@@ -431,6 +432,37 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
group_by_exprs: Vec<Expr>,
aggr_exprs: Vec<Expr>,
) -> Result<(LogicalPlan, Vec<Expr>, Option<Expr>)> {
+ let schema = input.schema();
Review Comment:
In here, we update group by exprs by the expressions that are in target indices (if they are in the select exprs). With this change, we can use depedents in the functional dependence in the select expression, when their determinant is inside group by expression. We can do so inside `Aggregate::try_new_with_schema` if we pass `select_expr` argument to it. I thought it is a bit weird for aggregate to receive `select_expr` argument. After this PR merges, I can file another PR, that implements this change. Then we can decide which implementation is better. In any case, I have moved groupby update logic to its own function to not clutter this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #7040: Utilize Primary key information, to support additional queries
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1646622985
Marking as draft given @ozankabak's suggestion https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1645516667 as feedback is incorporated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #7040: Utilize Primary key information, to support additional queries
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1646625239
Sorry -- I missed the update. I will review this shortly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ozankabak commented on pull request #7040: Initial support for functional dependencies handling primary key and unique constraints
Posted by "ozankabak (via GitHub)" <gi...@apache.org>.
ozankabak commented on PR #7040:
URL: https://github.com/apache/arrow-datafusion/pull/7040#issuecomment-1647622366
@alamb, we decided to do a couple more refactors to support composite keys, primary key constraints and uniqueness constraints neatly within this PR. We will only leave foreign keys as future work so that TD is minimal.
I am marking this as a draft until it is ready for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org