You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "jacobmarble (via GitHub)" <gi...@apache.org> on 2023/02/27 23:38:41 UTC
[GitHub] [arrow-adbc] jacobmarble opened a new pull request, #480: feat(golang/sqldriver): add simple flightsql package
jacobmarble opened a new pull request, #480:
URL: https://github.com/apache/arrow-adbc/pull/480
Helps https://github.com/apache/arrow/issues/34332
Golang/FlightSQL docs should be updated to point here for the canonical `database/sql` wrapper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #480: feat(golang/sqldriver): add simple flightsql package
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#issuecomment-1448168537
@zeroshade we should also perhaps consider a 0.3.0 at end of March/early April to get this out ASAP?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] zeroshade commented on a diff in pull request #480: feat(golang/sqldriver): add simple FlightSQL database/sql driver wrapper
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#discussion_r1121917941
##########
go/adbc/sqldriver/flightsql/flightsql_test.go:
##########
@@ -0,0 +1,141 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package flightsql_test
+
+import (
+ "context"
+ "database/sql"
+ _ "github.com/apache/arrow-adbc/go/adbc/sqldriver/flightsql"
+ "github.com/apache/arrow/go/v12/arrow/flight"
+ "github.com/apache/arrow/go/v12/arrow/flight/flightsql"
+ "github.com/apache/arrow/go/v12/arrow/flight/flightsql/example"
+ "github.com/apache/arrow/go/v12/arrow/memory"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+ "github.com/stretchr/testify/suite"
+ "google.golang.org/grpc"
+ "os"
+ "testing"
+)
+
+func Example() {
+ // Be sure to import the driver first:
+ // import _ "github.com/apache/arrow-adbc/go/adbc/sqldriver/flightsql"
+
+ db, err := sql.Open("flightsql", "uri=grpc://localhost:12345")
+ if err != nil {
+ panic(err)
+ }
+
+ if err = db.Ping(); err != nil {
+ panic(err)
+ }
+}
+
+type SQLDriverFlightSQLSuite struct {
+ suite.Suite
+
+ srv *example.SQLiteFlightSQLServer
+ s flight.Server
+ opts []grpc.ServerOption
+ sqliteDB *sql.DB
+
+ done chan bool
+ mem *memory.CheckedAllocator
+}
+
+func (suite *SQLDriverFlightSQLSuite) SetupTest() {
+ var err error
+
+ suite.sqliteDB, err = example.CreateDB()
+ require.NoError(suite.T(), err)
+
+ suite.mem = memory.NewCheckedAllocator(memory.DefaultAllocator)
+ suite.s = flight.NewServerWithMiddleware(nil, suite.opts...)
+ require.NoError(suite.T(), err)
+ suite.srv, err = example.NewSQLiteFlightSQLServer(suite.sqliteDB)
+ require.NoError(suite.T(), err)
+ suite.srv.Alloc = suite.mem
+
+ suite.s.RegisterFlightService(flightsql.NewFlightServer(suite.srv))
+ require.NoError(suite.T(), suite.s.Init("localhost:0"))
+ suite.s.SetShutdownOnSignals(os.Interrupt, os.Kill)
+ suite.done = make(chan bool)
+ go func() {
+ defer close(suite.done)
+ _ = suite.s.Serve()
+ }()
+}
+
+func (suite *SQLDriverFlightSQLSuite) TearDownTest() {
+ if suite.done == nil {
+ return
+ }
+
+ suite.s.Shutdown()
+ <-suite.done
+ suite.srv = nil
+ suite.mem.AssertSize(suite.T(), 0)
+ _ = suite.sqliteDB.Close()
+ suite.done = nil
+}
+
+func (suite *SQLDriverFlightSQLSuite) dsn() string {
+ return "uri=grpc+tcp://" + suite.s.Addr().String()
+}
+
+type srvQuery string
+
+func (s srvQuery) GetQuery() string { return string(s) }
+
+func (s srvQuery) GetTransactionId() []byte { return nil }
+
+func (suite *SQLDriverFlightSQLSuite) TestQuery() {
+ db, err := sql.Open("flightsql", suite.dsn())
+ require.NoError(suite.T(), err)
+ defer db.Close()
+
+ _, err = suite.srv.DoPutCommandStatementUpdate(
+ context.Background(), srvQuery("CREATE TABLE t (k, v)"))
+ require.NoError(suite.T(), err)
+ _, err = suite.srv.DoPutCommandStatementUpdate(
+ context.Background(), srvQuery("INSERT INTO t (k, v) VALUES ('one', 'alpha'), ('two', 'bravo')"))
+ require.NoError(suite.T(), err)
+
+ // TODO db.Exec() fails:
+ // Received unexpected error:
+ // Invalid State: SqlState: , msg: [Flight SQL Statement] must call Prepare before GetParameterSchema
+ //
+ //_, err = db.Exec("CREATE TABLE t (k, v)")
+ //require.NoError(suite.T(), err)
+ //defer db.Exec("DROP TABLE IF EXISTS t")
+ //_, err = db.Exec("INSERT INTO t (k, v) VALUES ('one', 'alpha'), ('two', 'bravo')")
+ //require.NoError(suite.T(), err)
Review Comment:
figured out the issue and pushed a fix, I also cleaned up the test a little (using `suite.` instead of having to pass `suite.T()` constantly).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] zeroshade commented on pull request #480: feat(golang/sqldriver): add simple flightsql package
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#issuecomment-1448420023
@lidavidm I agree, we should definitely work on a 0.3.0 release for march/april and get this out.
@jacobmarble I left one comment, but otherwise looks good to me other than agreeing with David's comment about the smoke test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #480: feat(golang/sqldriver): add simple FlightSQL database/sql driver wrapper
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#issuecomment-1452056230
I'll merge this, there's an open PR to fix ruby
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on a diff in pull request #480: feat(golang/sqldriver): add simple flightsql package
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#discussion_r1120797495
##########
go/adbc/sqldriver/flightsql/flightsql_test.go:
##########
@@ -0,0 +1,141 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package flightsql_test
+
+import (
+ "context"
+ "database/sql"
+ _ "github.com/apache/arrow-adbc/go/adbc/sqldriver/flightsql"
+ "github.com/apache/arrow/go/v12/arrow/flight"
+ "github.com/apache/arrow/go/v12/arrow/flight/flightsql"
+ "github.com/apache/arrow/go/v12/arrow/flight/flightsql/example"
+ "github.com/apache/arrow/go/v12/arrow/memory"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+ "github.com/stretchr/testify/suite"
+ "google.golang.org/grpc"
+ "os"
+ "testing"
+)
+
+func Example() {
+ // Be sure to import the driver first:
+ // import _ "github.com/apache/arrow-adbc/go/adbc/sqldriver/flightsql"
+
+ db, err := sql.Open("flightsql", "uri=grpc://localhost:12345")
+ if err != nil {
+ panic(err)
+ }
+
+ if err = db.Ping(); err != nil {
+ panic(err)
+ }
+}
+
+type SQLDriverFlightSQLSuite struct {
+ suite.Suite
+
+ srv *example.SQLiteFlightSQLServer
+ s flight.Server
+ opts []grpc.ServerOption
+ sqliteDB *sql.DB
+
+ done chan bool
+ mem *memory.CheckedAllocator
+}
+
+func (suite *SQLDriverFlightSQLSuite) SetupTest() {
+ var err error
+
+ suite.sqliteDB, err = example.CreateDB()
+ require.NoError(suite.T(), err)
+
+ suite.mem = memory.NewCheckedAllocator(memory.DefaultAllocator)
+ suite.s = flight.NewServerWithMiddleware(nil, suite.opts...)
+ require.NoError(suite.T(), err)
+ suite.srv, err = example.NewSQLiteFlightSQLServer(suite.sqliteDB)
+ require.NoError(suite.T(), err)
+ suite.srv.Alloc = suite.mem
+
+ suite.s.RegisterFlightService(flightsql.NewFlightServer(suite.srv))
+ require.NoError(suite.T(), suite.s.Init("localhost:0"))
+ suite.s.SetShutdownOnSignals(os.Interrupt, os.Kill)
+ suite.done = make(chan bool)
+ go func() {
+ defer close(suite.done)
+ _ = suite.s.Serve()
+ }()
+}
+
+func (suite *SQLDriverFlightSQLSuite) TearDownTest() {
+ if suite.done == nil {
+ return
+ }
+
+ suite.s.Shutdown()
+ <-suite.done
+ suite.srv = nil
+ suite.mem.AssertSize(suite.T(), 0)
+ _ = suite.sqliteDB.Close()
+ suite.done = nil
+}
+
+func (suite *SQLDriverFlightSQLSuite) dsn() string {
+ return "uri=grpc+tcp://" + suite.s.Addr().String()
+}
+
+type srvQuery string
+
+func (s srvQuery) GetQuery() string { return string(s) }
+
+func (s srvQuery) GetTransactionId() []byte { return nil }
+
+func (suite *SQLDriverFlightSQLSuite) TestQuery() {
+ db, err := sql.Open("flightsql", suite.dsn())
+ require.NoError(suite.T(), err)
+ defer db.Close()
+
+ _, err = suite.srv.DoPutCommandStatementUpdate(
+ context.Background(), srvQuery("CREATE TABLE t (k, v)"))
+ require.NoError(suite.T(), err)
+ _, err = suite.srv.DoPutCommandStatementUpdate(
+ context.Background(), srvQuery("INSERT INTO t (k, v) VALUES ('one', 'alpha'), ('two', 'bravo')"))
+ require.NoError(suite.T(), err)
+
+ // TODO db.Exec() fails:
+ // Received unexpected error:
+ // Invalid State: SqlState: , msg: [Flight SQL Statement] must call Prepare before GetParameterSchema
+ //
+ //_, err = db.Exec("CREATE TABLE t (k, v)")
+ //require.NoError(suite.T(), err)
+ //defer db.Exec("DROP TABLE IF EXISTS t")
+ //_, err = db.Exec("INSERT INTO t (k, v) VALUES ('one', 'alpha'), ('two', 'bravo')")
+ //require.NoError(suite.T(), err)
Review Comment:
@zeroshade got any clue here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] kou commented on pull request #480: feat(golang/sqldriver): add simple flightsql package
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#issuecomment-1448207244
The Ruby failure will be fixed after https://github.com/conda-forge/arrow-c-glib-feedstock/pull/8 is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] lidavidm merged pull request #480: feat(golang/sqldriver): add simple FlightSQL database/sql driver wrapper
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] jacobmarble commented on a diff in pull request #480: feat(golang/sqldriver): add simple FlightSQL database/sql driver wrapper
Posted by "jacobmarble (via GitHub)" <gi...@apache.org>.
jacobmarble commented on code in PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#discussion_r1122160122
##########
go/adbc/sqldriver/flightsql/flightsql_test.go:
##########
@@ -0,0 +1,141 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package flightsql_test
+
+import (
+ "context"
+ "database/sql"
+ _ "github.com/apache/arrow-adbc/go/adbc/sqldriver/flightsql"
+ "github.com/apache/arrow/go/v12/arrow/flight"
+ "github.com/apache/arrow/go/v12/arrow/flight/flightsql"
+ "github.com/apache/arrow/go/v12/arrow/flight/flightsql/example"
+ "github.com/apache/arrow/go/v12/arrow/memory"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+ "github.com/stretchr/testify/suite"
+ "google.golang.org/grpc"
+ "os"
+ "testing"
+)
+
+func Example() {
+ // Be sure to import the driver first:
+ // import _ "github.com/apache/arrow-adbc/go/adbc/sqldriver/flightsql"
+
+ db, err := sql.Open("flightsql", "uri=grpc://localhost:12345")
+ if err != nil {
+ panic(err)
+ }
+
+ if err = db.Ping(); err != nil {
+ panic(err)
+ }
+}
+
+type SQLDriverFlightSQLSuite struct {
+ suite.Suite
+
+ srv *example.SQLiteFlightSQLServer
+ s flight.Server
+ opts []grpc.ServerOption
+ sqliteDB *sql.DB
+
+ done chan bool
+ mem *memory.CheckedAllocator
+}
+
+func (suite *SQLDriverFlightSQLSuite) SetupTest() {
+ var err error
+
+ suite.sqliteDB, err = example.CreateDB()
+ require.NoError(suite.T(), err)
+
+ suite.mem = memory.NewCheckedAllocator(memory.DefaultAllocator)
+ suite.s = flight.NewServerWithMiddleware(nil, suite.opts...)
+ require.NoError(suite.T(), err)
+ suite.srv, err = example.NewSQLiteFlightSQLServer(suite.sqliteDB)
+ require.NoError(suite.T(), err)
+ suite.srv.Alloc = suite.mem
+
+ suite.s.RegisterFlightService(flightsql.NewFlightServer(suite.srv))
+ require.NoError(suite.T(), suite.s.Init("localhost:0"))
+ suite.s.SetShutdownOnSignals(os.Interrupt, os.Kill)
+ suite.done = make(chan bool)
+ go func() {
+ defer close(suite.done)
+ _ = suite.s.Serve()
+ }()
+}
+
+func (suite *SQLDriverFlightSQLSuite) TearDownTest() {
+ if suite.done == nil {
+ return
+ }
+
+ suite.s.Shutdown()
+ <-suite.done
+ suite.srv = nil
+ suite.mem.AssertSize(suite.T(), 0)
+ _ = suite.sqliteDB.Close()
+ suite.done = nil
+}
+
+func (suite *SQLDriverFlightSQLSuite) dsn() string {
+ return "uri=grpc+tcp://" + suite.s.Addr().String()
+}
+
+type srvQuery string
+
+func (s srvQuery) GetQuery() string { return string(s) }
+
+func (s srvQuery) GetTransactionId() []byte { return nil }
+
+func (suite *SQLDriverFlightSQLSuite) TestQuery() {
+ db, err := sql.Open("flightsql", suite.dsn())
+ require.NoError(suite.T(), err)
+ defer db.Close()
+
+ _, err = suite.srv.DoPutCommandStatementUpdate(
+ context.Background(), srvQuery("CREATE TABLE t (k, v)"))
+ require.NoError(suite.T(), err)
+ _, err = suite.srv.DoPutCommandStatementUpdate(
+ context.Background(), srvQuery("INSERT INTO t (k, v) VALUES ('one', 'alpha'), ('two', 'bravo')"))
+ require.NoError(suite.T(), err)
+
+ // TODO db.Exec() fails:
+ // Received unexpected error:
+ // Invalid State: SqlState: , msg: [Flight SQL Statement] must call Prepare before GetParameterSchema
+ //
+ //_, err = db.Exec("CREATE TABLE t (k, v)")
+ //require.NoError(suite.T(), err)
+ //defer db.Exec("DROP TABLE IF EXISTS t")
+ //_, err = db.Exec("INSERT INTO t (k, v) VALUES ('one', 'alpha'), ('two', 'bravo')")
+ //require.NoError(suite.T(), err)
Review Comment:
Cool, TIL `suite.Require()`.
I pushed another commit to fix the lint error and convert more uses of `suite.T()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-adbc] zeroshade commented on a diff in pull request #480: feat(golang/sqldriver): add simple flightsql package
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #480:
URL: https://github.com/apache/arrow-adbc/pull/480#discussion_r1120289738
##########
go/adbc/sqldriver/flightsql/flightsql.go:
##########
@@ -0,0 +1,30 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package flightsql
Review Comment:
Can we add a package level doc and/or a README file here just to increase the ease / searchability for users to understand how to leverage this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org