[jira] [Commented] (CALCITE-1190) Cross-Version Compatibility Test Harness

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227165#comment-15227165 ] 

Josh Elser commented on CALCITE-1190:
-------------------------------------

Concretely, let's say I have two versions of Avatica+Phoenix that I want to test, say versionA and versionB.

versionA is defined by client-A.jar and a JDBC url template of {{jdbc:avatica:remote:url=<url>}} (where {{<url>}} is the special marker to replace with the actual server URL) on the client side, and the server URL of http://localhost:8765.

versionB is define by client-B.jar and a JDBC url template of {{jdbc;avatica:remote:url=<url>}} on the client side and the server URL of http://localhost:8766.

>From this, the test harness could run a battery of tests with client-A.jar to localhost:8766 and client-B.jar to localhost:8765, reporting which tests did not match the expected outcome. This scales out easily to adding another version (versionC), computing the cartesian product of all versions (a->b, a->c, b->a, b->c, c->b, c->a) sans the identity for each (excluding a->a, b->b, c->c).

> Cross-Version Compatibility Test Harness
> ----------------------------------------
>
>                 Key: CALCITE-1190
>                 URL: https://issues.apache.org/jira/browse/CALCITE-1190
>             Project: Calcite
>          Issue Type: Test
>          Components: avatica
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: avatica-1.8.0
>
>
> One thing that the Protobuf serialization aimed to provide was a library which provides us the tools to make Avatica compatible across versions. However, Protobuf is just a tool and the developers can still misuse protobuf in such a way that breaks compatibility across versions. Not to mention, compatibility isn't even remotely certain without any tests.
> Because of Avatica's position as a library less than a product, we have to defer some logic to the concrete product being tested (e.g. Phoenix or Drill). I'm thinking something like the following:
> The user provides pairs of client and server "definitions" for a given version of compatibility. This would include some version of Avatica and some backing database. For example, Avatica-1.7.1 and Phoenix-4.7.0 or Avatica-1.8.0-SNAPSHOT and HSQLDB-2.3.1.
> The client half would be some template for the appropriate JDBC url to use (sans the URL to the Avatica server) and a JAR file containing the necessary classes to run the j.s.Driver. The server half would just be a URL to the Avatica server instance.
> The test harness itself can provide the logic to test the remote driver against the other servers, enumerating the possible combinations of client-server communication. Starting the server for each version to test is likely too difficult to automate well given the unknown of what the server will be, so that will be left as a prerequisite.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)