[jira] [Commented] (SOLR-9963) Add Calcite Avatica handler to Solr

["Kevin Risden (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-9963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959653#comment-15959653 ] 

Kevin Risden commented on SOLR-9963:
------------------------------------

[~joel.bernstein] - I had some time to dig into this. The response is formatted for Avatica JSON as documented here: https://calcite.apache.org/avatica/docs/json_reference.html

The result is actually streamed back as requested based on if its a single fetch or multiple fetches. Here is a script I wrote to demonstrate the streaming of results back.

{code}
./test_avatica_solr.sh "http://localhost:8983/solr/test/avatica" "select * from test limit 10"
{code}

{code}
#!/usr/bin/env bash

set -u
#set -x

AVATICA=$1
SQL=$2

CONNECTION_ID="conn-$(whoami)-$(date +%s)"
MAX_ROW_COUNT=100
NUM_ROWS=2
OFFSET=0

echo "Open connection"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"openConnection\",\"connectionId\": \"${CONNECTION_ID}\"}"

# Example of how to set connection properties with info key
#curl -i "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"openConnection\",\"connectionId\": \"${CONNECTION_ID}\",\"info\": {\"zk\": \"$ZK\",\"lex\": \"MYSQL\"}}"
echo

echo "Create statement"
STATEMENTRSP=$(curl -s "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"createStatement\",\"connectionId\": \"${CONNECTION_ID}\"}")
STATEMENTID=$(echo "$STATEMENTRSP" | jq .statementId)
echo

echo "PrepareAndExecuteRequest"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"prepareAndExecute\",\"connectionId\": \"${CONNECTION_ID}\",\"statementId\": $STATEMENTID,\"sql\": \"$SQL\",\"maxRowCount\": ${MAX_ROW_COUNT}, \"maxRowsInFirstFrame\": ${NUM_ROWS}}"
echo

# Loop through all the results
ISDONE=false
while ! $ISDONE; do
  OFFSET=$((OFFSET + NUM_ROWS))
  echo "FetchRequest - Offset=$OFFSET"
  FETCHRSP=$(curl -s "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"fetch\",\"connectionId\": \"${CONNECTION_ID}\",\"statementId\": $STATEMENTID,\"offset\": ${OFFSET},\"fetchMaxRowCount\": ${NUM_ROWS}}")
  echo "$FETCHRSP"
  ISDONE=$(echo "$FETCHRSP" | jq .frame.done)
  echo
done

echo "Close statement"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"closeStatement\",\"connectionId\": \"${CONNECTION_ID}\",\"statementId\": $STATEMENTID}"
echo

echo "Close connection"
curl -i -w "\n" "$AVATICA" -H "Content-Type: application/json" --data "{\"request\": \"closeConnection\",\"connectionId\": \"${CONNECTION_ID}\"}"
echo
{code}

The script is also being attached to this ticket.

> Add Calcite Avatica handler to Solr
> -----------------------------------
>
>                 Key: SOLR-9963
>                 URL: https://issues.apache.org/jira/browse/SOLR-9963
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: Parallel SQL
>            Reporter: Kevin Risden
>            Assignee: Kevin Risden
>         Attachments: SOLR-9963.patch, SOLR-9963.patch, SOLR-9963.patch
>
>
> Calcite Avatica has an http endpoint which allows Avatica drivers to connect to the server. This can be wired in as a handler to Solr. This would allow Solr to be used by any Avatica JDBC/ODBC driver. This depends on the Calcite work from SOLR-8593.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org