You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by GitBox <gi...@apache.org> on 2022/05/17 11:38:54 UTC
[GitHub] [knox] Bhargavi-Sagi opened a new pull request, #576: KNOX-2746 - Add presto/presto ui support in service definition
Bhargavi-Sagi opened a new pull request, #576:
URL: https://github.com/apache/knox/pull/576
## What changes were proposed in this pull request?
[Presto](https://prestodb.io/docs/current/) is a distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources. Add support for in knox service definition, so that knox can provide proxying service for presto.
## How was this patch tested?
Deployed knox in a cluster with presto, and checked access presto ui with knox server port 8442. Also checked presto apis with knox server as proxy
<img width="1762" alt="Screenshot 2022-05-17 at 4 58 23 PM" src="https://user-images.githubusercontent.com/92242165/168801653-1afbf65b-ee66-4570-95be-602af065db3e.png">
<img width="1762" alt="Screenshot 2022-05-17 at 4 58 49 PM" src="https://user-images.githubusercontent.com/92242165/168801685-a041b25a-7ad5-45d6-830b-4fbf8bb22959.png">
API testing:
`curl --insecure -X POST -d 'show catalogs' -H "X-Presto-User: user" -H "Content-Type: application/json" https://localhost:8442/gateway/default/presto/v1/statement`
{"id":"20220517_112614_00000_muw7k","infoUri":"https://localhost:8442/gateway/default/presto/ui/query.html?20220517_112614_00000_muw7k","nextUri":"https://localhost:8442/gateway/default/presto/v1/statement/queued/20220517_112614_00000_muw7k/1?slug=x31aa07b4664b44de90fe15c2e2a758cc","stats":{"state":"WAITING_FOR_PREREQUISITES","waitingForPrerequisites":true,"queued":false,"scheduled":false,"nodes":0,"totalSplits":0,"queuedSplits":0,"runningSplits":0,"completedSplits":0,"cpuTimeMillis":0,"wallTimeMillis":0,"waitingForPrerequisitesTimeMillis":0,"queuedTimeMillis":0,"elapsedTimeMillis":0,"processedRows":0,"processedBytes":0,"peakMemoryBytes":0,"peakTotalMemoryBytes":0,"peakTaskTotalMemoryBytes":0,"spilledBytes":0},"warnings":[]}
`curl --insecure https://localhost:8442/gateway/default/presto/v1/statement/queued/20220517_112614_00000_muw7k/1?slug=x31aa07b4664b44de90fe15c2e2a758cc`
{"id":"20220517_112614_00000_muw7k","infoUri":"https://localhost:8442/gateway/default/presto/ui/query.html?20220517_112614_00000_muw7k","nextUri":"https://localhost:8442/gateway/default/presto/v1/statement/queued/20220517_112614_00000_muw7k/2?slug=x31aa07b4664b44de90fe15c2e2a758cc","stats":{"state":"WAITING_FOR_PREREQUISITES","waitingForPrerequisites":true,"queued":false,"scheduled":false,"nodes":0,"totalSplits":0,"queuedSplits":0,"runningSplits":0,"completedSplits":0,"cpuTimeMillis":0,"wallTimeMillis":0,"waitingForPrerequisitesTimeMillis":6,"queuedTimeMillis":0,"elapsedTimeMillis":6,"processedRows":0,"processedBytes":0,"peakMemoryBytes":0,"peakTotalMemoryBytes":0,"peakTaskTotalMemoryBytes":0,"spilledBytes":0},"warnings":[]}
`curl --insecure https://localhost:8442/gateway/default/presto/v1/statement/queued/20220517_112614_00000_muw7k/2?slug=x31aa07b4664b44de90fe15c2e2a758cc`
{"id":"20220517_112614_00000_muw7k","infoUri":"https://localhost:8442/gateway/default/presto/ui/query.html?20220517_112614_00000_muw7k","partialCancelUri":"https://localhost:8442/gateway/default/presto/v1/stage/20220517_112614_00000_muw7k.0","nextUri":"https://localhost:8442/gateway/default/presto/v1/statement/executing/20220517_112614_00000_muw7k/1?slug=x31aa07b4664b44de90fe15c2e2a758cc","columns":[{"name":"Catalog","type":"varchar(14)","typeSignature":{"rawType":"varchar","typeArguments":[],"literalArguments":[],"arguments":[{"kind":"LONG_LITERAL","value":14}]}}],"data":[["awsdatacatalog"],["hive"],["system"]],"stats":{"state":"RUNNING","waitingForPrerequisites":false,"queued":false,"scheduled":true,"nodes":1,"totalSplits":19,"queuedSplits":0,"runningSplits":0,"completedSplits":19,"cpuTimeMillis":14,"wallTimeMillis":141,"waitingForPrerequisitesTimeMillis":15,"queuedTimeMillis":0,"elapsedTimeMillis":12522,"processedRows":0,"processedBytes":0,"peakMemoryBytes":0,"peakTotalMemoryBy
tes":171,"peakTaskTotalMemoryBytes":171,"spilledBytes":0,"rootStage":{"stageId":"0","state":"RUNNING","done":false,"nodes":1,"totalSplits":1,"queuedSplits":0,"runningSplits":0,"completedSplits":1,"cpuTimeMillis":1,"wallTimeMillis":3,"processedRows":3,"processedBytes":67,"subStages":[{"stageId":"1","state":"FINISHED","done":true,"nodes":1,"totalSplits":17,"queuedSplits":0,"runningSplits":0,"completedSplits":17,"cpuTimeMillis":13,"wallTimeMillis":138,"processedRows":3,"processedBytes":67,"subStages":[{"stageId":"2","state":"FINISHED","done":true,"nodes":1,"totalSplits":1,"queuedSplits":0,"runningSplits":0,"completedSplits":1,"cpuTimeMillis":0,"wallTimeMillis":0,"processedRows":3,"processedBytes":0,"subStages":[]}]}]},"runtimeStats":{"S1-driverCountPerTask":{"name":"S1-driverCountPerTask","sum":17,"count":1,"max":17,"min":17},"S2-driverCountPerTask":{"name":"S2-driverCountPerTask","sum":1,"count":1,"max":1,"min":1},"S1-taskElapsedTimeNanos":{"name":"S1-taskElapsedTimeNanos","sum":14279
9852,"count":1,"max":142799852,"min":142799852},"S2-taskElapsedTimeNanos":{"name":"S2-taskElapsedTimeNanos","sum":106038415,"count":1,"max":106038415,"min":106038415},"S0-driverCountPerTask":{"name":"S0-driverCountPerTask","sum":1,"count":1,"max":1,"min":1},"S0-taskElapsedTimeNanos":{"name":"S0-taskElapsedTimeNanos","sum":0,"count":1,"max":0,"min":0}},"progressPercentage":100.0},"warnings":[]}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 commented on pull request #576: KNOX-2746 - Add presto/presto ui support in service definition
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on PR #576:
URL: https://github.com/apache/knox/pull/576#issuecomment-1143746741
@Bhargavi-Sagi - this looks great!
There was another PR recently that added [Trino](https://trino.io/docs/current/) as a new service to Knox. Checking Trino's and Presto's pages I found exactly the same description of the 2 projects:
<img width="1402" alt="Screenshot 2022-06-01 at 17 17 50" src="https://user-images.githubusercontent.com/34065904/171439666-eb000130-86b5-4fff-9bf0-c61d5eebe4f7.png">
<img width="1565" alt="Screenshot 2022-06-01 at 17 17 41" src="https://user-images.githubusercontent.com/34065904/171439687-b4aa0ddc-86fe-4660-9c5a-4c3a693bea8f.png">
On Trino's UI there's even a link suggesting we are talking about the same (similar) services. Hence the question: do we need two separate service definitions for the same functionality?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] Bhargavi-Sagi commented on pull request #576: KNOX-2746 - Add presto/presto ui support in service definition
Posted by GitBox <gi...@apache.org>.
Bhargavi-Sagi commented on PR #576:
URL: https://github.com/apache/knox/pull/576#issuecomment-1148212095
Presto and Trino are different projects. Trino was initially forked from presto but then they have diverged quite a lot over time. There are apis in Trino which are not present in presto. Even though UI page looks same not all the apis are same(for example login/logout are not present in presto) and so we will need separate service definitions for Presto and Trino.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [knox] smolnar82 merged pull request #576: KNOX-2746 - Add presto/presto ui support in service definition
Posted by GitBox <gi...@apache.org>.
smolnar82 merged PR #576:
URL: https://github.com/apache/knox/pull/576
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@knox.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org