You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "dcoliversun (via GitHub)" <gi...@apache.org> on 2024/08/07 06:26:09 UTC
[PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
dcoliversun opened a new pull request, #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738
## What changes were proposed in this pull request?
Currently, gluten supports packaging iceberg into a bundle jar, and it’s best for us to maintain consistent behavior on the delta side. This brings several benefits:
1. Consistent behavior avoids increasing the learning burden;
2. Simplifies deployment;
3. Aligns better with users' expectations for manually specified configurations.
(Fixes: \#6737)
## How was this patch tested?
manual tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738#issuecomment-2272776953
cc @yma11 could you please review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
Posted by "yma11 (via GitHub)" <gi...@apache.org>.
yma11 commented on code in PR #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738#discussion_r1707135662
##########
docs/get-started/Velox.md:
##########
@@ -314,8 +314,7 @@ First of all, compile gluten-delta module by a `delta` profile, as follows:
mvn clean package -Pbackends-velox -Pspark-3.3 -Pdelta -DskipTests
```
-Then, put the additional `gluten-delta-XX-SNAPSHOT.jar` to the class path (usually it's `$SPARK_HOME/jars`).
-The gluten-delta jar is in `gluten-delta/target` directory.
+Once built successfully, delta features will be included in gluten-velox-bundle-X jar.
Review Comment:
Put line `318` together with this line please. Can you also help update the content for iceberg in this doc? seems there is some duplicated contents.
```
Once built successfully, iceberg features will be included in gluten-velox-bundle-X jar. Then you can query iceberg table by gluten/velox without scan's fallback.
After the two steps, you can query iceberg table by gluten/velox without scan's fallback.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on code in PR #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738#discussion_r1708290230
##########
docs/get-started/Velox.md:
##########
@@ -314,8 +314,7 @@ First of all, compile gluten-delta module by a `delta` profile, as follows:
mvn clean package -Pbackends-velox -Pspark-3.3 -Pdelta -DskipTests
```
-Then, put the additional `gluten-delta-XX-SNAPSHOT.jar` to the class path (usually it's `$SPARK_HOME/jars`).
-The gluten-delta jar is in `gluten-delta/target` directory.
+Once built successfully, delta features will be included in gluten-velox-bundle-X jar.
Review Comment:
Sure, it's done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
Posted by "yma11 (via GitHub)" <gi...@apache.org>.
yma11 merged PR #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on PR #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738#issuecomment-2276978686
@yma11 it's ok to merge, thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-6737]package delta into bundle jar when specify delta profile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6738:
URL: https://github.com/apache/incubator-gluten/pull/6738#issuecomment-2272713142
https://github.com/apache/incubator-gluten/issues/6737
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org