You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@wayang.apache.org by "kmoltke (via GitHub)" <gi...@apache.org> on 2023/05/11 14:09:19 UTC
[GitHub] [incubator-wayang] kmoltke opened a new pull request, #322: Refinement of SQL API
kmoltke opened a new pull request, #322:
URL: https://github.com/apache/incubator-wayang/pull/322
Hi, @michellesackmann and I removed the last hard coded parts of the SQL API.
Everything needed is specified in the model.json file which is located under resources. So, this model.json file is the configuration file like you typically see in other systems. We also added the possibility of specifying a separator for .csv files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] kbeedkar commented on pull request #322: Refinement of SQL API
Posted by "kbeedkar (via GitHub)" <gi...@apache.org>.
kbeedkar commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1545524456
Hi guys, I don't think this is the right way to support different data sources.
As we discussed in an earlier email, a better way to go about this, which is also extensible and maintainable, is to first extend the WayangRel Algebra, with specific table scan operators (e.g., one for Jdbc, one for csv, etc..) and write logical-to-physical rules. Each physical scan operator should then have methods for accessing sources.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] kmoltke commented on pull request #322: Refinement of SQL API
Posted by "kmoltke (via GitHub)" <gi...@apache.org>.
kmoltke commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1545563869
Hi, thank you for your comment. This is something that can be improved, unfortunately, given the limitations of this research project, we don't have resources to do that right now. Also, there are definitely more stuff that could be improved. In our opinion, this PR is still an improvement since it 1) gets rid of all the hard coded parts 2) cuts severely down on the lines of code that the user has to provide to run a query and 3) adds the functionality to provide delimiters/separators for csv-files. Yes, in terms of future maintainability/extensibility we agree with you, but it requires resources that we don't have right now.
We hope that your suggestions can be implemented in another PR :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] 2pk03 merged pull request #322: Refinement of SQL API
Posted by "2pk03 (via GitHub)" <gi...@apache.org>.
2pk03 merged PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] kbeedkar commented on pull request #322: Refinement of SQL API
Posted by "kbeedkar (via GitHub)" <gi...@apache.org>.
kbeedkar commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1546480979
Yes @2pk03, let's do it that way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] michellesackmann commented on pull request #322: Refinement of SQL API
Posted by "michellesackmann (via GitHub)" <gi...@apache.org>.
michellesackmann commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1546966915
Sounds great guys, thank you for approving the PR! Is there any chance the PR could be merged before tomorrow afternoon, since we have to hand in the report for our project, and it would be great if it was aligned with what is on the main branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] kmoltke commented on pull request #322: Refinement of SQL API
Posted by "kmoltke (via GitHub)" <gi...@apache.org>.
kmoltke commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1546235933
@2pk03 the license header should be there now :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] 2pk03 commented on pull request #322: Refinement of SQL API
Posted by "2pk03 (via GitHub)" <gi...@apache.org>.
2pk03 commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1546212814
> Hi, thank you for your comment. This is something that can be improved, unfortunately, given the limitations of this research project, we don't have resources to do that right now. Also, there are definitely more stuff that could be improved. In our opinion, this PR is still an improvement since it 1) gets rid of all the hard coded parts 2) cuts severely down on the lines of code that the user has to provide to run a query and 3) adds the functionality to provide delimiters/separators for csv-files. Yes, in terms of future maintainability/extensibility we agree with you, but it requires resources that we don't have right now.
>
> We hope that your suggestions can be implemented in another PR :)
Pls add the license header:
https://github.com/apache/incubator-wayang/actions/runs/4948812486/jobs/8880546014?pr=322#step:3:127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-wayang] 2pk03 commented on pull request #322: Refinement of SQL API
Posted by "2pk03 (via GitHub)" <gi...@apache.org>.
2pk03 commented on PR #322:
URL: https://github.com/apache/incubator-wayang/pull/322#issuecomment-1546209607
@kbeedkar can we merge and open a ticket to improve the commit? Multiple CSV files sounds great :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org