You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "piyushShama (via GitHub)" <gi...@apache.org> on 2023/12/26 15:49:08 UTC
[I] Drill | mongoDB | Issue with Alias naming conventions (drill)
piyushShama opened a new issue, #2858:
URL: https://github.com/apache/drill/issues/2858
Bug Description:
When using an alias with special characters in a Superset SQL query, the query returns null instead of the expected result.
Steps to Reproduce:
1. Use the following SQL query as an example:
`SELECT COUNT(*) AS `COUNT(*)`
FROM
(SELECT name,
grade
FROM mongo.test.`data`) AS virtual_table
WHERE grade = 'A'
LIMIT 50000;`
2. Observe that the query returns null when the alias contains special characters.
Expected Behavior:
The SQL query should return the count of records satisfying the condition, and not null, regardless of whether the alias contains special characters.
Actual Behavior:
The query returns null when an alias with special characters is used.
Additional Information:
Version of Apache Superset: 3.0.0
Database type and version: MongoDB 6.0
<img width="1277" alt="1" src="https://github.com/apache/drill/assets/90260638/9ea6b05f-50cb-424a-9c8b-409e5809e420">
<img width="1278" alt="2" src="https://github.com/apache/drill/assets/90260638/2adc464e-647f-497f-a7c1-6a33230e33cd">
<img width="1280" alt="3" src="https://github.com/apache/drill/assets/90260638/fddbeef4-c04b-467e-9999-facec60cb728">
<img width="1279" alt="4" src="https://github.com/apache/drill/assets/90260638/ab676cec-8053-4ddf-85fc-b76f0b054999">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "NomadBYlife (via GitHub)" <gi...@apache.org>.
NomadBYlife commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898813808
> But it should not be broken if the alias contains any special characters. The issue might be occurring at the MongoDB end, where a key with special characters could be causing the problem. However, this should be handled at the Drill end to create an alias without any special characters.
Have you found any solution? Totally the same situation. Drill 1.21 superset 3.0.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1900045823
@NomadBYlife the first change will be required in class Foreman.java in project drill/exec/java-exec
Method: run()
Line no: 268
Note: this code is working for all sql based queries. Though it is working for mongodb, it may fail for other plugins !! So you can use it locally, I’ll try to figure out the actual place in mongodb storage plugin project to implement this logic
I’ll try to push my changes to the git over this weekend
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "NomadBYlife (via GitHub)" <gi...@apache.org>.
NomadBYlife commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898838045
<img width="1440" alt="Снимок экрана 2024-01-18 в 17 31 21" src="https://github.com/apache/drill/assets/93082658/aeeaf6d3-3cc1-401f-9d62-5a07169e5062">
I tried changing this option. But nothing seems to change in the way queries work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1872470965
I'm using this apache-drill-1.21.1, I tried to use other versions as well. but it is failing for all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1900853684
@piyushShama Have you looked at the SQLAlchemy Driver for Drill? (https://github.com/JohnOmernik/sqlalchemy-drill)
IMHO, your fix have have gotten the query to work, but I'm fairly certain that likely broke other things.
My suggestion would be to take a look at the SQL Alchemy driver for Drill and you can have it rewrite queries. Simply have it rename the column something that Drill will accept.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1901647299
Not yet, but will do it !! Yes, the changes I have done re working for me but it may break other things
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898869429
Yes I’ll definitely help you !!
Please wait till tomorrow as I dont have access to my workstation now. I will provide you all the issues i have faced so far and their fixes as well !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1899533885
Yes , will do that over this weekend !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898896941
For now, you can clone the apache drill git repo in your local environment and you will need maven installed in your system
and once cloned, you can import them in your workspace
I’ll share the screenshots or lines of code along with the project name, class name, method name here tomorrow so that you can amend the changes in the required class and then can use the mvn to build your project i.e jar file for that project
once the jar is built , you will have to replace a jar available in lib folder in drill installation package by the built jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "NomadBYlife (via GitHub)" <gi...@apache.org>.
NomadBYlife commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898859028
> Yes, I have found a way to resolve this issue !! Keys with the special characters are the reason for the failure of mongodb query
>
> i don’t remember the project, class name and method name, will let you know tomorrow !! You will have to modify the existing code of apache drill to make it work
>
> you can check the logs once , to get an idea about the stacktrace to get the method where you need to do few modifications!! What i did ? Replaced alias count(*) by count !! Not valid for all cases, but working for me
>
> Additionally, you might get some issues with the filters with daterange for that there is a workaround
I will be very grateful if you tell me where to change what. I don't know Java, you'll just save me a lot of time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1870840956
What version of Drill are you using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1872471960
I found another software, Trino, to establish a connection between MongoDB and Superset. However, Trino is memory-intensive. Drill is a better option for my use cases. But, since a functionality is broken, I'll have to switch to Trino and explore it further to manage it according to my requirements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1872472578
But it should not be broken if the alias contains any special characters. The issue might be occurring at the MongoDB end, where a key with special characters could be causing the problem. However, this should be handled at the Drill end to create an alias without any special characters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898993482
All,
Instead of copying screen shots of code for this fix, would someone please create a pull request so that everyone will benefit from the fix?
Thanks,
-- C
> On Jan 18, 2024, at 12:17, piyushShama ***@***.***> wrote:
>
>
> For now, you can clone the apache drill git repo in your local environment and you will need maven installed in your system
>
> and once cloned, you can import them in your workspace
>
> I’ll share the screenshots or lines of code along with the project name, class name, method name here tomorrow so that you can amend the changes in the required class and then can use the mvn to build your project i.e jar file for that project
>
> once the jar is built , you will have to replace a jar available in lib folder in drill installation package by the built jar
>
> —
> Reply to this email directly, view it on GitHub <https://github.com/apache/drill/issues/2858#issuecomment-1898896941>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKB7PWRKRZKZJI7W4YJORDYPFKJNAVCNFSM6AAAAABBDMY7Q6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJYHA4TMOJUGE>.
> You are receiving this because you commented.
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Drill | mongoDB | Issue with Alias naming conventions (drill)
Posted by "piyushShama (via GitHub)" <gi...@apache.org>.
piyushShama commented on issue #2858:
URL: https://github.com/apache/drill/issues/2858#issuecomment-1898836255
Yes, I have found a way to resolve this issue !!
Keys with the special characters are the reason for the failure of mongodb query
i don’t remember the project, class name and method name, will let you know tomorrow !! You will have to modify the existing code of apache drill to make it work
you can check the logs once , to get an idea about the stacktrace to get the method where you need to do few modifications!! What i did ? Replaced alias count(*) by count !! Not valid for all cases, but working for me
Additionally, you might get some issues with the filters with daterange for that there is a workaround
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org