Re: [PR] Remove Whitespace Tokens from Parser [datafusion-sqlparser-rs]

["Viicos (via GitHub)" <gi...@apache.org>]

Viicos commented on code in PR #2077:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2077#discussion_r2515124489


##########
src/tokenizer.rs:
##########
@@ -449,29 +449,6 @@ impl Word {
     }
 }
 
-#[derive(Debug, Clone, PartialEq, PartialOrd, Eq, Ord, Hash)]
-#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
-#[cfg_attr(feature = "visitor", derive(Visit, VisitMut))]
-pub enum Whitespace {
-    Space,
-    Newline,
-    Tab,
-    SingleLineComment { comment: String, prefix: String },
-    MultiLineComment(String),
-}

Review Comment:
   While it would make sense for the parser to not have to deal with whitespace _including_ comments, it will be useful to preserve comment tokens. This is really useful when trying to provide IDE completions, where you don't want anything to be suggested in inside a comment.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org