Re: [PR] Ignore repetitions of semicolon [cassandra]

["sedulam (via GitHub)" <gi...@apache.org>]

sedulam commented on code in PR #3812:
URL: https://github.com/apache/cassandra/pull/3812#discussion_r1938470821


##########
pylib/cqlshlib/cqlhandling.py:
##########
@@ -151,6 +151,39 @@ def cql_split_statements(self, text):
                     in_batch = True
         return output, in_batch or in_pg_string
 
+    def group_tokens(self, items):
+        """
+        Split an iterable into sublists, using 'endtoken' to mark the end of each sublist.
+        Each sublist accumulates elements until an 'endtoken' is encountered. If the sublist
+        consists only of a single 'endtoken', it is excluded. An empty list is added to the
+        result after the last 'endtoken' for cases like autocompletion.
+
+        Parameters:
+        - items (iterable): An iterable of tokens, including 'endtoken' elements.
+
+        Returns:
+        - list: A list of sublists, with each sublist containing tokens split by 'endtoken'.
+        """
+        thisresult = []
+        output = []
+
+        for i in items:
+            thisresult.append(i)
+
+            # When an 'endtoken' is encountered, start a new sublist
+            if i[0] == 'endtoken':
+                # Skip adding sublist if it contains just one "endtoken"
+                if len(thisresult) > 1:
+                    output.append(thisresult)  # Add valid sublist
+
+                # Start a new sublist after an 'endtoken'
+                thisresult = []

Review Comment:
   To use `itertools.groupby` it needs to do some extra work like converting the groups to lists. It's possible but less efficient, and a bit less readable imo:
   
   ```
       def group_tokens(self, items):
           output = []
           sublist = []
   
           for key, group in itertools.groupby(items, lambda t: t[0] == 'endtoken'):
               group_list = list(group)
   
               if key:
                   if sublist:
                       sublist.append(group_list[0])
                       output.append(sublist)
                   sublist = []
               else:
                   sublist.extend(group_list)
   
           output.append(sublist)
   
           return output
   ```
   
   Let me know if this form is still preferable, and I'll commit the changes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org