[DISCUSS] LSP -Text offsets and encoding?

[antonio <an...@vieiro.net>]

Hi,

I found this spreadsheet [1] (can't tell how recent that is, though) 
detailing the encodings that different LSP servers expect from LSP clients.

The fact is that the LSP specification mandates that positions in files 
(lines/columns) to be computed when the text is encoded in UTF-16. This 
is understandable, since LSP is promoted by Microsoft, and modern 
Windows store strings in UTF-16 internally :-).

Some examples:

- The clangd LSP server supports three different encodings [2]. You can 
set which one you want to use using the 
--offset-encoding=[utf-8|utf-16|utf-32] (we currently don't use this in 
cpplite).

- AFAIK ccls (another LSP server for C/C++ that we use in cpplite) 
supports utf-32 only.

It seems different LSP implementors are asking Microsoft to modify the 
"utf-16" requirement [3]

In our "ide/lsp.client" module we do compute offsets in utf-8, right? 
Using utf-16 offsets could be time consuming, right?

Do we want
- to keep using utf-8 (if we do)?
- or do we want to move to utf-16 (to be compliant)?
- or shall we wait until [3] is more clear?

Also, in our own Java LSP server, do we use utf-16 or utf-8 instead?

Thanks,
Antonio


[1]
https://docs.google.com/spreadsheets/d/168jSz68po0R09lO0xFK4OmDsQukLzSPCXqB6-728PXQ/edit#gid=0

[2]
https://clangd.llvm.org/extensions.html#utf-8-offsets
clangd, an important LSP server, has proposed this to be an "extension" 
to the LSP protocol.

[3]
Implementors asking the utf-16 to be changed to utf-8
https://github.com/Microsoft/language-server-protocol/issues/376


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists