You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "davidpissarra (via GitHub)" <gi...@apache.org> on 2023/11/10 00:08:20 UTC
[PR] [Unity][nn.module] Rolling KVCache override method [tvm]
davidpissarra opened a new pull request, #16100:
URL: https://github.com/apache/tvm/pull/16100
Part of the effort in bringing Mistral to the compilation flow with nn.Module on [mlc-llm](https://github.com/mlc-ai/mlc-llm). Adding support for Rolling KVCache override method in the nn.module frontend.
cc @CharlieFRuan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][nn.module] Rolling KVCache override method [tvm]
Posted by "davidpissarra (via GitHub)" <gi...@apache.org>.
davidpissarra commented on PR #16100:
URL: https://github.com/apache/tvm/pull/16100#issuecomment-1804989914
> I think we should move all KVCache logic to MLC LLM instead, given we expect plenty of variants of KVCache implementation. @tqchen @davidpissarra what do you guys think?
Thanks @junrushao! Makes sense to move KVCache to MLC LLM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][nn.module] Rolling KVCache override method [tvm]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #16100:
URL: https://github.com/apache/tvm/pull/16100#issuecomment-1804959615
I think we should move all KVCache logic to MLC LLM instead, given we expect plenty of variants of KVCache implementation. @tqchen @davidpissarra what do you guys think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][nn.module] Rolling KVCache override method [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen closed pull request #16100: [Unity][nn.module] Rolling KVCache override method
URL: https://github.com/apache/tvm/pull/16100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][nn.module] Rolling KVCache override method [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #16100:
URL: https://github.com/apache/tvm/pull/16100#issuecomment-1804961691
Yes, i think moving to mlc is OK, we can move back if there is a point of stablization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][nn.module] Rolling KVCache override method [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #16100:
URL: https://github.com/apache/tvm/pull/16100#issuecomment-1804956842
cc @junrushao @cyx-6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org