Skip to main content

Larger FAST models with SO.

Currently SO by constrained decoding is supported by large models (>80B) only with tps <50. This is insanely slow for agentic architectures based on SGR, and you keep slowing down inference. Please provide at least one fast, large, good model with SO

Log in to comment and vote

No comments yet

Be the first to share your thoughts.