Larger FAST models with SO.
Currently SO by constrained decoding is supported by large models (>80B) only with tps <50. This is insanely slow for agentic architectures based on SGR, and you keep slowing down inference. Please provide at least one fast, large, good model with SO
Log in to comment and vote
No comments yet
Be the first to share your thoughts.