Ability to continue last message in /chat/completions / prefill response

If we want to use vision models, it seems we’re forced to use the

/chat/completions endpoint. If we want to continue a message, it seems we’re forced to use /completions endpoint. But if you want to do both with Nebius API as of the moment of writing, you are out of luck.

I propose having the ability to continue the last message (also known as “prefilling” with other API providers) which would solve both these problems.

Prefilling is very useful in case an AI message was interrupted and we want to naturally-infer it again without changing the context (which could include an image) to continue the last response.

Prefilling is also useful for more complex Text templates (what is what /completions is for, of course.) so being able to continue texts with a reference image that the model always observes all the time as the context evolves would elevate the creative use cases much further. But again, Vision models are only supported on /chat/completions so we must have prefilling / continuing last message at the bare minimum there.

Please authenticate to join the conversation.

Upvoters
Status

In Review

Board

💡 Feature request

Date

12 months ago

Author

L

Subscribe to post

Get notified by email when there are changes.