In Review

Planned

Completed

Won't do

Duplicat

Main Roadmap

Many other providers offer prompt-caching with discounted pricing for cache hits, either implicitly (e.g., OpenAI, DeepSeek, Google, DeepInfra, NovitaAI, Fireworks) or explicitly (most notably Anthropic). This capability can significantly reduce costs in agentic workflows, where a single session often re-sends the same context repeatedly (for example, when the model performs multiple tool calls in sequence and the shared conversation/context is included each time). Today, Nebius Token Factory is at a cost disadvantage in these repeated, input-token-heavy scenarios compared to providers that support prompt caching and pass the savings through to customers. Please add support for prompt caching (implicit or explicit), including discounted pricing for cached prompt tokens, to improve cost-efficiency for agentic and tool-using applications.

De nombreux autres fournisseurs proposent la mise en cache des invites avec des tarifs réduits pour les accès au cache, soit de manière implicite (par exemple, OpenAI, DeepSeek, Google, DeepInfra, NovitaAI, Fireworks), soit de manière explicite (notamment Anthropic). Cette fonctionnalité peut réduire considérablement les coûts dans les workflows agentiques, où une même session renvoie souvent le même contexte à plusieurs reprises (par exemple, lorsque le modèle effectue plusieurs appels d’outils à la suite et que la conversation ou le contexte partagé est inclus à chaque fois). À l’heure actuelle, Nebius Token Factory est désavantagée en termes de coûts dans ces scénarios répétitifs et gourmands en jetons d’entrée, par rapport aux fournisseurs qui prennent en charge la mise en cache des invites et répercutent les économies réalisées sur leurs clients. Veuillez ajouter la prise en charge de la mise en cache des invites (implicite ou explicite), y compris des tarifs réduits pour les jetons d’invite mis en cache, afin d’améliorer la rentabilité des applications agentiques et utilisant des outils.

Support Implicite/Explicite Prompt Caching

Billing

Сompute Cloud

Lukas Kreussel

Please add deepseek-r1 model. DeepSeek’s first-generation reasoning models, achieving performance comparable to OpenAI-o1 across math, code, and reasoning tasks.

New DeepSeek R1 Model

Amin Bagheri

Please add an API for viewing live usage with daily breakdowns and the ability to query our current balance as I can’t seem to find an auto top-up function on Token Factory.Thank you!

User interface

Usage and Balance API on Token Factory

Herren5939

Please add an API endpoint to retrieve the current account balance for a Nebius Token Factory account

Account Balance API endpoint for Nebius Token Factory

Ivan Nikitin

New Deepseek v3 scores really high on Aider leaderboards: <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://aider.chat/docs/leaderboards/">https://aider.chat/docs/leaderboards/</a> <img src="https://fb-usercontent.fra1.cdn.digitaloceanspaces.com/0193fea1-9f0f-7ea8-a714-6ec4c7d835ec.png" data-width="100%" data-align="left"> Model: <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/tree/main">https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/tree/main</a> Would probably be the new best model on the platform. Is 600B+ params a problem ? Would the pricing be similar to LLama 3 405B ? Merry Christmas!

<a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://aider.chat/docs/leaderboards/">https://aider.chat/docs/leaderboards/</a> <img src="https://fb-usercontent.fra1.cdn.digitaloceanspaces.com/0193fea1-9f0f-7ea8-a714-6ec4c7d835ec.png" data-width="100%" data-align="left"> モデル：<a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/tree/main">https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/tree/main</a> おそらくプラットフォームの新しいベストモデルになるでしょう。 600B以上のパラメータは問題でしょうか？

DeepSeek v3

Gaël CADO

Try inference engine with function call support + provide working example and list limitations (i.e. some inference libraries do not allow ‘auto’ tool_choice, for example)

Function call support

Cyril Kondratenko

Add Kimi K2 model

Krishna Adithya Venkatesh

It would be great to have native support for reranking in the Nebius AI API, especially for use in retrieval-augmented generation (RAG) or semantic search workflows.See <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html">https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html</a> for the /rerank and /score API.<ul><li>Add a dedicated endpoint for reranking such as the /score or the /rerank endpoind.</li><li>Add cross-encoder models to the available model library</li></ul>

Feature Request: Add Cross-Encoder Reranking Support via /rerank or /score Endpoint

llschutte

Show model quantization info

[Deleted User]

Would be nice to work with picture generation as well

Multimodal models

One of the best news we received at the end of 2024, DeepSeek V3 is a large and intelligent AI model, compared to Claude 3.5 Sonnet and even being superior in some aspects according to LM Arena (https://lmarena.ai/?leaderboard)

Una de las mejores noticias que recibimos a finales de 2024, DeepSeek V3 es un modelo de IA grande e inteligente, comparado con Claude 3.5 Sonnet e incluso siendo superior en algunos aspectos según LM Arena (https://lmarena.ai/?leaderboard)

DeepSeek V3

L'une des meilleures nouvelles que nous ayons reçues à la fin de 2024, DeepSeek V3 est un grand modèle d'IA intelligent, comparé à Claude 3.5 Sonnet et même supérieur à certains égards selon LM Arena (https://lmarena.ai/?leaderboard).

2024'ün sonunda aldığımız en iyi haberlerden biri olan DeepSeek V3, Claude 3.5 Sonnet ile karşılaştırılan ve hatta LM Arena'ya göre bazı yönlerden daha üstün olan büyük ve akıllı bir AI modelidir (https://lmarena.ai/?leaderboard)

Wilame Souza

It would be really helpful to have a solution library that offers end-to-end production solutions we can manage ourselves and install using Terraform. Right now, there's no easy way to get started with your products.