Support Implicite/Explicite Prompt Caching

Many other providers offer prompt-caching with discounted pricing for cache hits, either implicitly (e.g., OpenAI, DeepSeek, Google, DeepInfra, NovitaAI, Fireworks) or explicitly (most notably Anthropic). This capability can significantly reduce costs in agentic workflows, where a single session often re-sends the same context repeatedly (for example, when the model performs multiple tool calls in sequence and the shared conversation/context is included each time). Today, Nebius Token Factory is at a cost disadvantage in these repeated, input-token-heavy scenarios compared to providers that support prompt caching and pass the savings through to customers. Please add support for prompt caching (implicit or explicit), including discounted pricing for cached prompt tokens, to improve cost-efficiency for agentic and tool-using applications.

New DeepSeek R1 Model

Please add deepseek-r1 model. DeepSeek’s first-generation reasoning models, achieving performance comparable to OpenAI-o1 across math, code, and reasoning tasks.

Usage and Balance API on Token Factory

Please add an API for viewing live usage with daily breakdowns and the ability to query our current balance as I can’t seem to find an auto top-up function on Token Factory. Thank you!

Account Balance API endpoint for Nebius Token Factory

Please add an API endpoint to retrieve the current account balance for a Nebius Token Factory account

DeepSeek v3

New Deepseek v3 scores really high on Aider leaderboards: https://aider.chat/docs/leaderboards/ Model: https://huggingface.co/deepseek-ai/DeepSeek-V3-Base/tree/main Would probably be the new best model on the platform. Is 600B+ params a problem ? Would the pricing be similar to LLama 3 405B ? Merry Christmas!

Function call support

Try inference engine with function call support + provide working example and list limitations (i.e. some inference libraries do not allow ‘auto’ tool_choice, for example)

Add Kimi K2 model

Feature Request: Add Cross-Encoder Reranking Support via /rerank or /score Endpoint

It would be great to have native support for reranking in the Nebius AI API, especially for use in retrieval-augmented generation (RAG) or semantic search workflows. See https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html for the /rerank and /score API. Add a dedicated endpoint for reranking such as the /score or the /rerank endpoind. Add cross-encoder models to the available model library

Show model quantization info

Multimodal models

Would be nice to work with picture generation as well

DeepSeek V3

One of the best news we received at the end of 2024, DeepSeek V3 is a large and intelligent AI model, compared to Claude 3.5 Sonnet and even being superior in some aspects according to LM Arena (https://lmarena.ai/?leaderboard)

Solution library

It would be really helpful to have a solution library that offers end-to-end production solutions we can manage ourselves and install using Terraform. Right now, there's no easy way to get started with your products.