AI providers – connecting a language model¶
nexxcite does not contain a language model of its own. Every AI assistant uses the model of a provider you choose – a cloud service such as Azure OpenAI, OpenAI or Anthropic, or a model you run yourself. This chapter describes which providers can be connected, which settings they need and what you should know about data protection.
It is intended for administrators who set up AI assistants and for those responsible for data protection. How assistants themselves are configured is described in AI assistants.
1. Providers at a glance¶
The connection to a provider is a credentials definition in Admin Studio → Access → Credentials. The assistant refers to it in "Credentials"; the kind of credentials decides which interface nexxcite uses.
| Provider | Credentials kind | "API Schema" | "Model / Deployment" in the assistant |
|---|---|---|---|
| Azure OpenAI | "OpenAI Credentials" | "Azure OpenAI" | The name of the deployment in your Azure resource |
| OpenAI | "OpenAI Credentials" | "OpenAI compatible" | The model id, for example gpt-4o |
| Other services with an OpenAI-compatible interface – for example Mistral, Groq, OpenRouter | "OpenAI Credentials" | "OpenAI compatible" | The model id of the service, for example mistral-large-latest |
| Models you run yourself with an OpenAI-compatible server – for example Ollama, vLLM, llama.cpp | "OpenAI Credentials" | "OpenAI compatible" | The model name on that server, for example llama3.1:8b |
| Anthropic | "Anthropic Credentials" | – | The model id, for example claude-sonnet-4-5 |
nexxcite has no connection of its own for other interfaces. A provider can be used if it offers an interface compatible with OpenAI's chat completions. Whether a particular service or model works well with nexxcite – in particular with tools – depends on that service and model; test it before you rely on it.
Tools need a model that supports tool calls. Assistants that search records or work with the configuration call tools. Models without tool support can answer from their instructions only.
2. Settings¶
"OpenAI Credentials"¶
| Setting | Meaning | Default / Required |
|---|---|---|
| "API Schema" | "Azure OpenAI" or "OpenAI compatible". Azure OpenAI addresses a deployment; every other provider addresses a model on a base URL. | "Azure OpenAI" |
| "Endpoint" | For Azure OpenAI the URL of the resource, for example https://my-resource.openai.azure.com. Otherwise the base URL of the interface, for example https://api.mistral.ai/v1, https://api.openai.com/v1 or http://localhost:11434/v1. |
Required |
| "API Token" | The key of the service. | Required for Azure OpenAI. For "OpenAI compatible" only if the service requires one – a model you run yourself often does not. |
| "API Version" | Azure only: the version of the Azure OpenAI interface, for example 2024-10-21. |
empty = the default of the client library |
| "Structured Output" | Whether the service can answer in a given JSON format: "Auto (detect on first use)", "Supported", "Not supported" (section 3). | "Auto (detect on first use)" |
"Anthropic Credentials"¶
| Setting | Meaning | Default / Required |
|---|---|---|
| "API Key" | The key of your Anthropic account. | Required |
| "Endpoint" | Only needed for a gateway or proxy in front of Anthropic. | empty = https://api.anthropic.com |
| "API Version" | The value of the anthropic-version header. |
empty = 2023-06-01 |
Errors¶
nexxcite does not test the connection when you save. Errors appear when the assistant is used – in the chat as "Error: …" with the message of the provider, and in the application log. Errors that nexxcite detects before it contacts the provider:
| Message | Cause |
|---|---|
| "No credentials linked" | The assistant has no "Credentials". |
| "Linked credentials definition not found" | The credentials were deleted. |
| "No deployment/model configured on the assistant" | "Model / Deployment" is empty. |
| "OpenAIEndpoint not found" | "Endpoint" of the OpenAI credentials is empty. |
| "OpenAIToken not found" | "API Token" is empty although "API Schema" is "Azure OpenAI". |
| "Anthropic API key not found" | "API Key" is empty. |
| "Credentials type '…' is not supported for AI chat" | The assistant refers to credentials of another kind. |
3. Structured output and "AI Predict Value"¶
The action command "AI Predict Value" asks the model to choose one value from a list and to explain its choice. For this, nexxcite asks for the answer in a fixed JSON format.
| Provider | Behaviour |
|---|---|
| Anthropic | Always supported. |
| Azure OpenAI | Supported. "Auto (detect on first use)" counts as "Supported"; choose "Not supported" only for deployments that refuse it. |
| "OpenAI compatible" with "Auto (detect on first use)" | nexxcite tries the JSON format once. If the service refuses it, nexxcite repeats the request with the format described in the prompt and remembers that the service does not support it. |
| "OpenAI compatible" with "Not supported" | The format is always described in the prompt. The answer is then less reliable, depending on the model. |
Chats are not affected by this setting.
4. Prompt caching and costs¶
Providers usually charge per token – for what is sent to the model and for what it writes. A few points help to estimate the cost:
- An answer with tool calls consists of several requests. For every round of tool calls, the whole conversation so far – instructions, tool descriptions, attachments, messages, tool results – is sent again. An answer with five tool calls costs considerably more than five times a simple question.
- "Show Token Usage In Chat" in the assistant shows the tokens of every answer, summed over all its requests – as far as the provider reports them.
- Prompt caching: with Anthropic, nexxcite marks the tool descriptions, the instructions and the attached files as cacheable, so that repeated parts of a conversation are cheaper. "Disable Prompt Caching" switches this off. With OpenAI-style providers, caching is decided by the provider itself; in our assessment the switch has no effect there.
- "Max Output Tokens" limits the length of each answer, "Max Tool Iterations" the number of rounds of one answer.
The prices themselves are set by the provider. Check them in the provider's documentation.
5. Data protection¶
Summary¶
- nexxcite sends data directly from your server to the provider you configured. It does not route anything through servers of the manufacturer of nexxcite.
- What is sent: the instructions, the conversation, the attached files and the data the assistant reads with its tools – including record data.
- What the provider does with it – storage, logging, use for training, location of processing – is governed by your contract with the provider, not by nexxcite.
- With a model you run yourself, the data does not leave your network.
What is sent to the provider¶
| Data | Sent |
|---|---|
| The instructions of the assistant | with every request |
| The messages of the conversation | with every request of the conversation |
| Attached records | Their title and id; their field values when the assistant reads them with a tool |
| Records the assistant searches or reads | The fields the tool returns – within the permissions and read restrictions of the user |
| Attached files | Pictures and PDFs as they are; the text of other files |
| Definitions, documentation, execution history | Only for administration assistants with the corresponding tool groups, when they read them |
| Tool calls and their results | with the following requests of the same answer |
The tools of external MCP servers receive the arguments the model passes to them – which can contain data the model read before. See AI assistants.
What nexxcite keeps¶
| Data | Where | How long |
|---|---|---|
| Conversations | In the memory of the server | 60 minutes after the last activity, or until the server restarts |
| Uploaded chat files | In the configured file storage | Until they are deleted – they are not removed automatically when the conversation ends |
| Tool calls | Application log: user, assistant, tool, duration, size of the result. Arguments only at log level Debug. |
According to your log retention |
| A history of the conversation | Only if a chat block runs an action that writes it into an entity of your own | As you decide |
Keys and tokens¶
The keys of the credentials are stored as part of the credentials definition, without additional encryption by nexxcite, and are shown in plain text to everybody who may open the credentials. See API access and credentials. The access tokens of external MCP servers are stored in the assistant definition in the same way and are visible to everybody who may open the assistant.
Recommendations¶
For administrators
- Check the terms of the provider – in particular whether inputs are stored or used for training, and where they are processed. Many providers offer business terms or regions for this; the provider's documentation is authoritative.
- Narrow what the assistant can reach: "Searchable Entities", "Stored Query Tags" and "Enabled Tool Groups". Data the assistant cannot read is not sent.
- Leave the file upload off unless it is needed, and plan how uploaded files are cleaned up.
- Use a model you run yourself for data that must not leave your network – at the cost of the quality of answers, which depends on the model.
- Tell your users that they are talking to an AI and which data it uses, if your organisation or the law requires it (see AI assistants).
6. Limitations¶
- Only the two interfaces described here are supported: OpenAI-compatible and Anthropic.
- With OpenAI-compatible services other than OpenAI itself, PDF files are not passed to the model – only their names.
- When a request to the provider fails, the user sees the error in the chat and can send the message again. An answer that was interrupted is not continued.
- The connection is not tested when the credentials are saved.
7. Frequently asked questions¶
Can we use our own Azure OpenAI resource? Yes. Choose "Azure OpenAI" as "API Schema", enter the URL of the resource and its key, and the name of the deployment as "Model / Deployment" in the assistant.
Can we run nexxcite's AI without any cloud service? Yes, with a model you run yourself behind an OpenAI-compatible interface (for example Ollama or vLLM). Choose "OpenAI compatible" and its address as "Endpoint". The model must support tool calls for assistants that work with records or the configuration.
Does nexxcite use our data to train a model? No. nexxcite does not train models. Whether the provider does is governed by your contract with the provider.
Can different assistants use different providers? Yes. Every assistant names its own credentials and model.
Where do we see what an answer cost? Switch on "Show Token Usage In Chat" in the assistant. The prices per token are set by the provider.