Models and model cards

In the chat, users do not select a raw model but a model card. A card bundles a provider, a model and the presets that go with it under a comprehensible name. This page shows what you control with that as an administrator.

What a model card is

A card is a curated preset. Instead of a technical model name, the person reads a comprehensible label in the model picker and automatically gets the matching system prompt and the matching parameters. That has three advantages: the choice stays comprehensible, the system prompt is inserted server side and never lands in the browser, and you can tailor the selection per role and group.

A card carries, among other things, a visible label, a short description, its position in the picker, the provider and model behind it as well as presets such as temperature, greeting text and system prompt. Exactly one card is the default card.

You can additionally set whether only cards are selectable or whether the raw model list appears next to them.

Providers

The providers and the models they offer are maintained in the configuration. A connection test checks the connection without revealing the secret.

Keys only as a reference

An access key is never stored in the configuration in plain text, only as a reference to a value held server side. Plain text is rejected. Alternatively each person stores their own key in the chat.

Visibility per role and group

Which cards somebody sees is defined at the level of the role and of the group. Where there is no rule of its own, the shipped selection applies. The principle: the narrower rule wins.

In the role editor you switch the cards on or off individually and hide the raw model list if you want to; a group gets a selection of its own where needed. On both levels the selection can be reset to the shipped state in one step.

Save once more after changing cards

If you rename or add a card, save the affected role and group rules once more and then check that the visibility still stands as you intend it.

Restricting providers

If an endpoint runs through an intermediary that in turn uses several providers, you can restrict which of them may serve your requests: only providers without storage and without training on the prompts, and on request only providers with a registered office or data center in the EU, the EEA and Switzerland.

Two limitations

First: the information on registered office and data center comes from the intermediary, and the registered office of a provider is no guarantee of the place of processing. Treat “EU only” as a sensible restriction, not as an assurance.

Second: models without a matching provider will return an error afterwards. If you set the list narrowly, check every card that runs through this endpoint.

Model router

Next to the fixed cards there is the option of distributing requests automatically to a suitable model, instead of having every question answered by the most expensive one. If the assessment fails, the request goes to the stronger model: in case of doubt the router decides for quality, not for the saving.

Not active as shipped

The router is not rolled out in the customer template. After installation your team sees the shipped model cards. The context is described in Chat and models.