ChatCompletionRequest
Creates a completion for the chat message.
Constructors
Properties
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
Controls how the model responds to function calls. FunctionMode.None means the model does not call a function, and responds to the end-user. FunctionMode.Auto means the model can pick between an end-user or calling a function. Specifying a particular function via FunctionMode.Named forces the model to call that function. FunctionMode.None is the default when no functions are present. FunctionMode.Auto is the default if functions are present.
A list of functions the model may generate JSON inputs for.
A unique identifier representing the Multi LORA reserved instance.
The messages to generate chat completions for.
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
An object specifying the format that the model must output.
If specified, our system will make the best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the systemFingerprint response parameter to monitor changes in the backend.
Options for streaming response. Only used when in streaming mode.
What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
Controls which (if any) function is called by the model.
An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.