bmel:LLMInferenceBias¶
Category: LLM & AI Observability · Returns: bmel:number
bmel:LLMInferenceBias(prompt: bmel:string, response: bmel:string, model: bmel:string)
Description¶
Measures semantic bias between a prompt and an LLM response by computing their cosine similarity at the embedding level. Returns a value in [0, 1]: values close to 1.0 indicate the response is semantically aligned with the prompt (low bias / high relevance); values close to 0.0 indicate semantic drift or topic deviation (high bias). Useful for detecting hallucinations, off-topic responses, and systematic model bias.
Arguments¶
| Parameter | Type | Required | Description |
|---|---|---|---|
prompt | bmel:string | ✅ | The original prompt or instruction sent to the LLM. |
response | bmel:string | ✅ | The LLM-generated response to evaluate. |
model | bmel:string | ✅ | LLM model identifier (e.g. 'gpt-4o', 'claude-3-5-sonnet'). Determines which embedding model is used internally: OpenAI models use text-embedding-3-large, Anthropic models use voyage-3, others fall back to the platform default. |
Example¶
bmel:LLMInferenceBias({chat:Request Payload}.$.prompt, {chat:Response Payload}.$.content, 'gpt-4o')