The full-control path: any model, any endpoint — and every safety net is now your code. Part 3 of the External AI series.
Vantage Components manufactures industrial sensors. Their support engineers triage failure reports that mix free text with cryptic diagnostic codes. The data science team fine-tuned an open-weight model on ten years of failure reports and hosts it on the company's own private cloud — behind their VPN, on their GPUs, at a fixed monthly cost. It classifies failure modes better than any frontier model they tested, because it has seen their diagnostic codes.
Self-hosted means no Einstein Studio connection type reaches it — BYO LLM (Part 2) registers models on supported providers, not arbitrary private endpoints. So this is a legitimate Path 3 case from the decision guide: a raw Apex callout. This post builds it properly — credentials, client class, retries, error taxonomy — and then spells out when you should not be on this path at all.
Never put an API key in Apex, custom settings, custom metadata or labels. The platform's split model keeps the secret out of your code entirely:
callout:Name and never sees the secret.Setup for Vantage's bearer-token endpoint (Setup → Named Credentials):
Vantage_LLM_Auth.LLM_Service) and an authentication parameter holding the token — call it ApiKey. The value is encrypted at rest and write-only after save.Authorization, value {!'Bearer ' & $Credential.Vantage_LLM_Auth.ApiKey} — the formula injects the secret at callout time.Vantage_LLM with URL https://llm.internal.vantage.example, referencing the External Credential. Enable "Generate Authorization Header" only if you're not building it yourself via the custom header — here we are, so leave it off.LLM Callout Access) and assign it to the integration users. No permission set, no callout — this is your first access-control layer.One class owns the callout. Everything else in the org calls this and nothing else — that discipline is what makes Part 5's masking and audit enforceable later.
public with sharing class VantageLlmClient {
public class LlmException extends Exception {}
public class LlmRateLimitException extends Exception {}
private static final Integer TIMEOUT_MS = 120000;
private static final Integer MAX_ATTEMPTS = 3;
public static String complete(String systemPrompt, String userPrompt) {
Integer attempt = 0;
while (true) {
attempt++;
HttpResponse res;
try {
res = new Http().send(buildRequest(systemPrompt, userPrompt));
} catch (System.CalloutException e) {
// Network failure / timeout — retryable
if (attempt >= MAX_ATTEMPTS) {
throw new LlmException(
'LLM unreachable after ' + attempt + ' attempts: '
+ e.getMessage());
}
continue;
}
Integer code = res.getStatusCode();
if (code == 200) {
return parseAnswer(res.getBody());
}
if (code == 429 || code >= 500) {
// Transient: rate limit or server error — retryable
if (attempt >= MAX_ATTEMPTS) {
throw new LlmRateLimitException(
'LLM returned ' + code + ' after '
+ attempt + ' attempts');
}
continue;
}
// 4xx other than 429: OUR bug (bad auth, bad payload).
// Retrying cannot fix it — fail loud, immediately.
throw new LlmException('LLM rejected request: ' + code
+ ' ' + res.getBody().abbreviate(500));
}
}
private static HttpRequest buildRequest(String systemPrompt,
String userPrompt) {
HttpRequest req = new HttpRequest();
req.setEndpoint('callout:Vantage_LLM/v1/chat/completions');
req.setMethod('POST');
req.setHeader('Content-Type', 'application/json');
req.setTimeout(TIMEOUT_MS);
req.setBody(JSON.serialize(new Map<String, Object>{
'model' => 'vantage-failure-triage',
'temperature' => 0.2,
'max_tokens' => 400,
'messages' => new List<Object>{
new Map<String, Object>{
'role' => 'system', 'content' => systemPrompt },
new Map<String, Object>{
'role' => 'user', 'content' => userPrompt }
}
}));
return req;
}
private static String parseAnswer(String responseBody) {
Map<String, Object> parsed =
(Map<String, Object>) JSON.deserializeUntyped(responseBody);
List<Object> choices = (List<Object>) parsed.get('choices');
if (choices == null || choices.isEmpty()) {
throw new LlmException('LLM returned no choices: '
+ responseBody.abbreviate(500));
}
Map<String, Object> first =
(Map<String, Object>) choices[0];
Map<String, Object> message =
(Map<String, Object>) first.get('message');
return (String) message.get('content');
}
}Design decisions worth stealing:
LlmRateLimitException and defer to async, while a plain LlmException should page someone.deserializeUntyped plus null checks — an external model's error body will not match your happy-path shape, and a bare NullPointerException in production tells you nothing.Apex has no sleep. The loop above retries immediately, which is fine for a blip but wrong for a genuinely rate-limited endpoint. For background work — Vantage classifies new failure reports on creation, nobody is watching a spinner — the retry with actual delay comes from re-enqueueing:
public class FailureTriageJob implements Queueable,
Database.AllowsCallouts {
private Id caseId;
private Integer attempt;
public FailureTriageJob(Id caseId, Integer attempt) {
this.caseId = caseId;
this.attempt = attempt;
}
public void execute(QueueableContext ctx) {
Case c = [SELECT Description FROM Case
WHERE Id = :caseId WITH USER_MODE];
try {
String label = VantageLlmClient.complete(
'You are a failure-mode classifier. Answer with ' +
'exactly one label from: SENSOR_DRIFT, SEAL_FAILURE, ' +
'FIRMWARE, WIRING, UNKNOWN.',
c.Description);
update as user new Case(Id = caseId,
Failure_Mode__c = label.trim());
} catch (VantageLlmClient.LlmRateLimitException e) {
// Real backoff: re-enqueue with a delay, up to 5 tries.
if (attempt < 5) {
System.enqueueJob(
new FailureTriageJob(caseId, attempt + 1),
attempt * 2); // delay in minutes: 2, 4, 6, 8
} else {
update as user new Case(Id = caseId,
Failure_Mode__c = 'UNKNOWN');
}
}
}
}
The enqueue-with-delay overload gives you minutes-scale backoff without burning synchronous limits, and the terminal fallback (UNKNOWN, not an unhandled exception) keeps the pipeline moving when the model is down. If the value your job writes back came from an LLM, validate it like user input — label.trim() here should really be checked against the allowed picklist values before the update; Part 5 makes that pattern explicit.
This path is legitimate for Vantage because no platform path can reach their model. Be suspicious of every other justification. You are on the wrong path if:
Sources: Named Credentials and External Credentials — Salesforce Help · Named Credentials as Callout Endpoints — Apex Developer Guide · Queueable Apex — Apex Developer Guide