AI Capabilities Reference

BankLingo's AI Model System provides 36 built-in capabilities that you can use in BPMN workflows through ServiceTask nodes. Each capability is backed by a dedicated handler that knows how to call the right AI provider, format the input, and normalize the response.

How It Works — Architecture Overview

┌───────────────────────────────────────────────────────────────────┐
│                     BPMN ServiceTask                              │
│  serviceType = "AI Model"                                        │
│  aiConfigKey = "AI:fraud-detection-v3"                            │
│  aiInputMapping = { "data": { "amount": ${context.amount} } }    │
└───────────────────┬───────────────────────────────────────────────┘
                    │ doCmd('ExecuteAIModel', { configKey, input })
                    ▼
┌───────────────────────────────────────────────────────────────────┐
│              ExecuteAIModelCommand Handler                         │
│  1. Load AIModelConfig from Tenant Configuration (secure storage) │
│  2. Read config.Capability → route to matching handler            │
│  3. Handler calls external AI provider via HTTP                   │
│  4. Return standardized AIModelResult                             │
└───────────────────┬───────────────────────────────────────────────┘
                    │
                    ▼
┌───────────────────────────────────────────────────────────────────┐
│  ICapabilityHandler (e.g., AnomalyDetectionHandler)               │
│  [Capability("anomaly-detection")]                                │
│  → ExecuteAsync(config, input, timeoutMs, cancellationToken)      │
│  → Returns AIModelResult { Data, Confidence, Explanation, ... }   │
└───────────────────────────────────────────────────────────────────┘

Step-by-Step Flow

Designer sets serviceType = "AI Model" on a BPMN ServiceTask.
Designer picks a capability from the dropdown (populated by GetAIModelCapabilities command).
Designer sets aiConfigKey (e.g., "AI:fraud-detection-v3") — this is the key in Tenant Configuration that holds the provider credentials and parameters.
Designer sets aiInputMapping — a JSON template using ${context.xxx} syntax to map process variables to the AI model's expected input shape.
At runtime, the BPMN engine evaluates the input mapping, replacing ${context.xxx} with actual process variable values.
The engine calls doCmd('ExecuteAIModel', { configKey, input, timeoutMs }).
ExecuteAIModelCommandHandler loads the AIModelConfig JSON from secure tenant storage.
Based on config.Capability, it routes to the correct ICapabilityHandler.
The handler calls the external AI provider and returns an AIModelResult.
The engine stores the result in the process variables for downstream tasks.

Configuration Structure (AIModelConfig)

Every AI capability requires a JSON configuration stored in Tenant Configuration:

{
  "Capability": "anomaly-detection",
  "Provider": "internal",
  "Model": "fraud-detection-v3",
  "Endpoint": "https://ml.internal.bank.com/v3/fraud/predict",
  "ApiKey": "your-api-key-here",
  "TimeoutMs": 5000,
  "Parameters": {
    "threshold": "0.7",
    "windowSize": "100"
  }
}

Field	Type	Description
`Capability`	string	Required. The capability name (e.g., `"text-generation"`, `"anomaly-detection"`). Determines which handler processes the request.
`Provider`	string	Provider identifier (e.g., `"openai"`, `"azure-ai"`, `"huggingface"`, `"internal"`).
`Model`	string	Model name/version (e.g., `"gpt-4-turbo"`, `"fraud-detection-v3"`).
`Endpoint`	string	Required. The full API endpoint URL.
`ApiKey`	string	API key or token. For Azure services, sent as `Ocp-Apim-Subscription-Key` header. For others, sent as `Bearer` token.
`TimeoutMs`	int	Request timeout in milliseconds. Default: `30000`.
`Parameters`	Dictionary	Capability-specific parameters (temperature, topK, etc.). All values are strings.

Where to Store

Administration → Tenant Configuration

Key: AI:fraud-detection-v3 (matches the aiConfigKey in the BPMN ServiceTask)
Value: The JSON string above
IsSensitive: true (because it contains the ApiKey — will be encrypted at rest)

Standardized Response Format (AIModelResult)

Every capability returns the same envelope:

{
  "result": { ... },
  "confidence": 0.87,
  "modelVersion": "gpt-4-turbo",
  "processingTimeMs": 245,
  "explanation": "High risk due to unusual transaction pattern",
  "modelId": "fraud-detection-v3",
  "capability": "anomaly-detection",
  "provider": "internal",
  "metadata": { ... }
}

Field	Type	Description
`result`	object	The actual model output. Shape varies by capability (see each capability section below).
`confidence`	double	Confidence score, `0.0` to `1.0`. Defaults to `1.0` if the model doesn't provide one.
`modelVersion`	string	Model version string.
`processingTimeMs`	long	Time taken to process the request.
`explanation`	string	Optional human-readable explanation of the result.
`modelId`	string	The model identifier from the config key.
`capability`	string	The capability that was executed.
`provider`	string	The provider that was used.
`metadata`	object	Additional metadata (token usage, image source, etc.).

Accessing Results in Downstream Tasks

After the AI ServiceTask executes, the result is stored in process variables. You can access it in subsequent script tasks or conditional gateways:

// In a Script Task
var aiResult = context.aiModelResult;
var fraudScore = aiResult.result.anomalyScore;
var isHighRisk = aiResult.confidence > 0.8;

// In a Gateway condition
${context.aiModelResult.confidence > 0.7}

Capability Categories

Category	Capabilities	Count
Text & NLP	text-generation, sentiment-analysis, text-classification, named-entity-recognition, text-summarization, question-answering	6
Vision	image-classification, object-detection, face-detection, pose-estimation, optical-character-recognition, video-classification, image-segmentation	7
Audio & Speech	speech-to-text, text-to-speech, audio-classification, music-generation, translation	5
Creative	image-generation	1
Machine Learning	anomaly-detection, recommendation, forecasting, clustering, regression	5
Code	code-generation, code-completion, code-review, bug-detection	4
Advanced NLP	embeddings, semantic-search, moderation, toxicity-detection, zero-shot-classification, few-shot-learning, paraphrase-generation, grammar-correction	8
Custom	custom-model	1

Text & NLP

text-generation

What it does: Generate text responses using large language models (GPT-4, Claude, LLaMA). The most versatile capability — it can handle document analysis, customer response drafting, data extraction, and any general text-in/text-out scenario.

Real-life banking use cases:

Draft loan approval/rejection letters from structured data
Analyze unstructured customer complaint text to extract key issues
Generate personalized product recommendations as text
Extract fields from uploaded documents by prompting the model

Configuration:

{
  "Capability": "text-generation",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 60000,
  "Parameters": {
    "temperature": "0.7",
    "maxTokens": "4096",
    "systemPrompt": "You are a helpful banking assistant.",
    "topP": "1.0",
    "responseFormat": "json_object"
  }
}

Parameter	Type	Default	Description
`temperature`	string (double)	`"0.7"`	Controls randomness. `0.0` = deterministic, `1.0` = creative.
`maxTokens`	string (int)	`"4096"`	Maximum tokens in the response.
`systemPrompt`	string	`"You are a helpful AI assistant."`	The system message that sets the model's behavior.
`topP`	string (double)	`"1.0"`	Nucleus sampling parameter.
`responseFormat`	string	—	Set to `"json_object"` to force JSON output from OpenAI.

Input Mapping:

{ "prompt": "${context.analysisPrompt}" }

Input Field	Required	Description
`prompt`	✅ Yes	The text prompt to send to the model.

Response (result):

{
  "text": "Based on the analysis of your transaction history..."
}

When responseFormat = "json_object", the response is the parsed JSON object directly instead of { text }.

Metadata: completionTokens, promptTokens, totalTokens

sentiment-analysis

What it does: Analyze the emotional tone of text, returning positive/negative/neutral/mixed sentiment with a confidence score.

Real-life banking use cases:

Analyze customer feedback and complaints to prioritize follow-ups
Monitor customer satisfaction in chat transcripts
Auto-flag negative sentiment in branch visit surveys
Track sentiment trends across customer communication channels

Configuration:

{
  "Capability": "sentiment-analysis",
  "Provider": "azure-ai",
  "Model": "text-analytics-v3",
  "Endpoint": "https://your-resource.cognitiveservices.azure.com/text/analytics/v3.1/sentiment",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 10000,
  "Parameters": {
    "language": "en"
  }
}

Parameter	Type	Default	Description
`language`	string	`"en"`	Language code for the text to analyze.

Input Mapping:

{ "text": "${context.customerFeedback}" }

Input Field	Required	Description
`text`	✅ Yes	The text to analyze for sentiment.

Response (result): Raw provider response. For Azure Text Analytics:

{
  "documents": [
    {
      "id": "1",
      "sentiment": "negative",
      "confidenceScores": { "positive": 0.05, "neutral": 0.10, "negative": 0.85 }
    }
  ]
}

Providers: azure-ai, huggingface, openai

text-classification

What it does: Categorize text into predefined labels using zero-shot or fine-tuned models. You provide a set of candidate labels, and the model scores each label for the given text.

Real-life banking use cases:

Route support tickets to the correct department (loans, cards, complaints, fraud)
Classify uploaded documents by type (ID card, utility bill, bank statement)
Categorize transaction narrations (salary, bill payment, transfer, reversal)
Auto-tag customer communication themes

Configuration:

{
  "Capability": "text-classification",
  "Provider": "huggingface",
  "Model": "distilbert-base-uncased",
  "Endpoint": "https://api-inference.huggingface.co/models/distilbert-base-uncased",
  "ApiKey": "hf_...",
  "TimeoutMs": 10000,
  "Parameters": {
    "candidateLabels": "account-inquiry,loan-request,complaint,fraud-report,general"
  }
}

Parameter	Type	Default	Description
`candidateLabels`	string	—	Comma-separated list of classification labels.

Input Mapping:

{ "text": "${context.ticketDescription}" }

Input Field	Required	Description
`text`	✅ Yes	The text to classify.

Response (result): Raw HuggingFace response:

{
  "sequence": "I can't access my account and I'm being charged fees",
  "labels": ["complaint", "account-inquiry", "fraud-report", "loan-request", "general"],
  "scores": [0.72, 0.18, 0.05, 0.03, 0.02]
}

Providers: huggingface, azure-ai, openai

named-entity-recognition

What it does: Extract structured entities (names, amounts, dates, locations, organizations) from unstructured text. Returns entity text, type, and position.

Real-life banking use cases:

Extract customer name, amount, and date from free-text loan applications
Parse cheque images (after OCR) to pull out payee, amount, and date
Extract account numbers and transaction amounts from unstructured emails
Identify organization names in KYC documents

Configuration:

{
  "Capability": "named-entity-recognition",
  "Provider": "azure-ai",
  "Model": "text-analytics-ner-v3",
  "Endpoint": "https://your-resource.cognitiveservices.azure.com/text/analytics/v3.1/entities/recognition/general",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 10000,
  "Parameters": {
    "language": "en"
  }
}

Parameter	Type	Default	Description
`language`	string	`"en"`	Language code.

Input Mapping:

{ "text": "${context.documentText}" }

Input Field	Required	Description
`text`	✅ Yes	The unstructured text to extract entities from.

Response (result): Raw provider response. For Azure Text Analytics:

{
  "documents": [
    {
      "id": "1",
      "entities": [
        { "text": "John Smith", "category": "Person", "offset": 0, "length": 10, "confidenceScore": 0.99 },
        { "text": "₦500,000", "category": "Quantity", "subcategory": "Currency", "offset": 25, "length": 8, "confidenceScore": 0.95 },
        { "text": "January 15, 2026", "category": "DateTime", "offset": 38, "length": 16, "confidenceScore": 0.98 }
      ]
    }
  ]
}

Providers: azure-ai, huggingface, openai

text-summarization

What it does: Condense long documents into concise summaries. Uses an LLM with a summarization-focused system prompt.

Real-life banking use cases:

Summarize lengthy loan application documents for officer review
Condense customer communication history before escalation
Summarize regulatory documents for compliance team
Create executive summaries of audit reports

Configuration:

{
  "Capability": "text-summarization",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "systemPrompt": "Summarize the following text concisely in 3-5 sentences.",
    "maxTokens": "500",
    "temperature": "0.3"
  }
}

Parameter	Type	Default	Description
`systemPrompt`	string	`"Summarize the following text concisely."`	Customize the summarization instruction.
`maxTokens`	string (int)	`"500"`	Maximum tokens for the summary.
`temperature`	string (double)	`"0.3"`	Lower = more factual.

Input Mapping:

{ "text": "${context.documentContent}" }

Input Field	Required	Description
`text`	✅ Yes	The text to summarize.

Response (result):

{
  "summary": "The customer applied for a ₦5M personal loan on January 15. Employment verification confirms 8 years at current employer with a monthly income of ₦450,000. The debt-to-income ratio is 32%, within acceptable limits.",
  "originalLength": 4521,
  "summaryLength": 248
}

Metadata: completionTokens, promptTokens, totalTokens

question-answering

What it does: Answer specific questions, optionally based on provided context. If context is provided, the model answers strictly from that context. Without context, it uses its general knowledge.

Real-life banking use cases:

Answer policy questions from uploaded regulatory documents
FAQ system for internal staff based on procedure manuals
Extract specific facts from lengthy documents ("What is the customer's monthly income?")
Customer-facing Q&A chatbot backed by product documentation

Configuration:

{
  "Capability": "question-answering",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "systemPrompt": "Answer questions based on the provided context. Be concise and factual.",
    "temperature": "0.1"
  }
}

Parameter	Type	Default	Description
`systemPrompt`	string	`"Answer the question based on the provided context..."`	Sets the answering behavior.
`temperature`	string (double)	`"0.1"`	Low = factual answers.

Input Mapping:

{
  "question": "${context.userQuestion}",
  "context": "${context.policyDocument}"
}

Input Field	Required	Description
`question`	✅ Yes	The question to answer.
`context`	❌ No	Background context for the model to use when answering.

Response (result):

{
  "answer": "The maximum loan tenure for personal loans is 60 months (5 years) according to the current policy."
}

Metadata: completionTokens, promptTokens, totalTokens

Vision

image-classification

What it does: Classify images into predefined categories. The model looks at an image and returns labels with confidence scores.

Real-life banking use cases:

Classify uploaded documents (ID card, passport, utility bill, bank statement)
Verify document type matches what the customer claims to have uploaded
Sort scanned documents in batch processing

Configuration:

{
  "Capability": "image-classification",
  "Provider": "huggingface",
  "Model": "microsoft/resnet-50",
  "Endpoint": "https://api-inference.huggingface.co/models/microsoft/resnet-50",
  "ApiKey": "hf_...",
  "TimeoutMs": 15000,
  "Parameters": {
    "topK": "5"
  }
}

Parameter	Type	Default	Description
`topK`	string (int)	`"5"`	Number of top classification labels to return.

Input Mapping:

{ "imageUrl": "${context.uploadedDocumentUrl}" }

Input Field	Required	Description
`imageUrl`	✅ (one of)	URL of the image to classify.
`imageData`	✅ (one of)	Base64-encoded image data (alternative to URL).

Response (result): Raw provider response:

[
  { "label": "national_id_card", "score": 0.92 },
  { "label": "passport", "score": 0.05 },
  { "label": "utility_bill", "score": 0.02 }
]

Metadata: imageSource ("url" or "base64")

object-detection

What it does: Detect and locate objects in images with bounding boxes. Returns object labels, confidence scores, and position coordinates.

Real-life banking use cases:

Detect security threats in branch surveillance footage
Identify document elements (signatures, stamps, photos) in scanned documents
Verify that uploaded ID photos contain a face region

Configuration:

{
  "Capability": "object-detection",
  "Provider": "azure-ai",
  "Model": "yolo-v8",
  "Endpoint": "https://your-resource.cognitiveservices.azure.com/vision/v3.2/detect",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 15000,
  "Parameters": {
    "confidenceThreshold": "0.5"
  }
}

Parameter	Type	Default	Description
`confidenceThreshold`	string (double)	`"0.5"`	Minimum confidence to include a detection.

Input Mapping:

{ "imageUrl": "${context.imageUrl}" }

Input Field	Required	Description
`imageUrl`	✅ (one of)	URL of the image.
`imageData`	✅ (one of)	Base64-encoded image data.

Response (result): Raw provider response (provider-dependent bounding box format).

face-detection

What it does: Detect faces in images with optional attribute analysis (age, emotion, gender). Uses Azure Face API.

Real-life banking use cases:

KYC verification: Detect face in uploaded ID photo and compare with selfie
Liveness detection: Verify that video KYC captures contain a real face
Emotion analysis: Gauge customer emotion during video banking sessions

Configuration:

{
  "Capability": "face-detection",
  "Provider": "azure-ai",
  "Model": "azure-face-api-v1",
  "Endpoint": "https://your-resource.cognitiveservices.azure.com/face/v1.0/detect",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 15000,
  "Parameters": {
    "returnFaceAttributes": "age,gender,emotion",
    "returnFaceLandmarks": "true"
  }
}

Parameter	Type	Default	Description
`returnFaceAttributes`	string	—	Comma-separated attributes to return (age, gender, emotion, etc.).
`returnFaceLandmarks`	string	`"false"`	Whether to return facial landmark coordinates.

Input Mapping:

{ "imageUrl": "${context.customerPhotoUrl}" }

Input Field	Required	Description
`imageUrl`	✅ Yes	URL of the image containing face(s).

Response (result): Raw Azure Face API response:

[
  {
    "faceId": "...",
    "faceRectangle": { "top": 120, "left": 200, "width": 180, "height": 200 },
    "faceAttributes": {
      "age": 35,
      "gender": "male",
      "emotion": { "happiness": 0.8, "neutral": 0.15, "surprise": 0.05 }
    }
  }
]

optical-character-recognition

What it does: Extract text from images and scanned documents. Returns recognized text with region/line/word positions.

Real-life banking use cases:

Cheque reading: Extract payee name, amount (in words and figures), date, and account number
ID extraction: Pull customer name, date of birth, ID number from scanned national IDs
Document digitization: Convert paper statements or forms to searchable text
Signature verification preprocessing: Extract handwritten text regions

Configuration:

{
  "Capability": "optical-character-recognition",
  "Provider": "azure-ai",
  "Model": "azure-ocr-v4",
  "Endpoint": "https://your-resource.cognitiveservices.azure.com/vision/v3.2/ocr",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 20000,
  "Parameters": {
    "language": "en",
    "detectOrientation": "true"
  }
}

Parameter	Type	Default	Description
`language`	string	`"en"`	Expected language of text in the image.
`detectOrientation`	string	`"true"`	Auto-detect and correct page orientation.

Input Mapping:

{ "imageUrl": "${context.scannedDocumentUrl}" }

Input Field	Required	Description
`imageUrl`	✅ (one of)	URL of the image/document.
`imageData`	✅ (one of)	Base64-encoded image data.

Response (result): Raw Azure OCR response:

{
  "language": "en",
  "textAngle": 0,
  "orientation": "Up",
  "regions": [
    {
      "boundingBox": "...",
      "lines": [
        {
          "boundingBox": "...",
          "words": [
            { "boundingBox": "...", "text": "JOHN" },
            { "boundingBox": "...", "text": "SMITH" }
          ]
        }
      ]
    }
  ]
}

pose-estimation

What it does: Estimate human body pose keypoints from images. Returns skeleton data with joint positions.

Real-life banking use cases:

Specialized use case for physical security or biometric verification systems.

Configuration:

{
  "Capability": "pose-estimation",
  "Provider": "custom-http",
  "Model": "mediapipe-pose",
  "Endpoint": "https://your-pose-api.com/v1/estimate",
  "ApiKey": "YOUR_KEY",
  "TimeoutMs": 20000,
  "Parameters": {}
}

Input Mapping:

{ "imageUrl": "${context.imageUrl}" }

Input Field	Required	Description
`imageUrl`	✅ Yes	URL of the image.

Response (result): Raw provider response with pose keypoints.

video-classification

What it does: Classify video content into categories. Analyzes video frames to determine content type.

Real-life banking use cases:

Video KYC: Verify that the recorded video contains the expected steps (face shown, ID shown, statement read)
Security footage classification (normal activity, suspicious activity, after-hours access)

Configuration:

{
  "Capability": "video-classification",
  "Provider": "azure-ai",
  "Model": "azure-video-indexer",
  "Endpoint": "https://api.videoindexer.ai/...",
  "ApiKey": "YOUR_KEY",
  "TimeoutMs": 60000,
  "Parameters": {}
}

Input Mapping:

{ "videoUrl": "${context.videoRecordingUrl}" }

Input Field	Required	Description
`videoUrl`	✅ Yes	URL of the video to classify.

Response (result): Raw provider response.

image-segmentation

What it does: Segment images into distinct regions (foreground/background, object boundaries). Each pixel is assigned to a segment.

Real-life banking use cases:

Extract document regions from photographed documents with complex backgrounds
Isolate signature regions from scanned forms
Separate text from non-text areas in mixed-content documents

Configuration:

{
  "Capability": "image-segmentation",
  "Provider": "custom-http",
  "Model": "segment-anything",
  "Endpoint": "https://your-segmentation-api.com/v1/segment",
  "ApiKey": "YOUR_KEY",
  "TimeoutMs": 30000,
  "Parameters": {}
}

Input Mapping:

{ "imageUrl": "${context.imageUrl}" }

Input Field	Required	Description
`imageUrl`	✅ Yes	URL of the image to segment.

Response (result): Raw provider response with segment masks/regions.

Audio & Speech

speech-to-text

What it does: Transcribe audio recordings to text. Uses OpenAI Whisper or similar ASR models.

Real-life banking use cases:

Call recording transcription: Convert customer service calls to searchable text
Voice notes: Transcribe field officer voice notes attached to loan applications
Meeting minutes: Auto-transcribe branch meetings and compliance calls
Voice commands: Convert IVR voice input to text for processing

Configuration:

{
  "Capability": "speech-to-text",
  "Provider": "openai",
  "Model": "whisper-1",
  "Endpoint": "https://api.openai.com/v1/audio/transcriptions",
  "ApiKey": "sk-...",
  "TimeoutMs": 60000,
  "Parameters": {
    "language": "en",
    "responseFormat": "verbose_json"
  }
}

Parameter	Type	Default	Description
`language`	string	—	ISO language code (e.g., `"en"`, `"yo"`, `"ha"`).
`responseFormat`	string	—	Response format: `"json"`, `"verbose_json"`, `"text"`, `"srt"`, `"vtt"`.

Input Mapping:

{ "audioUrl": "${context.callRecordingUrl}" }

Input Field	Required	Description
`audioUrl`	✅ (one of)	URL of the audio file.
`audioData`	✅ (one of)	Base64-encoded audio data.

Response (result): Raw Whisper response:

{
  "text": "Hello, I would like to inquire about my account balance and recent transactions...",
  "language": "english",
  "duration": 45.2,
  "segments": [...]
}

text-to-speech

What it does: Convert text to spoken audio. Returns audio data in the specified format. Uses OpenAI TTS or similar services.

Real-life banking use cases:

IVR prompts: Generate dynamic voice prompts for interactive voice response systems
Accessibility: Read account statements aloud for visually impaired customers
Notifications: Convert text alerts to voice messages for phone delivery
Queue management: Generate audio announcements in branch

Configuration:

{
  "Capability": "text-to-speech",
  "Provider": "openai",
  "Model": "tts-1",
  "Endpoint": "https://api.openai.com/v1/audio/speech",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "defaultVoice": "alloy",
    "responseFormat": "mp3",
    "speed": "1.0"
  }
}

Parameter	Type	Default	Description
`defaultVoice`	string	`"alloy"`	Voice to use (e.g., `"alloy"`, `"echo"`, `"fable"`, `"onyx"`, `"nova"`, `"shimmer"`).
`responseFormat`	string	`"mp3"`	Audio format: `"mp3"`, `"opus"`, `"aac"`, `"flac"`.
`speed`	string (double)	`"1.0"`	Playback speed multiplier (`0.25` to `4.0`).

Input Mapping:

{ "text": "${context.messageToSpeak}", "voice": "alloy" }

Input Field	Required	Description
`text`	✅ Yes	The text to convert to speech.
`voice`	❌ No	Override the voice. Falls back to config `defaultVoice`.

Response (result): Raw audio response data (binary/base64).

Metadata: voice

audio-classification

What it does: Classify audio recordings into categories (speech, music, noise, specific sounds).

Real-life banking use cases:

Call center quality analysis: Classify calls by type (complaint, inquiry, escalation)
Detect background noise or distortion in voice recordings
Classify audio events in branch security recordings

Configuration:

{
  "Capability": "audio-classification",
  "Provider": "huggingface",
  "Model": "wav2vec2-base",
  "Endpoint": "https://api-inference.huggingface.co/models/facebook/wav2vec2-base-960h",
  "ApiKey": "hf_...",
  "TimeoutMs": 30000,
  "Parameters": {}
}

Input Mapping:

{ "audioUrl": "${context.audioFileUrl}" }

Input Field	Required	Description
`audioUrl`	✅ (one of)	URL of the audio file.
`audioData`	✅ (one of)	Base64-encoded audio data.

Response (result): Raw provider response with classification labels and scores.

music-generation

What it does: Generate music from text descriptions. Specialized capability for audio content creation.

Real-life banking use cases:

Generate hold music or IVR background soundscapes
Create branded audio jingles for notifications

Configuration:

{
  "Capability": "music-generation",
  "Provider": "custom-http",
  "Model": "musicgen-medium",
  "Endpoint": "https://your-music-api.com/v1/generate",
  "ApiKey": "YOUR_KEY",
  "TimeoutMs": 120000,
  "Parameters": {
    "duration": "30",
    "format": "mp3"
  }
}

Parameter	Type	Default	Description
`duration`	string (int)	`"30"`	Duration of generated music in seconds.
`format`	string	`"mp3"`	Output format.

Input Mapping:

{ "prompt": "${context.musicDescription}" }

Input Field	Required	Description
`prompt`	✅ Yes	Text description of the desired music.

Response (result): Raw provider response with generated audio data.

translation

What it does: Translate text between languages. Uses Azure Translator API or similar services.

Real-life banking use cases:

Multi-language communications: Translate customer-facing messages to local languages
Cross-border banking: Translate transaction narrations for international transfers
Regulatory compliance: Translate compliance documents between languages
Support channels: Real-time translation for customer support in multiple languages

Configuration:

{
  "Capability": "translation",
  "Provider": "azure-ai",
  "Model": "azure-translator-v3",
  "Endpoint": "https://api.cognitive.microsofttranslator.com/translate?api-version=3.0",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 10000,
  "Parameters": {
    "defaultTargetLanguage": "en",
    "region": "westeurope"
  }
}

Parameter	Type	Default	Description
`defaultTargetLanguage`	string	`"en"`	Default target language if not specified in input.
`region`	string	—	Azure Translator region (used as `Ocp-Apim-Subscription-Region` header).

Input Mapping:

{
  "text": "${context.textToTranslate}",
  "targetLanguage": "${context.targetLanguage}"
}

Input Field	Required	Description
`text`	✅ Yes	Text to translate.
`targetLanguage`	❌ No	Target language code (e.g., `"fr"`, `"yo"`, `"ha"`). Falls back to config default.

Response (result): Raw Azure Translator response:

[
  {
    "translations": [
      { "text": "Votre solde est de 500 000 nairas.", "to": "fr" }
    ]
  }
]

Metadata: targetLanguage

Creative

image-generation

What it does: Generate images from text descriptions using DALL-E or Stable Diffusion. Returns the generated image URL.

Real-life banking use cases:

Generate marketing visuals for product campaigns
Create personalized card designs based on customer preferences
Produce illustrative images for customer-facing documentation
Generate placeholder visuals for product prototypes

Configuration:

{
  "Capability": "image-generation",
  "Provider": "openai",
  "Model": "dall-e-3",
  "Endpoint": "https://api.openai.com/v1/images/generations",
  "ApiKey": "sk-...",
  "TimeoutMs": 60000,
  "Parameters": {
    "size": "1024x1024",
    "quality": "standard",
    "n": "1"
  }
}

Parameter	Type	Default	Description
`size`	string	`"1024x1024"`	Image dimensions (`"256x256"`, `"512x512"`, `"1024x1024"`, `"1792x1024"`, `"1024x1792"`).
`quality`	string	`"standard"`	Image quality (`"standard"` or `"hd"`).
`n`	string (int)	`"1"`	Number of images to generate.

Input Mapping:

{ "prompt": "${context.imagePrompt}" }

Input Field	Required	Description
`prompt`	✅ Yes	Text description of the image to generate.

Response (result):

{
  "imageUrl": "https://oaidalleapiprodscus.blob.core.windows.net/...",
  "revisedPrompt": "A modern professional banking lobby with marble floors...",
  "imageSize": "1024x1024"
}

Machine Learning

anomaly-detection

What it does: Detect anomalies and outliers in data points. The primary use case is real-time fraud detection — scoring transactions against a trained model.

Real-life banking use cases:

Transaction fraud detection: Score every transaction in real-time for fraud risk
Unusual account activity: Detect sudden spikes in transaction volume or amount
Login anomaly detection: Flag unusual login patterns (new device, unusual location)
AML monitoring: Detect money laundering patterns in transaction flows

Configuration:

{
  "Capability": "anomaly-detection",
  "Provider": "internal",
  "Model": "fraud-detection-v3",
  "Endpoint": "https://ml.internal.bank.com/v3/fraud/predict",
  "ApiKey": "YOUR_INTERNAL_KEY",
  "TimeoutMs": 5000,
  "Parameters": {
    "threshold": "0.7",
    "windowSize": "100"
  }
}

Parameter	Type	Default	Description
`threshold`	string (double)	—	Anomaly score threshold above which a transaction is flagged.
`windowSize`	string (int)	—	Number of recent transactions to consider for context.

Input Mapping:

{
  "data": {
    "amount": "${context.transactionAmount}",
    "channel": "${context.channel}",
    "accountNumber": "${context.senderAccount}"
  }
}

Input Field	Required	Description
`data`	✅ Yes	Object containing the data point features to evaluate.

Response (result): Raw provider response. The handler also extracts these fields into Confidence and Explanation if the provider returns them:

anomalyScore / confidence → AIModelResult.Confidence
explanation → AIModelResult.Explanation

{
  "isAnomaly": true,
  "anomalyScore": 0.92,
  "explanation": "Transaction amount ₦2,500,000 is 15x higher than average for this account",
  "contributingFactors": [
    { "feature": "amount", "impact": 0.6 },
    { "feature": "time_of_day", "impact": 0.2 },
    { "feature": "merchant_category", "impact": 0.12 }
  ]
}

Example BPMN workflow:

[Start] → [Prepare Transaction Data] → [AI: Fraud Detection] → <Gateway>
                                                                    ├── High Risk (confidence > 0.8) → [Block & Alert]
                                                                    ├── Medium Risk (0.5-0.8) → [Manual Review]
                                                                    └── Low Risk (< 0.5) → [Approve]

recommendation

What it does: Generate personalized recommendations based on user profile and context. Returns ranked list of recommended items.

Real-life banking use cases:

Next-best-product: Recommend financial products to customers (savings accounts, credit cards, loans)
Cross-sell suggestions: Suggest complementary services during customer interactions
Investment recommendations: Suggest investment options based on risk profile
Channel optimization: Recommend the best communication channel per customer

Configuration:

{
  "Capability": "recommendation",
  "Provider": "internal",
  "Model": "product-recommender-v2",
  "Endpoint": "https://ml.internal.bank.com/v2/recommend",
  "ApiKey": "YOUR_INTERNAL_KEY",
  "TimeoutMs": 10000,
  "Parameters": {
    "topK": "10"
  }
}

Parameter	Type	Default	Description
`topK`	string (int)	`"10"`	Number of top recommendations to return.

Input Mapping:

{ "userId": "${context.customerId}" }

Input Field	Required	Description
`userId`	✅ Yes	Customer/user identifier.
`context`	❌ No	Additional context object (current products, preferences, etc.).

Response (result): Raw provider response (typically a ranked list of products/actions).

forecasting

What it does: Predict future values from historical time-series data. Uses trained ML models to project trends.

Real-life banking use cases:

Cash demand forecasting: Predict ATM/branch cash needs for the next week
Transaction volume prediction: Forecast daily transaction counts for capacity planning
Revenue forecasting: Project monthly revenue based on historical trends
Loan default prediction: Forecast portfolio default rates

Configuration:

{
  "Capability": "forecasting",
  "Provider": "azure-ai",
  "Model": "azure-ml-forecast-v1",
  "Endpoint": "https://your-resource.azureml.ms/score",
  "ApiKey": "YOUR_AZURE_KEY",
  "TimeoutMs": 30000,
  "Parameters": {
    "defaultPeriods": "7",
    "confidenceInterval": "0.95"
  }
}

Parameter	Type	Default	Description
`defaultPeriods`	string (int)	`"7"`	Default number of periods to forecast (used if not specified in input).
`confidenceInterval`	string (double)	`"0.95"`	Confidence interval for forecast bounds.

Input Mapping:

{
  "historicalData": "${context.historicalValues}",
  "periods": "${context.forecastPeriods}"
}

Input Field	Required	Description
`historicalData`	✅ Yes	Array of historical data points (time series).
`periods`	❌ No	Number of periods to forecast. Falls back to config `defaultPeriods`.

Response (result): Raw provider response with forecasted values and confidence bounds.

clustering

What it does: Group data points into clusters based on similarity. Uses unsupervised learning algorithms.

Real-life banking use cases:

Customer segmentation: Group customers by behavior, demographics, or transaction patterns
Transaction pattern analysis: Cluster transaction types to identify spending categories
Risk grouping: Segment accounts by risk profile for differentiated monitoring
Branch catchment analysis: Cluster customers by geographic and behavioral similarity

Configuration:

{
  "Capability": "clustering",
  "Provider": "internal",
  "Model": "customer-segmentation-v1",
  "Endpoint": "https://ml.internal.bank.com/v1/cluster",
  "ApiKey": "YOUR_INTERNAL_KEY",
  "TimeoutMs": 15000,
  "Parameters": {
    "defaultClusters": "5",
    "algorithm": "kmeans"
  }
}

Parameter	Type	Default	Description
`defaultClusters`	string (int)	`"5"`	Default number of clusters (used if not specified in input).
`algorithm`	string	`"kmeans"`	Clustering algorithm (e.g., `"kmeans"`, `"dbscan"`, `"hierarchical"`).

Input Mapping:

{
  "data": "${context.customerData}",
  "numClusters": "${context.numberOfSegments}"
}

Input Field	Required	Description
`data`	✅ Yes	Array of data points (each point is an object or feature vector).
`numClusters`	❌ No	Number of clusters. Falls back to config `defaultClusters`.

Response (result): Raw provider response with cluster assignments and centroids.

regression

What it does: Predict numeric values from input features. Uses supervised ML models trained on historical data.

Real-life banking use cases:

Credit score prediction: Estimate credit score from customer features (income, employment, history)
Loan default probability: Predict the likelihood of loan default
Property valuation: Estimate property value for mortgage underwriting
Customer lifetime value: Predict long-term value of a customer relationship

Configuration:

{
  "Capability": "regression",
  "Provider": "internal",
  "Model": "credit-score-predictor-v2",
  "Endpoint": "https://ml.internal.bank.com/v2/predict",
  "ApiKey": "YOUR_INTERNAL_KEY",
  "TimeoutMs": 10000,
  "Parameters": {}
}

Input Mapping:

{
  "features": {
    "income": "${context.customerIncome}",
    "age": "${context.customerAge}",
    "employmentYears": "${context.yearsEmployed}"
  }
}

Input Field	Required	Description
`features`	✅ Yes	Object containing the feature values for prediction.

Response (result): Raw provider response. The handler also extracts confidence into AIModelResult.Confidence if available.

{
  "predictedValue": 720,
  "confidence": 0.89,
  "featureImportance": {
    "income": 0.35,
    "employmentYears": 0.28,
    "age": 0.15,
    "existingDebt": 0.22
  }
}

Code

code-generation

What it does: Generate code from natural language descriptions using an LLM with a coding-focused system prompt.

Real-life banking use cases:

Generate BPMN script task snippets from natural language descriptions
Auto-generate SQL queries from business requirements
Create API integration code from specification documents

Configuration:

{
  "Capability": "code-generation",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "systemPrompt": "You are an expert programmer. Generate clean, well-documented code.",
    "temperature": "0.2",
    "maxTokens": "4096"
  }
}

Parameter	Type	Default	Description
`systemPrompt`	string	`"You are an expert programmer..."`	System prompt for code generation behavior.
`temperature`	string (double)	`"0.2"`	Lower = more precise code.
`maxTokens`	string (int)	`"4096"`	Max tokens for generated code.

Input Mapping:

{
  "prompt": "${context.codePrompt}",
  "language": "${context.programmingLanguage}"
}

Input Field	Required	Description
`prompt`	✅ Yes	Natural language description of what to generate.
`language`	❌ No	Target programming language (e.g., `"csharp"`, `"javascript"`, `"sql"`).

Response (result):

{
  "code": "public class TransactionValidator\n{\n    public bool Validate(decimal amount)\n    {\n        return amount > 0 && amount <= 10000000;\n    }\n}",
  "language": "csharp"
}

Metadata: completionTokens, promptTokens, totalTokens

code-completion

What it does: Complete partial code snippets. Provide existing code and the model suggests the next logical continuation.

Real-life banking use cases:

Complete partially written script task code in the BPMN designer
Suggest completions for custom validation expressions

Configuration:

{
  "Capability": "code-completion",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 15000,
  "Parameters": {
    "temperature": "0.1",
    "maxTokens": "1024"
  }
}

Parameter	Type	Default	Description
`temperature`	string (double)	`"0.1"`	Very low for predictable completions.
`maxTokens`	string (int)	`"1024"`	Max tokens for the completion.

Input Mapping:

{
  "code": "${context.partialCode}",
  "language": "${context.programmingLanguage}"
}

Input Field	Required	Description
`code`	✅ Yes	The partial code to complete.
`language`	❌ No	Programming language of the code.

Response (result):

{
  "completion": "    if (transaction.Amount > threshold)\n    {\n        await FlagForReview(transaction);\n        return false;\n    }\n    return true;\n}"
}

Metadata: completionTokens, promptTokens, totalTokens

code-review

What it does: Analyze code for quality issues, patterns, and improvement suggestions. Returns structured feedback with issue descriptions.

Real-life banking use cases:

Review custom script task code before deployment
Analyze integration scripts for security issues
Quality-check generated code before use

Configuration:

{
  "Capability": "code-review",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "temperature": "0.2",
    "responseFormat": "json_object"
  }
}

Parameter	Type	Default	Description
`systemPrompt`	string	Built-in code review prompt	Customizable review criteria.
`temperature`	string (double)	`"0.2"`	Low for consistent reviews.
`responseFormat`	string	`"json_object"`	Forces JSON output with issues array.

Input Mapping:

{
  "code": "${context.codeToReview}",
  "language": "${context.programmingLanguage}"
}

Input Field	Required	Description
`code`	✅ Yes	The code to review.
`language`	❌ No	Programming language.

Response (result): Parsed JSON from the model:

{
  "issues": [
    {
      "severity": "high",
      "line": 15,
      "description": "SQL injection vulnerability — use parameterized queries",
      "suggestion": "Replace string concatenation with SqlParameter"
    },
    {
      "severity": "medium",
      "line": 8,
      "description": "Missing null check on customer object",
      "suggestion": "Add null guard: if (customer == null) throw new ArgumentNullException()"
    }
  ],
  "overallScore": 6.5
}

bug-detection

What it does: Analyze code specifically for bugs and security vulnerabilities. Returns structured bug reports.

Real-life banking use cases:

Scan custom scripts for potential bugs before production deployment
Detect security vulnerabilities in integration code
Validate user-written expressions in BPMN process definitions

Configuration:

{
  "Capability": "bug-detection",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "temperature": "0.1",
    "responseFormat": "json_object"
  }
}

Parameter	Type	Default	Description
`temperature`	string (double)	`"0.1"`	Very low for thorough analysis.
`responseFormat`	string	`"json_object"`	Forces structured JSON bug report.

Input Mapping:

{
  "code": "${context.codeToAnalyze}",
  "language": "${context.programmingLanguage}"
}

Input Field	Required	Description
`code`	✅ Yes	The code to analyze.
`language`	❌ No	Programming language.

Response (result): Parsed JSON:

{
  "bugs": [
    {
      "severity": "critical",
      "line": 23,
      "type": "race-condition",
      "description": "Shared variable modified without synchronization in multi-threaded context",
      "fix": "Use lock statement or ConcurrentDictionary"
    }
  ],
  "bugsFound": 1
}

Advanced NLP

embeddings

What it does: Convert text into high-dimensional vector representations (embeddings). These vectors capture semantic meaning and can be used for similarity matching, clustering, and search.

Real-life banking use cases:

Duplicate detection: Embed customer complaints to find similar/duplicate cases
Knowledge base search: Embed queries and documents for semantic matching
Customer profiling: Convert transaction narrations to embeddings for behavioral analysis
Recommendation input: Generate embeddings as input features for recommendation models

Configuration:

{
  "Capability": "embeddings",
  "Provider": "openai",
  "Model": "text-embedding-3-small",
  "Endpoint": "https://api.openai.com/v1/embeddings",
  "ApiKey": "sk-...",
  "TimeoutMs": 10000,
  "Parameters": {
    "dimensions": "1536"
  }
}

Parameter	Type	Default	Description
`dimensions`	string (int)	—	Desired embedding dimensions (depends on model).

Input Mapping:

{ "text": "${context.textToEmbed}" }

Input Field	Required	Description
`text`	✅ Yes	The text to generate an embedding vector for.

Response (result):

{
  "embeddings": [0.0023, -0.0045, 0.0312, ...],
  "dimensions": 1536
}

semantic-search

What it does: Search for semantically similar content in an index. Unlike keyword search, this finds results based on meaning.

Real-life banking use cases:

Knowledge base Q&A: Search internal policies and procedures by meaning
Case matching: Find similar past fraud cases for investigation reference
Product discovery: Find relevant products based on natural language queries
Regulatory search: Search compliance documents by concept rather than exact terms

Configuration:

{
  "Capability": "semantic-search",
  "Provider": "custom-http",
  "Model": "search-index-v2",
  "Endpoint": "https://search.internal.bank.com/v2/query",
  "ApiKey": "YOUR_KEY",
  "TimeoutMs": 10000,
  "Parameters": {
    "topK": "10",
    "minScore": "0.7"
  }
}

Parameter	Type	Default	Description
`topK`	string (int)	`"10"`	Number of top results to return.
`minScore`	string (double)	`"0.7"`	Minimum similarity score to include.

Input Mapping:

{ "query": "${context.searchQuery}" }

Input Field	Required	Description
`query`	✅ Yes	The search query.

Response (result): Raw provider response (typically a list of documents with similarity scores).

moderation

What it does: Check text content for inappropriate, harmful, or policy-violating material. Returns flagged categories and scores.

Real-life banking use cases:

Message screening: Screen customer messages before displaying them to staff
Content compliance: Ensure marketing content meets policy standards
Chat moderation: Filter abusive language in customer chat channels
UGC review: Review user-generated content (reviews, feedback) before publishing

Configuration:

{
  "Capability": "moderation",
  "Provider": "openai",
  "Model": "text-moderation-latest",
  "Endpoint": "https://api.openai.com/v1/moderations",
  "ApiKey": "sk-...",
  "TimeoutMs": 10000,
  "Parameters": {}
}

Input Mapping:

{ "text": "${context.contentToModerate}" }

Input Field	Required	Description
`text`	✅ Yes	The text content to check.

Response (result):

{
  "flagged": false,
  "categories": {
    "hate": false,
    "hate/threatening": false,
    "self-harm": false,
    "sexual": false,
    "violence": false
  },
  "scores": {
    "hate": 0.0001,
    "violence": 0.0003,
    "sexual": 0.0002
  }
}

toxicity-detection

What it does: Score text for toxicity and abusive language. Uses Perspective API or similar services.

Real-life banking use cases:

Score customer chat messages for abusive language to trigger escalation
Monitor social media mentions of the bank for toxic content
Screen employee communications for compliance

Configuration:

{
  "Capability": "toxicity-detection",
  "Provider": "custom-http",
  "Model": "perspective-api",
  "Endpoint": "https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze",
  "ApiKey": "YOUR_GOOGLE_KEY",
  "TimeoutMs": 10000,
  "Parameters": {
    "languages": "en"
  }
}

Parameter	Type	Default	Description
`languages`	string	`"en"`	Language code for analysis.

Input Mapping:

{ "text": "${context.textToCheck}" }

Input Field	Required	Description
`text`	✅ Yes	The text to analyze for toxicity.

Response (result): Raw provider response with toxicity scores.

zero-shot-classification

What it does: Classify text into custom labels without any training data. You provide the labels at runtime and the model determines which label(s) best match the text. Uses HuggingFace's BART-large-MNLI or similar.

Real-life banking use cases:

Dynamic ticket routing: Classify support tickets into ad-hoc categories without retraining
Transaction categorization: Classify transaction narrations into custom categories
Document intent detection: Determine the purpose of an uploaded document
Dynamic fraud rules: Classify transaction descriptions as "fraud", "legitimate", "suspicious"

Configuration:

{
  "Capability": "zero-shot-classification",
  "Provider": "huggingface",
  "Model": "facebook/bart-large-mnli",
  "Endpoint": "https://api-inference.huggingface.co/models/facebook/bart-large-mnli",
  "ApiKey": "hf_...",
  "TimeoutMs": 15000,
  "Parameters": {}
}

Input Mapping:

{
  "text": "${context.textToClassify}",
  "labels": ["fraud", "legitimate", "suspicious"]
}

Input Field	Required	Description
`text`	✅ Yes	The text to classify.
`labels`	✅ Yes	Array of candidate labels to score.

Response (result): Raw HuggingFace response:

{
  "sequence": "Large wire transfer to unknown overseas account at 2 AM",
  "labels": ["fraud", "suspicious", "legitimate"],
  "scores": [0.78, 0.19, 0.03]
}

few-shot-learning

What it does: Classify or generate based on a few examples provided at runtime. You give the model example input→output pairs and then a query, and it learns the pattern from the examples.

Real-life banking use cases:

Custom entity extraction: Provide a few examples of extracting fields from bank statements, then extract from new statements
Custom classification: Teach the model new categories with 3-5 examples
Pattern matching: Provide example transaction patterns and ask the model to match new transactions

Configuration:

{
  "Capability": "few-shot-learning",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 30000,
  "Parameters": {
    "systemPrompt": "Learn from the examples provided and apply the same pattern.",
    "temperature": "0.1"
  }
}

Parameter	Type	Default	Description
`systemPrompt`	string	`"Learn from the provided examples..."`	Instructions for how to use the examples.
`temperature`	string (double)	`"0.1"`	Low for consistent pattern matching.

Input Mapping:

{
  "examples": [
    { "input": "Transfer to John for rent", "output": "rent-payment" },
    { "input": "Salary from ABC Corp", "output": "salary" },
    { "input": "MTN airtime purchase", "output": "utility" }
  ],
  "query": "${context.transactionNarration}"
}

Input Field	Required	Description
`examples`	✅ Yes	Array of `{ input, output }` example pairs.
`query`	✅ Yes	The new input to classify/process based on the examples.

Response (result): Parsed JSON from the model (attempts to match the output format of examples):

{
  "result": "rent-payment",
  "confidence": 0.92
}

paraphrase-generation

What it does: Rephrase text while preserving the original meaning. Generates multiple alternative wordings.

Real-life banking use cases:

Customer communication: Generate alternative wordings for rejection/approval messages
A/B testing: Create multiple versions of marketing copy
Accessibility: Simplify complex banking terms into plain language
Template variation: Generate multiple notification templates from a single base

Configuration:

{
  "Capability": "paraphrase-generation",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 15000,
  "Parameters": {
    "temperature": "0.8",
    "responseFormat": "json_object"
  }
}

Parameter	Type	Default	Description
`systemPrompt`	string	Built-in paraphrase prompt	Customizable paraphrasing instructions.
`temperature`	string (double)	`"0.8"`	Higher = more varied paraphrases.
`responseFormat`	string	`"json_object"`	Forces structured JSON output.

Input Mapping:

{ "text": "${context.textToParaphrase}" }

Input Field	Required	Description
`text`	✅ Yes	The text to paraphrase.

Response (result): Parsed JSON:

{
  "paraphrases": [
    "Your loan application has been declined due to insufficient documentation.",
    "Unfortunately, we are unable to approve your loan request at this time because the required documents were not provided.",
    "We regret to inform you that your loan application was not successful as the supporting documents were incomplete."
  ]
}

grammar-correction

What it does: Correct grammar and spelling errors in text. Returns the corrected text alongside a list of specific corrections made.

Real-life banking use cases:

Customer-facing communications: Clean up auto-generated or template-based messages before sending
Document quality: Correct grammar in loan applications and correspondence
Chat cleanup: Polish customer-facing chat responses
Report generation: Ensure generated reports are grammatically correct

Configuration:

{
  "Capability": "grammar-correction",
  "Provider": "openai",
  "Model": "gpt-4-turbo",
  "Endpoint": "https://api.openai.com/v1/chat/completions",
  "ApiKey": "sk-...",
  "TimeoutMs": 10000,
  "Parameters": {
    "temperature": "0.1",
    "responseFormat": "json_object"
  }
}

Parameter	Type	Default	Description
`temperature`	string (double)	`"0.1"`	Very low for accurate corrections.
`responseFormat`	string	`"json_object"`	Forces structured JSON output.

Input Mapping:

{ "text": "${context.textToCorrect}" }

Input Field	Required	Description
`text`	✅ Yes	The text to correct.

Response (result): Parsed JSON:

{
  "correctedText": "Dear Customer, your account has been credited with ₦500,000. Please find the attached statement for your records.",
  "corrections": [
    { "original": "youre", "corrected": "your", "type": "grammar" },
    { "original": "recieved", "corrected": "received", "type": "spelling" },
    { "original": "statment", "corrected": "statement", "type": "spelling" }
  ]
}

Custom

custom-model

What it does: A generic pass-through handler for any custom AI/ML endpoint that doesn't fit the predefined capabilities. It sends the input JSON as-is to the configured endpoint and returns the raw response — no input transformation, no output parsing. You control the input shape via aiInputMapping and handle the output shape in downstream tasks.

When to use this:

Internal ML models (credit scoring, fraud detection, risk models)
Third-party APIs with non-standard input/output formats
Any HTTP endpoint that accepts JSON POST and returns JSON
Prototyping new AI capabilities before creating a dedicated handler
Vendor-specific APIs (e.g., a custom NLP service, proprietary scoring engine)

Real-life banking use cases:

Call an internal credit scoring API that returns a custom score format
Integrate with a third-party KYC verification service
Call a proprietary document extraction service
Connect to an internal transaction risk engine

Configuration:

{
  "Capability": "custom-model",
  "Provider": "custom-http",
  "Model": "my-internal-model-v1",
  "Endpoint": "https://ml.internal.bank.com/v1/predict",
  "ApiKey": "YOUR_KEY",
  "TimeoutMs": 30000,
  "Parameters": {
    "customHeaders": "{\"X-Custom-Header\": \"value\"}"
  }
}

Parameter	Type	Default	Description
`customHeaders`	string (JSON)	—	Additional HTTP headers as a JSON object string.

Input Mapping:

The input is sent exactly as mapped — no transformation. You define the shape entirely:

{
  "customerId": "${context.customerId}",
  "transactionAmount": "${context.amount}",
  "channel": "${context.channel}",
  "anyFieldYouNeed": "${context.anyValue}"
}

Input Field	Required	Description
(any)	Depends on your endpoint	You define the fields — they are sent as-is to the endpoint.

Response (result): Raw JSON response from the endpoint, returned as-is:

{
  "score": 0.85,
  "category": "high-risk",
  "details": {
    "reason": "Multiple transactions in short timeframe",
    "flags": ["velocity", "new-device"]
  }
}

If the response contains a confidence or score field at the root level, it is automatically extracted into AIModelResult.Confidence. Similarly, explanation or message is extracted into AIModelResult.Explanation.

Metadata: endpointCalled, rawResponseLength

Providers: custom-http, internal, azure-ai, openai (any — the Provider field controls the auth header style)

BPMN Integration — Quick Start

Step 1: Add a ServiceTask to Your Process

In the BPMN designer, add a ServiceTask and set:

Property	Value
Service Type	`AI Model`
AI Config Key	`AI:fraud-detection-v3`
AI Input Mapping	`{ "data": { "amount": ${context.transactionAmount}, "channel": "${context.channel}" } }`

Step 2: Store Configuration in Tenant Settings

Go to Administration → Tenant Configuration and add:

Field	Value
Key	`AI:fraud-detection-v3`
Value	`{"Capability":"anomaly-detection","Provider":"internal","Model":"fraud-detection-v3","Endpoint":"https://ml.internal.bank.com/v3/fraud/predict","ApiKey":"your-key","TimeoutMs":5000,"Parameters":{"threshold":"0.7"}}`
Is Sensitive	✅ Yes

Step 3: Use the Result in Downstream Tasks

In a subsequent Script Task or Gateway:

// Script Task — check the AI result
var aiResult = context.aiModelResult;

if (aiResult.confidence > 0.8) {
    context.riskLevel = "HIGH";
    context.requiresManualReview = true;
} else if (aiResult.confidence > 0.5) {
    context.riskLevel = "MEDIUM";
    context.requiresManualReview = true;
} else {
    context.riskLevel = "LOW";
    context.requiresManualReview = false;
}

Step 4: Route Based on AI Output (Gateway)

Add an Exclusive Gateway after the script task:

Sequence Flow	Condition
High Risk	`${context.riskLevel == "HIGH"}`
Medium Risk	`${context.riskLevel == "MEDIUM"}`
Low Risk	(default)

Available Providers

Provider	Description	Authentication
`openai`	OpenAI API (GPT-4, DALL-E, Whisper, TTS, Embeddings)	Bearer token (`Authorization: Bearer sk-...`)
`azure-ai`	Azure Cognitive Services (Text Analytics, Face API, OCR, Translator, etc.)	`Ocp-Apim-Subscription-Key` header
`huggingface`	HuggingFace Inference API (classification, NER, zero-shot, etc.)	Bearer token
`anthropic`	Anthropic Claude API	Bearer token
`google-cloud`	Google Cloud AI (Vision, Speech, NLP)	Bearer token
`internal`	Internal ML services (fraud detection, recommendation, etc.)	Bearer token or custom auth
`custom-http`	Any HTTP endpoint that accepts JSON POST	Bearer token

Querying Available Capabilities

To get the full list of available capabilities at runtime (e.g., to populate a dropdown in the BPMN designer):

POST /api/process/command
{
  "CommandName": "GetAIModelCapabilitiesQuery"
}

Response:

{
  "isSuccessful": true,
  "statusCode": "00",
  "message": "36 AI capabilities available",
  "data": [
    {
      "capability": "text-generation",
      "displayName": "Text Generation",
      "category": "Text & NLP",
      "description": "Generate text responses using LLMs...",
      "defaultConfigKey": "AI:text-generation",
      "sampleConfigJson": "{ ... }",
      "expectedInputMapping": "{\"prompt\": \"${context.analysisPrompt}\"}",
      "providers": ["openai", "azure-ai", "anthropic", "huggingface"],
      "isRegistered": true
    }
  ]
}

The isRegistered flag tells you whether a handler is actually deployed and available for that capability.

Troubleshooting

Error	Cause	Fix
`"configKey is required"`	Missing `aiConfigKey` on ServiceTask	Set the `aiConfigKey` property in the BPMN designer.
`"No AI model configuration found for key '...'"`	Config not in Tenant Configuration	Add the JSON config under the specified key in Administration → Tenant Configuration.
`"No handler registered for capability '...'"`	Typo in `Capability` field or handler not deployed	Check the `Capability` value matches one of the 36 supported capabilities exactly.
`"AI model call timed out"` (408)	Endpoint too slow or `TimeoutMs` too low	Increase `TimeoutMs` in the config or check endpoint health.
`"AI endpoint returned 401"`	Wrong or expired API key	Update the `ApiKey` in Tenant Configuration.
`"AI endpoint returned 429"`	Rate limit exceeded	Add retry logic or upgrade the AI provider plan.
`"text is required"` / `"prompt is required"`	Missing required field in input mapping	Check `aiInputMapping` in the ServiceTask — ensure all required fields are mapped.

How It Works — Architecture Overview​

Step-by-Step Flow​

Configuration Structure (AIModelConfig)​

Where to Store​

Standardized Response Format (AIModelResult)​

Accessing Results in Downstream Tasks​

Capability Categories​

Text & NLP​

text-generation​

sentiment-analysis​

text-classification​

named-entity-recognition​

text-summarization​

question-answering​

Vision​

image-classification​

object-detection​

face-detection​

optical-character-recognition​

pose-estimation​

video-classification​

image-segmentation​

Audio & Speech​

speech-to-text​

text-to-speech​

audio-classification​

music-generation​

translation​

Creative​

image-generation​

Machine Learning​

anomaly-detection​

recommendation​

forecasting​

clustering​

regression​

Code​

code-generation​

code-completion​

code-review​

bug-detection​

Advanced NLP​

embeddings​

semantic-search​

moderation​

toxicity-detection​

zero-shot-classification​

few-shot-learning​

paraphrase-generation​

grammar-correction​

Custom​

custom-model​

BPMN Integration — Quick Start​

Step 1: Add a ServiceTask to Your Process​

Step 2: Store Configuration in Tenant Settings​

Step 3: Use the Result in Downstream Tasks​

Step 4: Route Based on AI Output (Gateway)​

Available Providers​

Querying Available Capabilities​

Troubleshooting​

How It Works — Architecture Overview

Step-by-Step Flow

Configuration Structure (AIModelConfig)

Where to Store

Standardized Response Format (AIModelResult)

Accessing Results in Downstream Tasks

Capability Categories

Text & NLP

text-generation

sentiment-analysis

text-classification

named-entity-recognition

text-summarization

question-answering

Vision

image-classification

object-detection

face-detection

optical-character-recognition

pose-estimation

video-classification

image-segmentation

Audio & Speech

speech-to-text

text-to-speech

audio-classification

music-generation

translation

Creative

image-generation

Machine Learning

anomaly-detection

recommendation

forecasting

clustering

regression

Code

code-generation

code-completion

code-review

bug-detection

Advanced NLP

embeddings

semantic-search

moderation

toxicity-detection

zero-shot-classification

few-shot-learning

paraphrase-generation

grammar-correction

Custom

custom-model

BPMN Integration — Quick Start

Step 1: Add a ServiceTask to Your Process

Step 2: Store Configuration in Tenant Settings

Step 3: Use the Result in Downstream Tasks

Step 4: Route Based on AI Output (Gateway)

Available Providers

Querying Available Capabilities

Troubleshooting