Collection Processing Patterns

Advanced patterns for processing collections using multi-instance subprocess with result aggregation, filtering, transformation, and error handling.

Overview

Collection processing enables:

✅ Map operations - Transform each item
✅ Filter operations - Select items based on criteria
✅ Reduce operations - Aggregate results
✅ Partition operations - Split into groups
✅ Validation operations - Check all items

Pattern 1: Map-Reduce

Transform collection and aggregate results.

Example: Calculate Total Loan Values

<bpmn:scriptTask id="GetLoans" name="Get Loans">
  <bpmn:script>
    var loans = BankLingo.ExecuteCommand('GetActiveLoans', {
      status: 'ACTIVE'
    });
    
    context.loans = loans.result;
    context.loanCount = loans.result.length;
    
    logger.info('Processing ' + context.loanCount + ' active loans');
    
    return {
      loanCount: context.loanCount
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- MAP: Calculate value for each loan -->
<bpmn:callActivity id="CalculateLoanValues" 
                   name="Calculate Loan Values"
                   calledElement="CalculateSingleLoanValue">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.loans"/>
    <custom:property name="elementVariable" value="currentLoan"/>
    <custom:property name="aggregationVariable" value="loanValues"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Subprocess: Calculate single loan value -->
<bpmn:process id="CalculateSingleLoanValue">
  <bpmn:startEvent id="Start"/>
  
  <bpmn:scriptTask id="CalculateValue" name="Calculate Value">
    <bpmn:script>
      var loan = context.currentLoan;
      
      // Calculate current value
      var principal = loan.principalBalance;
      var interest = loan.accruedInterest;
      var fees = loan.outstandingFees;
      var totalValue = principal + interest + fees;
      
      // Calculate risk rating
      var daysPastDue = loan.daysPastDue || 0;
      var riskRating = 'LOW';
      if (daysPastDue > 90) {
        riskRating = 'HIGH';
      } else if (daysPastDue > 30) {
        riskRating = 'MEDIUM';
      }
      
      logger.info('Loan ' + loan.id + ': $' + totalValue + ', risk: ' + riskRating);
      
      // Return result for aggregation
      return {
        loanId: loan.id,
        customerId: loan.customerId,
        principal: principal,
        interest: interest,
        fees: fees,
        totalValue: totalValue,
        riskRating: riskRating,
        daysPastDue: daysPastDue
      };
    </bpmn:script>
  </bpmn:scriptTask>
  
  <bpmn:endEvent id="End"/>
</bpmn:process>

<!-- REDUCE: Aggregate all loan values -->
<bpmn:scriptTask id="AggregateLoanValues" name="Aggregate Values">
  <bpmn:script>
    var values = context.loanValues;
    
    // Total values
    var totalPrincipal = values.reduce((sum, v) => sum + v.principal, 0);
    var totalInterest = values.reduce((sum, v) => sum + v.interest, 0);
    var totalFees = values.reduce((sum, v) => sum + v.fees, 0);
    var grandTotal = values.reduce((sum, v) => sum + v.totalValue, 0);
    
    // Risk breakdown
    var highRisk = values.filter(v => v.riskRating === 'HIGH');
    var mediumRisk = values.filter(v => v.riskRating === 'MEDIUM');
    var lowRisk = values.filter(v => v.riskRating === 'LOW');
    
    var highRiskValue = highRisk.reduce((sum, v) => sum + v.totalValue, 0);
    var mediumRiskValue = mediumRisk.reduce((sum, v) => sum + v.totalValue, 0);
    var lowRiskValue = lowRisk.reduce((sum, v) => sum + v.totalValue, 0);
    
    context.portfolioSummary = {
      totalLoans: values.length,
      totalPrincipal: totalPrincipal,
      totalInterest: totalInterest,
      totalFees: totalFees,
      grandTotal: grandTotal,
      highRiskCount: highRisk.length,
      highRiskValue: highRiskValue,
      mediumRiskCount: mediumRisk.length,
      mediumRiskValue: mediumRiskValue,
      lowRiskCount: lowRisk.length,
      lowRiskValue: lowRiskValue
    };
    
    logger.info('Portfolio: $' + grandTotal + ' total, ' + 
                highRisk.length + ' high risk ($' + highRiskValue + '), ' +
                mediumRisk.length + ' medium risk ($' + mediumRiskValue + '), ' +
                lowRisk.length + ' low risk ($' + lowRiskValue + ')');
    
    return context.portfolioSummary;
  </bpmn:script>
</bpmn:scriptTask>

Use Cases:

Portfolio valuation
Risk analysis
Financial reporting
Data aggregation

Pattern 2: Filter-Process

Filter collection before processing.

Example: Process High-Value Transactions

<bpmn:scriptTask id="GetAndFilterTransactions" name="Get & Filter">
  <bpmn:script>
    // Get all transactions
    var allTransactions = BankLingo.ExecuteCommand('GetDailyTransactions', {
      date: context.businessDate
    });
    
    // Filter for high-value transactions (>$10,000)
    var highValueTransactions = allTransactions.result.filter(function(txn) {
      return txn.amount > 10000;
    });
    
    context.allTransactionCount = allTransactions.result.length;
    context.highValueTransactions = highValueTransactions;
    context.highValueCount = highValueTransactions.length;
    
    logger.info('Found ' + context.highValueCount + ' high-value transactions out of ' + 
                context.allTransactionCount + ' total');
    
    return {
      totalTransactions: context.allTransactionCount,
      highValueCount: context.highValueCount
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Gateway: Any high-value transactions? -->
<bpmn:exclusiveGateway id="HasHighValue" name="Has High Value?"/>

<bpmn:sequenceFlow sourceRef="HasHighValue" targetRef="ProcessHighValue">
  <bpmn:conditionExpression>${context.highValueCount > 0}</bpmn:conditionExpression>
</bpmn:sequenceFlow>

<!-- Process only high-value transactions -->
<bpmn:callActivity id="ProcessHighValue" 
                   name="Process High-Value Transactions"
                   calledElement="ProcessHighValueTransaction">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.highValueTransactions"/>
    <custom:property name="elementVariable" value="currentTransaction"/>
    <custom:property name="aggregationVariable" value="processingResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Subprocess: Enhanced monitoring for high-value -->
<bpmn:process id="ProcessHighValueTransaction">
  <bpmn:startEvent id="Start"/>
  
  <bpmn:scriptTask id="PerformEnhancedChecks" name="Enhanced Checks">
    <bpmn:script>
      var txn = context.currentTransaction;
      
      logger.info('Enhanced monitoring for transaction ' + txn.id + ': $' + txn.amount);
      
      // AML check
      var amlCheck = BankLingo.ExecuteCommand('PerformAMLCheck', {
        transactionId: txn.id,
        amount: txn.amount,
        customerId: txn.customerId
      });
      
      context.amlPassed = amlCheck.result.passed;
      context.amlRiskScore = amlCheck.result.riskScore;
      
      // Fraud detection
      var fraudCheck = BankLingo.ExecuteCommand('DetectFraud', {
        transactionId: txn.id,
        customerId: txn.customerId,
        amount: txn.amount
      });
      
      context.fraudPassed = fraudCheck.result.passed;
      context.fraudScore = fraudCheck.result.fraudScore;
      
      var passed = context.amlPassed && context.fraudPassed;
      
      return {
        transactionId: txn.id,
        amount: txn.amount,
        passed: passed,
        amlRiskScore: context.amlRiskScore,
        fraudScore: context.fraudScore
      };
    </bpmn:script>
  </bpmn:scriptTask>
  
  <bpmn:endEvent id="End"/>
</bpmn:process>

<!-- Analyze results -->
<bpmn:scriptTask id="AnalyzeResults" name="Analyze Results">
  <bpmn:script>
    var results = context.processingResults;
    
    var passed = results.filter(r => r.passed);
    var flagged = results.filter(r => !r.passed);
    
    var flaggedAmount = flagged.reduce((sum, r) => sum + r.amount, 0);
    
    context.flaggedTransactions = flagged;
    context.flaggedCount = flagged.length;
    context.flaggedAmount = flaggedAmount;
    
    if (flagged.length > 0) {
      logger.warn('Flagged ' + flagged.length + ' transactions totaling $' + flaggedAmount);
    }
    
    return {
      passed: passed.length,
      flagged: flagged.length,
      flaggedAmount: flaggedAmount
    };
  </bpmn:script>
</bpmn:scriptTask>

Use Cases:

AML monitoring
Fraud detection
Exception processing
Priority handling

Pattern 3: Partition and Process

Split collection into groups and process separately.

Example: Process Transactions by Type

<bpmn:scriptTask id="GetAndPartitionTransactions" name="Get & Partition">
  <bpmn:script>
    var transactions = BankLingo.ExecuteCommand('GetTransactions', {
      date: context.businessDate
    });
    
    // Partition by transaction type
    var deposits = [];
    var withdrawals = [];
    var transfers = [];
    
    transactions.result.forEach(function(txn) {
      if (txn.type === 'DEPOSIT') {
        deposits.push(txn);
      } else if (txn.type === 'WITHDRAWAL') {
        withdrawals.push(txn);
      } else if (txn.type === 'TRANSFER') {
        transfers.push(txn);
      }
    });
    
    context.deposits = deposits;
    context.withdrawals = withdrawals;
    context.transfers = transfers;
    
    context.depositCount = deposits.length;
    context.withdrawalCount = withdrawals.length;
    context.transferCount = transfers.length;
    
    logger.info('Partitioned: ' + deposits.length + ' deposits, ' + 
                withdrawals.length + ' withdrawals, ' + 
                transfers.length + ' transfers');
    
    return {
      depositCount: deposits.length,
      withdrawalCount: withdrawals.length,
      transferCount: transfers.length
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Process each partition separately -->

<!-- Deposits -->
<bpmn:callActivity id="ProcessDeposits" 
                   name="Process Deposits"
                   calledElement="ProcessDeposit">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.deposits"/>
    <custom:property name="elementVariable" value="currentDeposit"/>
    <custom:property name="aggregationVariable" value="depositResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Withdrawals -->
<bpmn:callActivity id="ProcessWithdrawals" 
                   name="Process Withdrawals"
                   calledElement="ProcessWithdrawal">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.withdrawals"/>
    <custom:property name="elementVariable" value="currentWithdrawal"/>
    <custom:property name="aggregationVariable" value="withdrawalResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Transfers -->
<bpmn:callActivity id="ProcessTransfers" 
                   name="Process Transfers"
                   calledElement="ProcessTransfer">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.transfers"/>
    <custom:property name="elementVariable" value="currentTransfer"/>
    <custom:property name="aggregationVariable" value="transferResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Aggregate results from all partitions -->
<bpmn:scriptTask id="AggregateAllResults" name="Aggregate Results">
  <bpmn:script>
    var depositResults = context.depositResults || [];
    var withdrawalResults = context.withdrawalResults || [];
    var transferResults = context.transferResults || [];
    
    // Calculate totals
    var depositTotal = depositResults.reduce((sum, r) => sum + r.amount, 0);
    var withdrawalTotal = withdrawalResults.reduce((sum, r) => sum + r.amount, 0);
    var transferTotal = transferResults.reduce((sum, r) => sum + r.amount, 0);
    
    // Calculate success rates
    var depositSuccess = depositResults.filter(r => r.success).length;
    var withdrawalSuccess = withdrawalResults.filter(r => r.success).length;
    var transferSuccess = transferResults.filter(r => r.success).length;
    
    context.summary = {
      deposits: {
        count: depositResults.length,
        total: depositTotal,
        success: depositSuccess
      },
      withdrawals: {
        count: withdrawalResults.length,
        total: withdrawalTotal,
        success: withdrawalSuccess
      },
      transfers: {
        count: transferResults.length,
        total: transferTotal,
        success: transferSuccess
      }
    };
    
    logger.info('Summary: Deposits: ' + depositSuccess + '/' + depositResults.length + 
                ', Withdrawals: ' + withdrawalSuccess + '/' + withdrawalResults.length +
                ', Transfers: ' + transferSuccess + '/' + transferResults.length);
    
    return context.summary;
  </bpmn:script>
</bpmn:scriptTask>

Use Cases:

Type-specific processing
Parallel processing by category
Different workflows per type
Category-based reporting

Pattern 4: Validation and Error Collection

Validate all items and collect errors.

Example: Validate Customer Applications

<bpmn:scriptTask id="GetApplications" name="Get Applications">
  <bpmn:script>
    var applications = BankLingo.ExecuteCommand('GetSubmittedApplications', {
      status: 'SUBMITTED',
      date: context.businessDate
    });
    
    context.applications = applications.result;
    context.applicationCount = applications.result.length;
    
    logger.info('Validating ' + context.applicationCount + ' applications');
    
    return {
      applicationCount: context.applicationCount
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Validate all applications -->
<bpmn:callActivity id="ValidateApplications" 
                   name="Validate Applications"
                   calledElement="ValidateSingleApplication">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.applications"/>
    <custom:property name="elementVariable" value="currentApplication"/>
    <custom:property name="aggregationVariable" value="validationResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Subprocess: Validate single application -->
<bpmn:process id="ValidateSingleApplication">
  <bpmn:startEvent id="Start"/>
  
  <bpmn:scriptTask id="ValidateFields" name="Validate Fields">
    <bpmn:script>
      var app = context.currentApplication;
      var errors = [];
      
      // Required field validation
      if (!app.customerId) {
        errors.push({ field: 'customerId', error: 'Required' });
      }
      if (!app.amount || app.amount <= 0) {
        errors.push({ field: 'amount', error: 'Must be greater than 0' });
      }
      if (!app.purpose) {
        errors.push({ field: 'purpose', error: 'Required' });
      }
      
      // Business rule validation
      if (app.amount > 1000000) {
        errors.push({ field: 'amount', error: 'Exceeds maximum loan amount' });
      }
      
      // Document validation
      var requiredDocs = ['ID', 'PROOF_OF_INCOME', 'BANK_STATEMENT'];
      var submittedDocs = app.documents || [];
      
      requiredDocs.forEach(function(docType) {
        var hasDoc = submittedDocs.some(function(doc) {
          return doc.type === docType;
        });
        if (!hasDoc) {
          errors.push({ field: 'documents', error: 'Missing ' + docType });
        }
      });
      
      var isValid = errors.length === 0;
      
      logger.info('Application ' + app.id + ': ' + 
                  (isValid ? 'Valid' : errors.length + ' errors'));
      
      // Return result
      return {
        applicationId: app.id,
        customerId: app.customerId,
        isValid: isValid,
        errors: errors,
        errorCount: errors.length
      };
    </bpmn:script>
  </bpmn:scriptTask>
  
  <bpmn:endEvent id="End"/>
</bpmn:process>

<!-- Analyze validation results -->
<bpmn:scriptTask id="AnalyzeValidation" name="Analyze Validation">
  <bpmn:script>
    var results = context.validationResults;
    
    // Separate valid and invalid
    var valid = results.filter(r => r.isValid);
    var invalid = results.filter(r => !r.isValid);
    
    // Collect all errors
    var allErrors = [];
    invalid.forEach(function(result) {
      result.errors.forEach(function(error) {
        allErrors.push({
          applicationId: result.applicationId,
          field: error.field,
          error: error.error
        });
      });
    });
    
    // Error frequency analysis
    var errorCounts = {};
    allErrors.forEach(function(err) {
      var key = err.field + ': ' + err.error;
      errorCounts[key] = (errorCounts[key] || 0) + 1;
    });
    
    context.validApplications = valid;
    context.invalidApplications = invalid;
    context.validCount = valid.length;
    context.invalidCount = invalid.length;
    context.allErrors = allErrors;
    context.errorCounts = errorCounts;
    
    logger.info('Validation: ' + valid.length + ' valid, ' + 
                invalid.length + ' invalid, ' + 
                allErrors.length + ' total errors');
    
    // Log most common errors
    var sortedErrors = Object.keys(errorCounts).sort(function(a, b) {
      return errorCounts[b] - errorCounts[a];
    });
    
    logger.info('Most common errors:');
    sortedErrors.slice(0, 5).forEach(function(err) {
      logger.info('  ' + err + ': ' + errorCounts[err] + ' occurrences');
    });
    
    return {
      validCount: valid.length,
      invalidCount: invalid.length,
      errorCount: allErrors.length
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Gateway: Any invalid? -->
<bpmn:exclusiveGateway id="HasInvalid" name="Has Invalid?"/>

<bpmn:sequenceFlow sourceRef="HasInvalid" targetRef="ProcessValid">
  <bpmn:conditionExpression>${context.invalidCount === 0}</bpmn:conditionExpression>
</bpmn:sequenceFlow>

<bpmn:sequenceFlow sourceRef="HasInvalid" targetRef="HandleInvalid">
  <bpmn:conditionExpression>${context.invalidCount > 0}</bpmn:conditionExpression>
</bpmn:sequenceFlow>

<!-- Handle invalid applications -->
<bpmn:scriptTask id="HandleInvalid" name="Handle Invalid">
  <bpmn:script>
    // Notify customers of validation errors
    context.invalidApplications.forEach(function(app) {
      BankLingo.ExecuteCommand('SendValidationErrors', {
        applicationId: app.applicationId,
        customerId: app.customerId,
        errors: app.errors
      });
    });
    
    logger.info('Sent validation error notifications to ' + 
                context.invalidApplications.length + ' customers');
  </bpmn:script>
</bpmn:scriptTask>

<!-- Process valid applications -->
<bpmn:callActivity id="ProcessValid" 
                   name="Process Valid Applications"
                   calledElement="ProcessApplication">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.validApplications"/>
    <custom:property name="elementVariable" value="validApplication"/>
    <custom:property name="aggregationVariable" value="processingResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

Use Cases:

Batch validation
Error reporting
Data quality analysis
Application screening

Pattern 5: Dynamic Batch Processing

Process large datasets in manageable batches.

Example: Process Large Transaction Set

<bpmn:scriptTask id="InitializeBatchProcessing" name="Initialize">
  <bpmn:script>
    // Initialize batch processing
    context.batchSize = 1000;
    context.currentOffset = 0;
    context.totalProcessed = 0;
    context.hasMoreRecords = true;
    
    logger.info('Starting batch processing with batch size ' + context.batchSize);
    
    return {
      batchSize: context.batchSize
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Batch processing loop -->
<bpmn:scriptTask id="GetNextBatch" name="Get Next Batch">
  <bpmn:script>
    logger.info('Fetching batch at offset ' + context.currentOffset);
    
    var batch = BankLingo.ExecuteCommand('GetTransactionBatch', {
      batchSize: context.batchSize,
      offset: context.currentOffset
    });
    
    context.currentBatch = batch.result.transactions;
    context.batchRecordCount = batch.result.transactions.length;
    context.hasMoreRecords = batch.result.hasMore;
    context.totalRecordsAvailable = batch.result.totalCount;
    
    logger.info('Fetched ' + context.batchRecordCount + ' records, ' + 
                'hasMore: ' + context.hasMoreRecords);
    
    return {
      batchCount: context.batchRecordCount,
      hasMore: context.hasMoreRecords
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Process current batch -->
<bpmn:callActivity id="ProcessBatch" 
                   name="Process Batch"
                   calledElement="ProcessTransaction">
  <bpmn:multiInstanceLoopCharacteristics isSequential="false">
    <custom:property name="collection" value="context.currentBatch"/>
    <custom:property name="elementVariable" value="currentTransaction"/>
    <custom:property name="aggregationVariable" value="batchResults"/>
  </bpmn:multiInstanceLoopCharacteristics>
</bpmn:callActivity>

<!-- Update progress -->
<bpmn:scriptTask id="UpdateProgress" name="Update Progress">
  <bpmn:script>
    var results = context.batchResults;
    
    var successful = results.filter(r => r.success).length;
    var failed = results.filter(r => !r.success).length;
    
    // Update cumulative counters
    context.totalProcessed += context.batchRecordCount;
    context.totalSuccessful = (context.totalSuccessful || 0) + successful;
    context.totalFailed = (context.totalFailed || 0) + failed;
    context.currentOffset += context.batchSize;
    
    var progress = context.totalRecordsAvailable > 0 ?
      (context.totalProcessed / context.totalRecordsAvailable * 100).toFixed(2) : 0;
    
    logger.info('Progress: ' + context.totalProcessed + '/' + 
                context.totalRecordsAvailable + ' (' + progress + '%), ' +
                'Success: ' + context.totalSuccessful + ', ' +
                'Failed: ' + context.totalFailed);
    
    return {
      totalProcessed: context.totalProcessed,
      progress: progress,
      hasMore: context.hasMoreRecords
    };
  </bpmn:script>
</bpmn:scriptTask>

<!-- Gateway: More batches? -->
<bpmn:exclusiveGateway id="HasMoreBatches" name="More Batches?"/>

<bpmn:sequenceFlow sourceRef="HasMoreBatches" targetRef="GetNextBatch">
  <bpmn:conditionExpression>${context.hasMoreRecords === true}</bpmn:conditionExpression>
</bpmn:sequenceFlow>

<bpmn:sequenceFlow sourceRef="HasMoreBatches" targetRef="ProcessingComplete">
  <bpmn:conditionExpression>${context.hasMoreRecords === false}</bpmn:conditionExpression>
</bpmn:sequenceFlow>

<!-- Processing complete -->
<bpmn:scriptTask id="ProcessingComplete" name="Complete">
  <bpmn:script>
    logger.info('Batch processing complete. ' + 
                'Total: ' + context.totalProcessed + ', ' +
                'Successful: ' + context.totalSuccessful + ', ' +
                'Failed: ' + context.totalFailed);
    
    var successRate = (context.totalSuccessful / context.totalProcessed * 100).toFixed(2);
    
    context.successRate = successRate;
    
    return {
      totalProcessed: context.totalProcessed,
      successRate: successRate
    };
  </bpmn:script>
</bpmn:scriptTask>

Use Cases:

Large dataset processing
Memory-efficient operations
Progress tracking
Paginated data processing

Performance Tips

Batch Size Optimization

// ✅ Optimal batch sizes based on operation type

// Small batches (10-100) for:
// - Complex processing
// - External API calls with rate limits
// - High memory usage per item
var batchSize = 50;

// Medium batches (100-1000) for:
// - Moderate processing
// - Database operations
// - Balanced workload
var batchSize = 500;

// Large batches (1000-5000) for:
// - Simple transformations
// - In-memory operations
// - Low resource usage per item
var batchSize = 2000;

Parallel Processing Strategy

// ✅ Choose parallel vs sequential based on operation

// Parallel (isSequential: false) for:
// - I/O-bound operations (API calls, DB queries)
// - Independent items (no shared state)
// - When speed is priority
isSequential = false;

// Sequential (isSequential: true) for:
// - CPU-intensive operations (PDF generation)
// - Order-dependent processing
// - Resource-constrained operations
// - When order matters
isSequential = true;

Memory Management

// ✅ Prevent memory issues in large collections

// Clear intermediate results
if (context.currentBatch) {
  delete context.currentBatch;
}

// Keep only summary data
context.summary = {
  count: results.length,
  total: results.reduce((sum, r) => sum + r.value, 0)
};
// Don't keep full results array if not needed

// Process in smaller chunks
if (totalRecords > 10000) {
  batchSize = 1000; // Smaller batches for large datasets
}

Best Practices

✅ Do This

// ✅ Return consistent result structure
return {
  id: item.id,
  success: true,
  result: data,
  processedAt: new Date().toISOString()
};

// ✅ Handle errors gracefully
try {
  // process item
  return { id: item.id, success: true, result: data };
} catch (error) {
  logger.error('Error: ' + error.message);
  return { id: item.id, success: false, error: error.message };
}

// ✅ Use appropriate batch sizes
var batchSize = totalRecords > 10000 ? 1000 : 500;

// ✅ Track progress for long-running processes
logger.info('Progress: ' + processed + '/' + total + ' (' + percent + '%)');

// ✅ Aggregate efficiently
var total = results.reduce((sum, r) => sum + r.value, 0);

❌ Don't Do This

// ❌ Inconsistent return structures
// Some return { success: true }, others return { status: 'ok' }

// ❌ Unhandled errors stop all processing
processItem(item); // No try-catch

// ❌ Very large batches
var batchSize = 50000; // Too large, memory issues

// ❌ No progress tracking
// User has no idea how long it will take

// ❌ Keeping all results in memory
context.allResults.push(result); // For 100,000 items!

Multi-Instance Subprocess - Multi-instance fundamentals
Scheduled Tasks - Batch scheduling
Error Recovery - Error handling patterns
Call Activity - Subprocess execution

Features Used:

Phase 1: Multi-Instance Subprocess

Status: ✅ Production Ready
Version: 2.0
Last Updated: January 2026

Overview​

Pattern 1: Map-Reduce​

Example: Calculate Total Loan Values​

Pattern 2: Filter-Process​

Example: Process High-Value Transactions​

Pattern 3: Partition and Process​

Example: Process Transactions by Type​

Pattern 4: Validation and Error Collection​

Example: Validate Customer Applications​

Pattern 5: Dynamic Batch Processing​

Example: Process Large Transaction Set​

Performance Tips​

Batch Size Optimization​

Parallel Processing Strategy​

Memory Management​

Best Practices​

✅ Do This​

❌ Don't Do This​

Related Documentation​

Overview

Pattern 1: Map-Reduce

Example: Calculate Total Loan Values

Pattern 2: Filter-Process

Example: Process High-Value Transactions

Pattern 3: Partition and Process

Example: Process Transactions by Type

Pattern 4: Validation and Error Collection

Example: Validate Customer Applications

Pattern 5: Dynamic Batch Processing

Example: Process Large Transaction Set

Performance Tips

Batch Size Optimization

Parallel Processing Strategy

Memory Management

Best Practices

✅ Do This

❌ Don't Do This

Related Documentation