Workflow Automation

A comprehensive automation skill for designing, implementing, and managing automated workflows that connect multiple services, handle data transformations, and orchestrate complex business processes.

When to Use

Choose Workflow Automation when:

• Connecting multiple SaaS tools and APIs into automated data pipelines
• Building approval workflows, notification chains, and multi-step processes
• Automating repetitive business tasks like data entry, reporting, and file management
• Creating event-driven integrations between internal and external systems

Consider alternatives when:

• Building a single API integration — use direct HTTP calls
• Processing large data volumes — use ETL tools like dbt or Apache Airflow
• Real-time event streaming — use Kafka or Redis Streams

Quick Start

Copy
// n8n-style workflow definition
interface WorkflowStep {
  id: string;
  type: 'trigger' | 'action' | 'condition' | 'transform';
  service: string;
  operation: string;
  config: Record<string, any>;
  next?: string[];
  onError?: string;
}

class WorkflowEngine {
  private steps: Map<string, WorkflowStep> = new Map();
  private context: Record<string, any> = {};

  addStep(step: WorkflowStep) {
    this.steps.set(step.id, step);
  }

  async execute(triggerData: any) {
    this.context = { trigger: triggerData, results: {} };

    const trigger = Array.from(this.steps.values())
      .find(s => s.type === 'trigger');
    if (!trigger) throw new Error('No trigger defined');

    await this.runStep(trigger);
    return this.context.results;
  }

  private async runStep(step: WorkflowStep) {
    try {
      const result = await this.executeStep(step);
      this.context.results[step.id] = result;

      if (step.type === 'condition') {
        const branch = result ? step.next?.[0] : step.next?.[1];
        if (branch) await this.runStep(this.steps.get(branch)!);
      } else if (step.next) {
        for (const nextId of step.next) {
          await this.runStep(this.steps.get(nextId)!);
        }
      }
    } catch (error) {
      if (step.onError) {
        await this.runStep(this.steps.get(step.onError)!);
      } else {
        throw error;
      }
    }
  }

  private async executeStep(step: WorkflowStep): Promise<any> {
    const handlers: Record<string, Function> = {
      'http:request': async (config: any) => {
        const resp = await fetch(config.url, {
          method: config.method || 'GET',
          headers: config.headers,
          body: config.body ? JSON.stringify(config.body) : undefined
        });
        return resp.json();
      },
      'transform:map': (config: any) => {
        const input = this.resolveRef(config.input);
        return Array.isArray(input) ? input.map(config.mapper) : config.mapper(input);
      },
      'email:send': async (config: any) => {
        return await sendEmail(config.to, config.subject, config.body);
      }
    };

    const handler = handlers[`${step.service}:${step.operation}`];
    return handler ? handler(this.resolveConfig(step.config)) : null;
  }

  private resolveConfig(config: any): any {
    if (typeof config === 'string' && config.startsWith('{{')) {
      return this.resolveRef(config);
    }
    if (typeof config === 'object') {
      const resolved: any = {};
      for (const [key, value] of Object.entries(config)) {
        resolved[key] = this.resolveConfig(value);
      }
      return resolved;
    }
    return config;
  }

  private resolveRef(ref: string): any {
    const path = ref.replace(/\{\{|\}\}/g, '').trim().split('.');
    let value: any = this.context;
    for (const key of path) value = value?.[key];
    return value;
  }
}

Core Concepts

Workflow Building Blocks

Block	Purpose	Example
Trigger	Start workflow on event	Webhook, schedule, form submit
Action	Perform an operation	Send email, create record, API call
Condition	Branch based on data	If amount > $100, route to approval
Transform	Modify data between steps	Map fields, filter records, aggregate
Loop	Iterate over a collection	Process each order item
Delay	Wait before continuing	Wait 24 hours, wait for event
Error Handler	Handle step failures	Retry, notify, fallback

Common Automation Patterns

Copy
// Pattern: Approval Workflow
const approvalWorkflow = [
  { id: 'trigger', type: 'trigger', service: 'form', operation: 'submit' },
  { id: 'check-amount', type: 'condition',
    config: { expression: '{{trigger.amount}} > 5000' },
    next: ['manager-approval', 'auto-approve'] },
  { id: 'manager-approval', type: 'action', service: 'slack',
    operation: 'send', config: {
      channel: '#approvals',
      message: 'Approval needed: ${{trigger.amount}} from {{trigger.requester}}'
    }},
  { id: 'auto-approve', type: 'action', service: 'database',
    operation: 'update', config: { status: 'approved' }}
];

// Pattern: Data Sync Pipeline
const syncPipeline = [
  { id: 'trigger', type: 'trigger', service: 'cron', operation: 'schedule',
    config: { cron: '*/15 * * * *' }},  // Every 15 min
  { id: 'fetch-source', type: 'action', service: 'http',
    operation: 'request', config: { url: 'https://api.source.com/data' }},
  { id: 'transform', type: 'transform', service: 'transform',
    operation: 'map', config: { mapper: (item) => ({
      externalId: item.id,
      name: item.full_name,
      email: item.contact_email
    })}},
  { id: 'upsert', type: 'action', service: 'database',
    operation: 'upsert', config: { table: 'contacts', key: 'externalId' }}
];

Configuration

Option	Description	Default
max_steps	Maximum steps per workflow execution	100
timeout	Overall workflow timeout (seconds)	300
retry_policy	Default retry policy for steps	{ attempts: 3, backoff: "exponential" }
error_handling	Global error behavior: stop, continue, notify	"stop"
logging_level	Execution logging detail	"info"
concurrent_executions	Max parallel workflow runs	10
data_retention	How long to keep execution logs	"30 days"
webhook_secret	Secret for webhook trigger verification	Generated

Best Practices

1. Make every step idempotent so workflows can be safely retried from any point without causing duplicate actions — use idempotency keys for API calls and check-before-write for database operations
2. Add error handlers at critical steps rather than relying on global error handling — a payment step should have specific error handling (refund, notify) while a logging step can silently fail
3. Use data transformation steps between service calls to normalize data formats rather than building complex transformation logic into action steps — this keeps each step focused and testable
4. Implement dead letter queues for workflows that fail after all retries so failed executions are captured for manual review rather than silently lost
5. Version your workflow definitions so you can safely update automation logic while running workflows complete with the version they started on, preventing mid-execution breaking changes

Common Issues

Workflows failing silently on third-party API errors: External services return errors that workflows do not handle, causing data inconsistencies. Add explicit error handling after every external API call, validate response status codes, and send alerts for unexpected response formats.

Data mapping errors between services: Different services use different field names, data types, and date formats. Build a transformation layer between each pair of services that handles field mapping, type conversion, and null/missing value defaults.

Circular workflow triggers: A workflow that updates a record can trigger another workflow watching that record, creating infinite loops. Add cycle detection by including a workflow execution ID in each action and checking for it in triggers, or use debouncing to prevent rapid re-triggering.