📖 Campaign Documentation

app/api/campaign/service.py contains ~1 800 lines of business logic that underpin every campaign operation.
Below is a comprehensive tour of all functions, grouped by responsibility.

1. Validation & Variable Extraction

find_and_validate_variables()

• Scans a content payload for {{variable}} placeholders using regex patterns.
• Handles URL variable extraction: For media URLs, only scans the path portion to avoid domain/protocol issues.
• Verifies that each variable is either a built-in CampaignVariableType or an existing custom variable previously stored on an active CampaignLead.
• Applies whitelisting rules: marketing templates cannot use certain order/product variables (product_name, product_price, order_id, etc.).
• Two-pass processing:
• Pass 1: Find and validate all unique variables, assign numeric IDs
• Pass 2: Replace variable names with numbered tokens ({{1}}, {{2}})
• Returns:
• variables_map – {"1": "lead_name", "2": "merchant_name", …}
• numbered_content_context – the original JSON with placeholders replaced by numbered tokens as required by Twilio Content API

map_variables()

Runtime variable resolver that converts a variable name into its concrete value:

Supported Variables: - merchant_name, merchant_phone – from Merchant table - merchant_user_phone – from MerchantUser table
- lead_name, lead_phone – from ConversationUser (with fallback logic for names) - product_name – from Product (supports both product_id and product_sku_id) - product_price – from ProductSKU with currency formatting (whole numbers vs decimals) - product_quantity – from ProductSKU inventory - order_id, order_state, order_shipping_fee – from Order with currency formatting - order_delivery_address – from Address with fallback to coordinates

Error Handling: Raises ValueError for missing required IDs or invalid variable names.

2. Template CRUD Operations

create_message_template()

End-to-end campaign creation pipeline:

1. Schema Validation: Validates content_context against appropriate Pydantic schema
2. Media Requirements Check: Ensures required files are provided for media content types
3. File Type Validation: Checks against extensive whitelist of Twilio-supported formats
4. Database Setup: Creates initial Campaign record to get ID for media naming
5. Media Upload: Uploads to Supabase with standardized naming (media_campaign_id_{id}.{ext})
6. Variable Processing: Calls find_and_validate_variables() to extract and number variables
7. Twilio Integration: Creates Content Template via TwilioClient.create_message_template()
8. Status Setting: Sets status = unsubmitted for background job pickup
9. Database Finalization: Updates campaign record with all processed data

Supported Content Types: twilio/text, twilio/media, twilio/card, twilio/call-to-action, twilio/quick-reply

update_message_template()

Versioned campaign update system:

1. Ownership Verification: Ensures merchant owns the campaign and it's the latest version
2. Change Detection: Compares new content against existing campaign
3. Version Creation: Creates new Campaign record (preserves history)
4. Media Handling: Re-uploads media if content type changes or new file provided
5. Variable Re-extraction: Processes variables with existing custom variables context
6. Twilio Update: Creates new Content Template with updated content
7. Linking: Updates old campaign's next_campaign_id to point to new version
8. Archive Logic: Archives rejected campaigns when creating new versions

Key Behavior: Never modifies existing campaigns, always creates new versions with linked history.

3. Background Job Integration

Template Approval Monitoring

The campaign module relies on SAQ background jobs to monitor Twilio approval status:

• update_message_template_status (every 2 minutes) – Polls non-approved campaigns
• update_message_template_status_approved (minutes 5, 35) – Monitors approved campaigns for changes

Status Validation Before Sending

if campaign.status not in [CampaignStatus.approved]:
    logger.warning(
        f"Attempting to send campaign {campaign_id} with status {campaign.status}. 
         Twilio will likely reject."
    )

Critical Flow: Campaigns must reach approved status before successful delivery. The background jobs continuously sync with Twilio's approval system to update local database status.

4. Message Sending Operations

send_message_template()

Low-level single-recipient message sender:

1. Phone Number Validation: Ensures recipient starts with '+' country code
2. Self-Send Prevention: Blocks sending to registered merchant phone numbers
3. Campaign Status Check: Warns about non-approved campaigns
4. Content SID Validation: Ensures Twilio template exists
5. User/Conversation Creation: Creates ConversationUser and Conversation if needed
6. Variable Hydration:
7. Merges provided context with map_variables() output
8. Prioritizes context values over automatic mapping
9. Message Record Creation:
10. Creates assistant message (internal record)
11. Creates response message (actual WhatsApp payload)
12. Twilio Delivery: Calls TwilioClient.send_message() with hydrated variables
13. Status Tracking: Updates message with Twilio Message SID or error status
14. Conversation Update: Sets last_message_sent_at timestamp

send_campaign_messages()

Bulk campaign orchestration system:

1. Campaign Validation: Verifies campaign exists and type
2. Lead Resolution:
3. Uses provided conversation_user_ids OR all active campaign leads
4. Creates new leads for phone numbers not in system
5. Duplicate Prevention:
6. Marketing campaigns: Checks CampaignLeadMessage history, skips successful sends
7. Service campaigns: Allows multiple sends to same lead
8. Batch Processing:
9. Caps at MAX_LEADS = 1000 to prevent mass-spam
10. Creates placeholder CampaignLeadMessage for redirect URLs
11. Variable Context Building:
12. Merges global context with lead-specific custom_variables
13. Injects campaign_lead_message_id, campaign_lead_id, campaign_id
14. Individual Sending: Calls send_message_template() for each lead
15. Error Handling: Continues processing on individual failures, logs errors

5. Default Template Management

create_default_templates()

System template bootstrapping:

1. Plan Detection: Determines basic vs commerce templates based on BillingPlanName
2. Selective Processing: Optionally filters by provided campaign_names list
3. Existence Check: Looks for latest version (no next_campaign_id) of each template
4. Update Logic:
5. Missing: Creates new template via create_message_template()
6. Rejected: Archives old version, creates new one
7. Up-to-date: Skips processing
8. Error Aggregation: Collects creation/update errors without stopping batch
9. Results Reporting: Returns counts of created/updated/skipped/failed templates

Template Sources: - DEFAULT_CAMPAIGNS (6 basic templates) - DEFAULT_COMMERCE_CAMPAIGNS (adds 18 commerce-specific templates)

6. Specialized Service Helpers

send_service_campaign_messages()

Generic service template sender:

1. Template Resolution: Finds latest approved service campaign by name
2. Lead Validation: Ensures conversation_user_ids provided
3. Context Merging: Combines provided context with lead variables
4. Delegation: Calls send_campaign_messages() with resolved campaign

send_merchant_user_support_request_campaign_messages()

Support escalation notification system:

1. Campaign Lookup: Finds merchant_user_override_request template
2. Merchant User Discovery: Gets all merchant users with phone numbers
3. Context Building: Adds customer phone number to template variables
4. Bulk Notification: Sends to all merchant users simultaneously

send_merchant_user_order_placed_campaign_messages()

Order notification system:

1. Order Loading: Loads order with conversation and merchant relationships
2. Template Resolution: Finds merchant_user_order_placed service template
3. Recipient Discovery: Gets merchant user phone numbers
4. ConversationUser Mapping: Maps merchant phones to conversation user IDs
5. Context Injection: Adds order_id to template variables
6. Notification Delivery: Sends to all merchant staff

7. Error Handling & Logging Strategies

Validation Errors

• Schema Validation: Pydantic validation errors → ValueError with detailed messages
• Variable Validation: Unknown variables → ValueError with suggestion text
• File Validation: Unsupported formats → ValueError with format requirements

External Service Failures

• Twilio Errors: Logged with full payload, stored in Message.other_data
• Supabase Errors: File upload failures → Exception with context
• Database Errors: Transaction failures logged and re-raised

Business Logic Errors

• Ownership Violations: Campaign access → HTTPException 403/404
• Status Violations: Send attempts on unapproved campaigns → warnings + attempt
• Lead Management: Invalid lead operations → HTTPException 400

Logging Patterns

logger.error(f"Failed to send Twilio message for campaign {campaign_id} to {recipient}: {e}")
logger.warning(f"Attempting to send campaign {campaign_id} with status {campaign.status}")
logger.info(f"Creating default template: {campaign_name} for merchant {merchant_id}")

8. Integration Points

Database Interactions

• Campaign CRUD: Direct SQLAlchemy operations with relationship loading
• Lead Management: Bulk operations with soft-delete patterns
• Message Tracking: Links campaign sends to conversation messages

External Service Integration

• Twilio Content API: Template creation/updates, status polling
• Twilio Messaging API: Message delivery with template variables
• Supabase Storage: Media file upload/download with merchant isolation

Cross-Module Dependencies

• Conversation Module: Creates conversations and messages
• Merchant Module: Validates ownership, gets user lists
• Order/Product Modules: Provides dynamic variable data
• SAQ Module: Background job scheduling and execution

9. Messaging Capacity & WhatsApp Rate-Limiting

get_customers_messaged_last_24h()

• Pure-SQL query that counts distinct ConversationUser.id values that received a campaign message in the rolling 24-hour window.
• Mirrors WhatsApp Business Platform definition of a customer messaged ‑ only rows with a non-NULL message_id are counted.

get_available_messaging_capacity()

• Reads Merchant.whatsapp_messaging_limit (persisted from Twilio Sender API status tasks).
• Applies a 10 % safety buffer to avoid fence-posting issues around the rolling window.
• Returns: max(0, limit – safety – customers_used)

These two helpers are called at the very top of send_campaign_messages():

available_capacity = await get_available_messaging_capacity(merchant_id)
if available_capacity <= 0:
    # capacity exhausted – schedule retry job and exit early

Behaviour by Campaign Type

10. Retry Strategy & Safety Net

Immediate Retry Job

• When either capacity exhausted or Twilio error 63018 (WhatsAppRateLimitError) is encountered the loop breaks early and the worker enqueues a follow-up job:

  from time import time
  await queue.enqueue(
      "send_campaign_messages_job",
      campaign_id=campaign_id,
      scheduled=int(time()) + 3600,  # 🕐 1 hour later
  )

• Uses SAQ’s scheduled parameter (epoch seconds). There is no delay arg in SAQ – this was fixed in code.

Backup Cron Job – resume_pending_campaigns_job

• Runs hourly at minute :15 (see app/saq/saq.py).
• Scans for any CampaignLeadMessage rows where message_id IS NULL and queues work just like the primary retry.
• Guarantees forward-progress even if individual jobs crash or workers go offline.

Duplicate-Send Protection

• The canonical source-of-truth is the CampaignLeadMessage row:
• Placeholder inserted before send (message_id NULL).
• On success, message_id is filled – future queries exclude the lead.
• On 63018 rate-limit, placeholder left intact (still NULL) so retries will pick it up.
• Marketing campaigns with the conversation_user_ids override currently bypass the successful-send check. A TODO comment is present; until patched, callers must avoid passing a user that's already received the campaign.

11. Campaign State Management

Service Campaign Enable/Disable Logic

Service campaigns have special behavior controlled by the enabled field:

send_service_campaign_messages()

• Pre-check: Before sending any service campaign messages, the function verifies campaign.enabled IS NOT NULL
• Early exit: If campaign is disabled (enabled = NULL), logs info message and returns without sending
• Purpose: Allows merchants to pause service automation (order confirmations, stock alerts, etc.) without deleting campaigns

Enable/Disable Workflow

# Enable a campaign (sets timestamp)
await DBClient.db_update_rows(Campaign, [campaign_id], {"enabled": datetime.now()})

# Disable a campaign (sets to NULL) 
await DBClient.db_update_rows(Campaign, [campaign_id], {"enabled": None})

Integration with Background Jobs

• resume_pending_campaigns_job: Updated to only process enabled marketing campaigns
• Service automation: All service campaign triggers check enabled status before proceeding
• UI indicators: Frontend shows green circle for enabled campaigns

Campaign Type Behavior Differences

State Transition Examples

# Marketing campaign workflow
campaign = create_message_template(name="promo", campaign_type="marketing")
# → campaign.enabled = None (disabled)

await enable_campaign(campaign.id)  
# → campaign.enabled = datetime.now() (enabled)

await send_campaign_messages(campaign.id)
# → Proceeds with sending (enabled check passes)

# Service campaign workflow  
service_campaign = get_service_campaign("customer_order_confirmation")
# → service_campaign.enabled = None (disabled by default)

await enable_campaign(service_campaign.id)
# → service_campaign.enabled = datetime.now() (enabled)

# Later, when order is placed...
await send_service_campaign_messages(
    merchant_id=merchant.id,
    campaign_name="customer_order_confirmation", 
    conversation_user_ids=[customer.id]
)
# → Checks enabled status, proceeds if enabled

Where to Look Next

• Routes – how this service layer is exposed over HTTP.
• Models – data structures manipulated here.
• Schemas – validation constraints enforced at input time.