bmc_hub/docs/TICKET_EMAIL_INTEGRATION.md

Ticket System - Email Integration

Overview

The ticket system integrates with BMC Hub's email workflow engine to automatically create tickets from incoming emails. Uses Option B: Message-ID Threading for robust email-to-ticket linkage.

Architecture

Incoming Email → Email Classifier → Email Workflow Engine → Ticket Integration
                                    ↓
                            create_ticket action
                                    ↓
                    EmailTicketIntegration.process_email_for_ticket()
                                    ↓
                            ┌───────┴───────┐
                            ↓               ↓
                    New Ticket          Reply to Existing
                    (creates TKT-XXX)   (links via Message-ID)

Email Threading Logic

Message-ID Based Threading (Option B)

The system uses email headers to detect if an email is part of an existing ticket thread:

1. In-Reply-To Header: Check if contains TKT-YYYYMMDD-XXX pattern
2. References Header: Check all message IDs in chain for ticket number
3. Subject Line: Check for [TKT-YYYYMMDD-XXX] or Re: TKT-YYYYMMDD-XXX

If ticket number found → Link to existing ticket
If NOT found → Create new ticket

Ticket Number Format

Pattern: TKT-YYYYMMDD-XXX

• TKT: Prefix
• YYYYMMDD: Date (e.g., 20251215)
• XXX: Sequential number (001-999)

Example: TKT-20251215-001

Workflow Actions

1. create_ticket

Creates new ticket OR links to existing ticket (smart routing).

Workflow Definition:

{
  "name": "Support Request → Ticket",
  "classification_trigger": "support_request",
  "workflow_steps": [
    {
      "action": "create_ticket",
      "params": {
        "customer_id": 123,
        "assigned_to_user_id": 5
      }
    }
  ]
}

Parameters:

• customer_id (optional): BMC customer ID to link ticket to
• assigned_to_user_id (optional): User ID to assign ticket to
• priority (optional): Override priority detection (low/normal/high/critical)

Returns:

{
  "ticket_id": 42,
  "ticket_number": "TKT-20251215-001",
  "created": true,
  "linked": false
}

2. link_email_to_ticket

Explicitly link email to a specific ticket (manual routing).

Workflow Definition:

{
  "action": "link_email_to_ticket",
  "params": {
    "ticket_number": "TKT-20251215-001"
  }
}

Parameters:

• ticket_number (required): Target ticket number

Returns:

{
  "ticket_id": 42,
  "ticket_number": "TKT-20251215-001",
  "linked": true
}

Automatic Features

Tag Extraction

Extracts #hashtags from email body and adds as ticket tags:

Email body: "We need help with #billing #urgent issue"
→ Ticket tags: ["billing", "urgent"]

• Max 10 tags per ticket
• Minimum 3 characters
• Converted to lowercase

Priority Detection

Automatically determines ticket priority from email content:

Critical: kritisk, critical, down, nede, urgent, akut
High: høj, high, vigtig, important, haster
Low: lav, low, spørgsmål, question, info
Normal: Everything else (default)

Checks both subject line and extracted tags.

Email Metadata

Stores rich metadata in ticket:

• Description: Formatted with sender email, received date, original body
• Custom Fields: email_from, email_message_id, created_from_email
• Email Log: Full threading data in tticket_email_log table

Database Storage

tticket_email_log Table

Stores all email-ticket linkages for audit trail:

CREATE TABLE tticket_email_log (
    id SERIAL PRIMARY KEY,
    ticket_id INTEGER NOT NULL REFERENCES tticket_tickets(id),
    email_message_id TEXT NOT NULL,
    email_subject TEXT,
    email_from TEXT NOT NULL,
    email_received_at TIMESTAMP,
    is_reply BOOLEAN DEFAULT FALSE,
    thread_data JSONB,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

thread_data JSONB structure:

{
  "message_id": "<abc123@example.com>",
  "in_reply_to": "<TKT-20251215-001@bmcnetworks.dk>",
  "references": "<ref1@example.com> <ref2@example.com>"
}

Usage Examples

Example 1: Basic Support Request

Incoming Email:

From: customer@example.com
Subject: Help with network issue
Body: Our internet is down. Can you help?

Workflow Configuration:

{
  "name": "Support Email → Ticket",
  "classification_trigger": "support_request",
  "confidence_threshold": 0.7,
  "workflow_steps": [
    {
      "action": "create_ticket",
      "params": {
        "assigned_to_user_id": 5
      }
    }
  ]
}

Result:

• Creates ticket TKT-20251215-001
• Priority: normal (no urgent keywords)
• Tags: [] (no hashtags found)
• Assigned to user 5
• Email logged in tticket_email_log

Example 2: Email Reply to Ticket

Incoming Email:

From: customer@example.com
Subject: Re: TKT-20251215-001
In-Reply-To: <TKT-20251215-001@bmcnetworks.dk>
Body: Thanks, issue is resolved now

Result:

• Detects existing ticket via In-Reply-To header
• Adds comment to ticket TKT-20251215-001
• Does NOT create new ticket
• Logs as reply (is_reply=true)

Example 3: Tagged Urgent Request

Incoming Email:

From: vip@example.com
Subject: URGENT: Server down!
Body: Production server is down #critical #server

Result:

• Creates ticket with priority critical (subject keyword)
• Tags: ["critical", "server"]
• Custom field: created_from_email=true

API Endpoints

Get Ticket Email Thread

GET /api/v1/tickets/{ticket_id}/emails

Returns chronological list of all emails linked to ticket.

Response:

{
  "ticket_id": 42,
  "ticket_number": "TKT-20251215-001",
  "emails": [
    {
      "id": 1,
      "email_message_id": "<abc123@example.com>",
      "email_from": "customer@example.com",
      "email_subject": "Help with issue",
      "email_received_at": "2025-12-15T10:00:00Z",
      "is_reply": false,
      "created_at": "2025-12-15T10:01:00Z"
    },
    {
      "id": 2,
      "email_message_id": "<def456@example.com>",
      "email_from": "customer@example.com",
      "email_subject": "Re: TKT-20251215-001",
      "email_received_at": "2025-12-15T11:00:00Z",
      "is_reply": true,
      "created_at": "2025-12-15T11:01:00Z"
    }
  ]
}

Find Tickets by Email Address

GET /api/v1/tickets/by-email/{email_address}

Returns all tickets associated with an email address.

Response:

{
  "email_address": "customer@example.com",
  "tickets": [
    {
      "id": 42,
      "ticket_number": "TKT-20251215-001",
      "subject": "Network issue",
      "status": "open",
      "created_at": "2025-12-15T10:00:00Z"
    }
  ]
}

Configuration

Settings (.env)

# Ticket system enabled
TICKET_ENABLED=true

# Email integration enabled (requires email system)
TICKET_EMAIL_INTEGRATION=true

# Auto-assign new tickets (requires user_id)
TICKET_AUTO_ASSIGN=false
TICKET_DEFAULT_ASSIGNEE_ID=5

# Default priority for new tickets
TICKET_DEFAULT_PRIORITY=normal

Testing

Test Email-to-Ticket Creation

curl -X POST http://localhost:8001/api/v1/test/email-to-ticket \
  -H "Content-Type: application/json" \
  -d '{
    "email_data": {
      "message_id": "<test123@example.com>",
      "subject": "Test ticket",
      "from_address": "test@example.com",
      "body": "This is a test #testing",
      "received_at": "2025-12-15T10:00:00Z"
    }
  }'

Test Email Reply Linking

curl -X POST http://localhost:8001/api/v1/test/email-to-ticket \
  -H "Content-Type: application/json" \
  -d '{
    "email_data": {
      "message_id": "<reply456@example.com>",
      "subject": "Re: TKT-20251215-001",
      "from_address": "test@example.com",
      "body": "Reply to existing ticket",
      "received_at": "2025-12-15T11:00:00Z",
      "in_reply_to": "<TKT-20251215-001@bmcnetworks.dk>"
    }
  }'

Troubleshooting

Tickets Not Created

Check:

1. Email workflow engine enabled (EMAIL_WORKFLOWS_ENABLED=true)
2. Workflow exists for classification trigger
3. Confidence threshold met
4. Workflow action is create_ticket (NOT old create_ticket stub)

Debug:

-- Check workflow executions
SELECT * FROM email_workflow_executions 
WHERE status = 'failed' 
ORDER BY created_at DESC 
LIMIT 10;

Email Not Linked to Ticket

Check:

1. Ticket number format correct (TKT-YYYYMMDD-XXX)
2. Ticket exists in database
3. Email headers contain ticket number (In-Reply-To, References, Subject)

Debug:

-- Check email logs
SELECT * FROM tticket_email_log 
WHERE ticket_id = 42 
ORDER BY created_at DESC;

Duplicate Tickets Created

Check:

1. Email reply headers missing ticket number
2. Subject line doesn't match pattern (e.g., Re: Ticket 123 instead of Re: TKT-20251215-001)

Solution:

• Ensure outgoing ticket emails include ticket number in subject: [TKT-20251215-001]
• Add ticket number to Message-ID: <TKT-20251215-001-reply@bmcnetworks.dk>

Best Practices

1. Include Ticket Number in Replies: Always include [TKT-YYYYMMDD-XXX] in subject line
2. Use Message-ID with Ticket Number: Format: <TKT-YYYYMMDD-XXX@bmcnetworks.dk>
3. Set Customer ID in Workflow: Improves ticket organization and reporting
4. Monitor Workflow Executions: Check email_workflow_executions table regularly
5. Review Failed Actions: Alert on repeated workflow failures

Security Considerations

1. No Email Body Storage: Only stores metadata and body in ticket description
2. Sender Validation: Consider implementing sender verification (SPF/DKIM)
3. Spam Prevention: Email classifier should filter spam before workflow execution
4. Customer Isolation: Ensure customer_id properly set to prevent data leakage

Future Enhancements

• Attachment Handling: Link email attachments to ticket attachments
• Email Templates: Auto-reply with ticket number
• SLA Integration: Start SLA timer on ticket creation from email
• Multi-Ticket Threading: Support one email creating multiple tickets
• Smart Customer Detection: Auto-detect customer from sender domain

Related Documentation

• Ticket System Architecture
• Email Workflow System
• Database Schema