bmc_hub/docs/TICKET_EMAIL_INTEGRATION.md

# Ticket System - Email Integration

## Overview

The ticket system integrates with BMC Hub's email workflow engine to automatically create tickets from incoming emails. Uses **Option B: Message-ID Threading** for robust email-to-ticket linkage.

## Architecture

```
Incoming Email → Email Classifier → Email Workflow Engine → Ticket Integration
                                    ↓
                            create_ticket action
                                    ↓
                    EmailTicketIntegration.process_email_for_ticket()
                                    ↓
                            ┌───────┴───────┐
                            ↓               ↓
                    New Ticket          Reply to Existing
                    (creates TKT-XXX)   (links via Message-ID)
```

## Email Threading Logic

### Message-ID Based Threading (Option B)

The system uses email headers to detect if an email is part of an existing ticket thread:

1. **In-Reply-To Header**: Check if contains `TKT-YYYYMMDD-XXX` pattern
2. **References Header**: Check all message IDs in chain for ticket number
3. **Subject Line**: Check for `[TKT-YYYYMMDD-XXX]` or `Re: TKT-YYYYMMDD-XXX`

If ticket number found → **Link to existing ticket**  
If NOT found → **Create new ticket**

### Ticket Number Format

Pattern: `TKT-YYYYMMDD-XXX`
- `TKT`: Prefix
- `YYYYMMDD`: Date (e.g., 20251215)
- `XXX`: Sequential number (001-999)

Example: `TKT-20251215-001`

## Workflow Actions

### 1. `create_ticket`

Creates new ticket OR links to existing ticket (smart routing).

**Workflow Definition:**
```json
{
  "name": "Support Request → Ticket",
  "classification_trigger": "support_request",
  "workflow_steps": [
    {
      "action": "create_ticket",
      "params": {
        "customer_id": 123,
        "assigned_to_user_id": 5
      }
    }
  ]
}
```

**Parameters:**
- `customer_id` (optional): BMC customer ID to link ticket to
- `assigned_to_user_id` (optional): User ID to assign ticket to
- `priority` (optional): Override priority detection (low/normal/high/critical)

**Returns:**
```json
{
  "ticket_id": 42,
  "ticket_number": "TKT-20251215-001",
  "created": true,
  "linked": false
}
```

### 2. `link_email_to_ticket`

Explicitly link email to a specific ticket (manual routing).

**Workflow Definition:**
```json
{
  "action": "link_email_to_ticket",
  "params": {
    "ticket_number": "TKT-20251215-001"
  }
}
```

**Parameters:**
- `ticket_number` (required): Target ticket number

**Returns:**
```json
{
  "ticket_id": 42,
  "ticket_number": "TKT-20251215-001",
  "linked": true
}
```

## Automatic Features

### Tag Extraction

Extracts `#hashtags` from email body and adds as ticket tags:

```
Email body: "We need help with #billing #urgent issue"
→ Ticket tags: ["billing", "urgent"]
```

- Max 10 tags per ticket
- Minimum 3 characters
- Converted to lowercase

### Priority Detection

Automatically determines ticket priority from email content:

**Critical**: kritisk, critical, down, nede, urgent, akut  
**High**: høj, high, vigtig, important, haster  
**Low**: lav, low, spørgsmål, question, info  
**Normal**: Everything else (default)

Checks both subject line and extracted tags.

### Email Metadata

Stores rich metadata in ticket:

- **Description**: Formatted with sender email, received date, original body
- **Custom Fields**: `email_from`, `email_message_id`, `created_from_email`
- **Email Log**: Full threading data in `tticket_email_log` table

## Database Storage

### `tticket_email_log` Table

Stores all email-ticket linkages for audit trail:

```sql
CREATE TABLE tticket_email_log (
    id SERIAL PRIMARY KEY,
    ticket_id INTEGER NOT NULL REFERENCES tticket_tickets(id),
    email_message_id TEXT NOT NULL,
    email_subject TEXT,
    email_from TEXT NOT NULL,
    email_received_at TIMESTAMP,
    is_reply BOOLEAN DEFAULT FALSE,
    thread_data JSONB,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```

**`thread_data` JSONB structure:**
```json
{
  "message_id": "<abc123@example.com>",
  "in_reply_to": "<TKT-20251215-001@bmcnetworks.dk>",
  "references": "<ref1@example.com> <ref2@example.com>"
}
```

## Usage Examples

### Example 1: Basic Support Request

**Incoming Email:**
```
From: customer@example.com
Subject: Help with network issue
Body: Our internet is down. Can you help?
```

**Workflow Configuration:**
```json
{
  "name": "Support Email → Ticket",
  "classification_trigger": "support_request",
  "confidence_threshold": 0.7,
  "workflow_steps": [
    {
      "action": "create_ticket",
      "params": {
        "assigned_to_user_id": 5
      }
    }
  ]
}
```

**Result:**
- Creates ticket `TKT-20251215-001`
- Priority: `normal` (no urgent keywords)
- Tags: [] (no hashtags found)
- Assigned to user 5
- Email logged in `tticket_email_log`

### Example 2: Email Reply to Ticket

**Incoming Email:**
```
From: customer@example.com
Subject: Re: TKT-20251215-001
In-Reply-To: <TKT-20251215-001@bmcnetworks.dk>
Body: Thanks, issue is resolved now
```

**Result:**
- **Detects existing ticket** via In-Reply-To header
- **Adds comment** to ticket `TKT-20251215-001`
- Does NOT create new ticket
- Logs as reply (`is_reply=true`)

### Example 3: Tagged Urgent Request

**Incoming Email:**
```
From: vip@example.com
Subject: URGENT: Server down!
Body: Production server is down #critical #server
```

**Result:**
- Creates ticket with priority `critical` (subject keyword)
- Tags: `["critical", "server"]`
- Custom field: `created_from_email=true`

## API Endpoints

### Get Ticket Email Thread

```http
GET /api/v1/tickets/{ticket_id}/emails
```

Returns chronological list of all emails linked to ticket.

**Response:**
```json
{
  "ticket_id": 42,
  "ticket_number": "TKT-20251215-001",
  "emails": [
    {
      "id": 1,
      "email_message_id": "<abc123@example.com>",
      "email_from": "customer@example.com",
      "email_subject": "Help with issue",
      "email_received_at": "2025-12-15T10:00:00Z",
      "is_reply": false,
      "created_at": "2025-12-15T10:01:00Z"
    },
    {
      "id": 2,
      "email_message_id": "<def456@example.com>",
      "email_from": "customer@example.com",
      "email_subject": "Re: TKT-20251215-001",
      "email_received_at": "2025-12-15T11:00:00Z",
      "is_reply": true,
      "created_at": "2025-12-15T11:01:00Z"
    }
  ]
}
```

### Find Tickets by Email Address

```http
GET /api/v1/tickets/by-email/{email_address}
```

Returns all tickets associated with an email address.

**Response:**
```json
{
  "email_address": "customer@example.com",
  "tickets": [
    {
      "id": 42,
      "ticket_number": "TKT-20251215-001",
      "subject": "Network issue",
      "status": "open",
      "created_at": "2025-12-15T10:00:00Z"
    }
  ]
}
```

## Configuration

### Settings (`.env`)

```bash
# Ticket system enabled
TICKET_ENABLED=true

# Email integration enabled (requires email system)
TICKET_EMAIL_INTEGRATION=true

# Auto-assign new tickets (requires user_id)
TICKET_AUTO_ASSIGN=false
TICKET_DEFAULT_ASSIGNEE_ID=5

# Default priority for new tickets
TICKET_DEFAULT_PRIORITY=normal
```

## Testing

### Test Email-to-Ticket Creation

```bash
curl -X POST http://localhost:8001/api/v1/test/email-to-ticket \
  -H "Content-Type: application/json" \
  -d '{
    "email_data": {
      "message_id": "<test123@example.com>",
      "subject": "Test ticket",
      "from_address": "test@example.com",
      "body": "This is a test #testing",
      "received_at": "2025-12-15T10:00:00Z"
    }
  }'
```

### Test Email Reply Linking

```bash
curl -X POST http://localhost:8001/api/v1/test/email-to-ticket \
  -H "Content-Type: application/json" \
  -d '{
    "email_data": {
      "message_id": "<reply456@example.com>",
      "subject": "Re: TKT-20251215-001",
      "from_address": "test@example.com",
      "body": "Reply to existing ticket",
      "received_at": "2025-12-15T11:00:00Z",
      "in_reply_to": "<TKT-20251215-001@bmcnetworks.dk>"
    }
  }'
```

## Troubleshooting

### Tickets Not Created

**Check:**
1. Email workflow engine enabled (`EMAIL_WORKFLOWS_ENABLED=true`)
2. Workflow exists for classification trigger
3. Confidence threshold met
4. Workflow action is `create_ticket` (NOT old `create_ticket` stub)

**Debug:**
```sql
-- Check workflow executions
SELECT * FROM email_workflow_executions 
WHERE status = 'failed' 
ORDER BY created_at DESC 
LIMIT 10;
```

### Email Not Linked to Ticket

**Check:**
1. Ticket number format correct (`TKT-YYYYMMDD-XXX`)
2. Ticket exists in database
3. Email headers contain ticket number (In-Reply-To, References, Subject)

**Debug:**
```sql
-- Check email logs
SELECT * FROM tticket_email_log 
WHERE ticket_id = 42 
ORDER BY created_at DESC;
```

### Duplicate Tickets Created

**Check:**
1. Email reply headers missing ticket number
2. Subject line doesn't match pattern (e.g., `Re: Ticket 123` instead of `Re: TKT-20251215-001`)

**Solution:**
- Ensure outgoing ticket emails include ticket number in subject: `[TKT-20251215-001]`
- Add ticket number to Message-ID: `<TKT-20251215-001-reply@bmcnetworks.dk>`

## Best Practices

1. **Include Ticket Number in Replies**: Always include `[TKT-YYYYMMDD-XXX]` in subject line
2. **Use Message-ID with Ticket Number**: Format: `<TKT-YYYYMMDD-XXX@bmcnetworks.dk>`
3. **Set Customer ID in Workflow**: Improves ticket organization and reporting
4. **Monitor Workflow Executions**: Check `email_workflow_executions` table regularly
5. **Review Failed Actions**: Alert on repeated workflow failures

## Security Considerations

1. **No Email Body Storage**: Only stores metadata and body in ticket description
2. **Sender Validation**: Consider implementing sender verification (SPF/DKIM)
3. **Spam Prevention**: Email classifier should filter spam before workflow execution
4. **Customer Isolation**: Ensure `customer_id` properly set to prevent data leakage

## Future Enhancements

- **Attachment Handling**: Link email attachments to ticket attachments
- **Email Templates**: Auto-reply with ticket number
- **SLA Integration**: Start SLA timer on ticket creation from email
- **Multi-Ticket Threading**: Support one email creating multiple tickets
- **Smart Customer Detection**: Auto-detect customer from sender domain

## Related Documentation

- [Ticket System Architecture](TICKET_SYSTEM_ARCHITECTURE.md)
- [Email Workflow System](EMAIL_WORKFLOWS.md)
- [Database Schema](../migrations/025_ticket_module.sql)