bmc_hub/REMINDER_SYSTEM_IMPLEMENTATION.md

# Reminder System Implementation - BMC Hub

## Overview

The Reminder System for BMC Hub's Sag (Case) module provides flexible, multi-channel notification delivery with support for:

- **Time-based reminders**: Scheduled at specific times or recurring (daily, weekly, monthly)
- **Status-change triggered reminders**: Automatically triggered when case status changes
- **Multi-channel delivery**: Mattermost, Email, Frontend popup notifications
- **User preferences**: Global defaults with per-reminder overrides
- **Rate limiting**: Max 5 notifications per user per hour (global)
- **Smart scheduling**: Database triggers for status changes, APScheduler for time-based

## Architecture

### Database Schema

**4 Main Tables** (created in `migrations/096_reminder_system.sql`):

1. **`user_notification_preferences`** - User default notification settings
   - Default channels (mattermost, email, frontend)
   - Quiet hours configuration
   - Email override option

2. **`sag_reminders`** - Reminder rules/templates
   - Trigger configuration (status_change, deadline_approaching, time_based)
   - Recipient configuration (user IDs or email addresses)
   - Recurrence setup (once, daily, weekly, monthly)
   - Scheduling info (scheduled_at, next_check_at)

3. **`sag_reminder_queue`** - Event queue from database triggers
   - Holds events generated by status-change trigger
   - Processing status tracking (pending, processing, sent, failed, rate_limited)
   - Batch processed by scheduler job

4. **`sag_reminder_logs`** - Execution log
   - Every notification sent/failed is logged
   - User interactions (snooze, dismiss, acknowledge)
   - Used for rate limiting verification

**Database Triggers**:
- `sag_status_change_reminder_trigger()` - Fires on status UPDATE, queues relevant reminders

**Helper Functions**:
- `check_reminder_rate_limit(user_id)` - Verifies user hasn't exceeded 5 per hour

**Helper Views**:
- `v_pending_reminders` - Time-based reminders ready to send
- `v_pending_reminder_queue` - Queued events ready for processing

### Backend Services

**`app/services/reminder_notification_service.py`**:
- Unified notification delivery via Mattermost, Email, Frontend
- Merges user preferences with per-reminder overrides
- Rate limit checking
- Event logging
- Email template rendering (Jinja2)

**`app/services/email_service.py`** (extended):
- Added `send_email()` async method using `aiosmtplib`
- SMTP configuration from `.env`
- Supports plain text + HTML bodies
- Safety flag: `REMINDERS_DRY_RUN=true` logs without sending

### API Endpoints

**User Preferences** (in `app/modules/sag/backend/reminders.py`):
```
GET  /api/v1/users/me/notification-preferences
PATCH /api/v1/users/me/notification-preferences
```

**Reminder CRUD**:
```
GET    /api/v1/sag/{sag_id}/reminders           - List reminders for case
POST   /api/v1/sag/{sag_id}/reminders           - Create new reminder
PATCH  /api/v1/sag/reminders/{reminder_id}      - Update reminder
DELETE /api/v1/sag/reminders/{reminder_id}      - Soft-delete reminder
```

**Reminder Interactions**:
```
POST /api/v1/sag/reminders/{reminder_id}/snooze   - Snooze for X minutes
POST /api/v1/sag/reminders/{reminder_id}/dismiss  - Permanently dismiss
GET  /api/v1/reminders/pending/me                 - Get pending (for polling)
```

### Scheduler Job

**`app/jobs/check_reminders.py`**:
- Processes time-based reminders (`next_check_at <= NOW()`)
- Processes queued status-change events
- Calculates next recurrence (`daily` +24h, `weekly` +7d, `monthly` +30d)
- Respects rate limiting

**Registration in `main.py`**:
```python
backup_scheduler.scheduler.add_job(
    func=check_reminders,
    trigger=IntervalTrigger(minutes=5),
    id='check_reminders',
    name='Check Reminders',
    max_instances=1
)
```

Runs **every 5 minutes** (configurable via `REMINDERS_CHECK_INTERVAL_MINUTES`)

### Frontend Notifications

**`static/js/notifications.js`**:
- Bootstrap 5 Toast-based notification popups
- Polls `/api/v1/reminders/pending/me` every 30 seconds
- Snooze presets: 15min, 30min, 1h, 4h, 1day, custom
- Dismiss action
- Auto-removes when hidden
- Pauses polling when tab not visible

**Integration**:
- Loaded in `app/shared/frontend/base.html`
- Auto-initializes on page load if user authenticated
- User ID extracted from JWT token

## Configuration

### Environment Variables

```env
# Master switches (default: disabled for safety)
REMINDERS_ENABLED=false
REMINDERS_EMAIL_ENABLED=false
REMINDERS_MATTERMOST_ENABLED=false
REMINDERS_DRY_RUN=true              # Log without sending if true

# Scheduler settings
REMINDERS_CHECK_INTERVAL_MINUTES=5   # Frequency of reminder checks
REMINDERS_MAX_PER_USER_PER_HOUR=5    # Rate limit
REMINDERS_QUEUE_BATCH_SIZE=10        # Batch size for queue processing

# SMTP Configuration (for email reminders)
EMAIL_SMTP_HOST=smtp.gmail.com
EMAIL_SMTP_PORT=587
EMAIL_SMTP_USER=noreply@bmcnetworks.dk
EMAIL_SMTP_PASSWORD=<secret>
EMAIL_SMTP_USE_TLS=true
EMAIL_SMTP_FROM_ADDRESS=noreply@bmcnetworks.dk
EMAIL_SMTP_FROM_NAME=BMC Hub
```

### Pydantic Configuration

Added to `app/core/config.py`:
```python
REMINDERS_ENABLED: bool = False
REMINDERS_EMAIL_ENABLED: bool = False
REMINDERS_MATTERMOST_ENABLED: bool = False
REMINDERS_DRY_RUN: bool = True
REMINDERS_CHECK_INTERVAL_MINUTES: int = 5
REMINDERS_MAX_PER_USER_PER_HOUR: int = 5
REMINDERS_QUEUE_BATCH_SIZE: int = 10

EMAIL_SMTP_HOST: str = ""
EMAIL_SMTP_PORT: int = 587
EMAIL_SMTP_USER: str = ""
EMAIL_SMTP_PASSWORD: str = ""
EMAIL_SMTP_USE_TLS: bool = True
EMAIL_SMTP_FROM_ADDRESS: str = "noreply@bmcnetworks.dk"
EMAIL_SMTP_FROM_NAME: str = "BMC Hub"
```

## Safety Features

### Rate Limiting
- **Global per user**: Max 5 notifications per hour
- Checked before sending via `check_reminder_rate_limit(user_id)`
- Queued events marked as `rate_limited` if limit exceeded

### Dry Run Mode
- `REMINDERS_DRY_RUN=true` (default)
- All operations logged to console/logs
- No emails actually sent
- No Mattermost webhooks fired
- Useful for testing

### Soft Delete
- Reminders never hard-deleted from DB
- `deleted_at` timestamp + `is_active=false`
- Full audit trail preserved

### Per-Reminder Override
- Reminder can override user's default channels
- `override_user_preferences` flag
- Useful for critical reminders (urgent priority)

## Usage Examples

### Create a Status-Change Reminder

```bash
curl -X POST http://localhost:8000/api/v1/sag/123/reminders \
  -H "Content-Type: application/json" \
  -d {
    "title": "Case entered In Progress",
    "message": "Case #123 has moved to 'i_gang' status",
    "priority": "high",
    "trigger_type": "status_change",
    "trigger_config": {"target_status": "i_gang"},
    "recipient_user_ids": [1, 2],
    "notify_mattermost": true,
    "notify_email": true,
    "recurrence_type": "once"
  }
```

### Create a Scheduled Reminder

```bash
curl -X POST http://localhost:8000/api/v1/sag/123/reminders \
  -H "Content-Type: application/json" \
  -d {
    "title": "Follow up needed",
    "message": "Check in with customer",
    "priority": "normal",
    "trigger_type": "time_based",
    "trigger_config": {},
    "scheduled_at": "2026-02-10T14:30:00",
    "recipient_user_ids": [1],
    "recurrence_type": "once"
  }
```

### Create a Daily Recurring Reminder

```bash
curl -X POST http://localhost:8000/api/v1/sag/123/reminders \
  -H "Content-Type: application/json" \
  -d {
    "title": "Daily status check",
    "priority": "low",
    "trigger_type": "time_based",
    "trigger_config": {},
    "scheduled_at": "2026-02-04T09:00:00",
    "recipient_user_ids": [1],
    "recurrence_type": "daily"
  }
```

### Update User Preferences

```bash
curl -X PATCH http://localhost:8000/api/v1/users/me/notification-preferences \
  -H "Content-Type: application/json" \
  -d {
    "notify_mattermost": true,
    "notify_email": false,
    "notify_frontend": true,
    "quiet_hours_enabled": true,
    "quiet_hours_start": "18:00",
    "quiet_hours_end": "08:00"
  }
```

## Testing Checklist

### Database
- [ ] Run migration: `docker-compose exec -T postgres psql -U bmc_hub -d bmc_hub -f /migrations/096_reminder_system.sql`
- [ ] Verify tables created: `SELECT * FROM sag_reminders LIMIT 0;`
- [ ] Verify trigger exists: `SELECT * FROM information_schema.triggers WHERE trigger_name LIKE 'sag%reminder%';`

### API
- [ ] Test create reminder endpoint
- [ ] Test list reminders endpoint
- [ ] Test update reminder endpoint
- [ ] Test snooze endpoint
- [ ] Test dismiss endpoint
- [ ] Test user preferences endpoints

### Scheduler
- [ ] Enable `REMINDERS_ENABLED=true` in `.env`
- [ ] Restart container
- [ ] Check logs for "Reminder job scheduled" message
- [ ] Verify job runs every 5 minutes: `✅ Checking for pending reminders...`

### Status Change Trigger
- [ ] Create reminder with `trigger_type: status_change`
- [ ] Change case status
- [ ] Verify event inserted in `sag_reminder_queue`
- [ ] Wait for scheduler to process
- [ ] Verify log entry in `sag_reminder_logs`

### Email Sending
- [ ] Configure SMTP in `.env`
- [ ] Set `REMINDERS_EMAIL_ENABLED=true`
- [ ] Set `REMINDERS_DRY_RUN=false`
- [ ] Create reminder with `notify_email=true`
- [ ] Verify email sent or check logs

### Frontend Popup
- [ ] Ensure `static/js/notifications.js` included in base.html
- [ ] Open browser console
- [ ] Log in to system
- [ ] Should see "✅ Reminder system initialized"
- [ ] Create a pending reminder
- [ ] Should see popup toast within 30 seconds
- [ ] Test snooze dropdown
- [ ] Test dismiss button

### Rate Limiting
- [ ] Create 6 reminders for user with trigger_type=time_based
- [ ] Manually trigger scheduler job
- [ ] Verify only 5 sent, 1 marked `rate_limited`
- [ ] Check `sag_reminder_logs` for status

## Deployment Notes

### Local Development
- All safety switches OFF by default (`_ENABLED=false`, `DRY_RUN=true`)
- No SMTP configured - reminders won't send
- No Mattermost webhook - notifications go to logs only
- Test via dry-run mode

### Production Deployment
1. Configure SMTP credentials in `.env`
2. Set `REMINDERS_ENABLED=true`
3. Set `REMINDERS_EMAIL_ENABLED=true` if using email
4. Set `REMINDERS_MATTERMOST_ENABLED=true` if using Mattermost
5. Set `REMINDERS_DRY_RUN=false` to actually send
6. Deploy with `docker-compose -f docker-compose.prod.yml up -d --build`
7. Monitor logs for errors: `docker-compose logs -f api`

## Files Modified/Created

### New Files
- `migrations/096_reminder_system.sql` - Database schema
- `app/services/reminder_notification_service.py` - Notification service
- `app/jobs/check_reminders.py` - Scheduler job
- `app/modules/sag/backend/reminders.py` - API endpoints
- `static/js/notifications.js` - Frontend notification system
- `templates/emails/reminder.html` - Email template

### Modified Files
- `app/core/config.py` - Added reminder settings
- `app/services/email_service.py` - Added `send_email()` method
- `main.py` - Imported reminders router, registered scheduler job
- `app/shared/frontend/base.html` - Added notifications.js script tag
- `requirements.txt` - Added `aiosmtplib` dependency
- `.env` - Added reminder configuration

## Troubleshooting

### Reminders not sending
1. Check `REMINDERS_ENABLED=true` in `.env`
2. Check scheduler logs: "Reminder check complete"
3. Verify `next_check_at` <= NOW() for reminders
4. Check rate limit: count in `sag_reminder_logs` last hour

### Frontend popups not showing
1. Check browser console for errors
2. Verify JWT token contains `sub` (user_id)
3. Check `GET /api/v1/reminders/pending/me` returns data
4. Ensure `static/js/notifications.js` loaded

### Email not sending
1. Verify SMTP credentials in `.env`
2. Check `REMINDERS_EMAIL_ENABLED=true`
3. Check `REMINDERS_DRY_RUN=false`
4. Review application logs for SMTP errors
5. Test SMTP connection separately

### Database trigger not working
1. Verify migration applied successfully
2. Check `sag_status_change_reminder_trigger_exec` trigger exists
3. Update case status manually
4. Check `sag_reminder_queue` for new events
5. Review PostgreSQL logs if needed

## Future Enhancements

- [ ] Escalation rules (auto-escalate if not acknowledged)
- [ ] SMS/WhatsApp integration (Twilio)
- [ ] Calendar integration (iCal export)
- [ ] User notification history/statistics
- [ ] Webhook support for external services
- [ ] AI-powered reminder suggestions
- [ ] Mobile app push notifications