docs: move MailOps webhook handover document to the docs directory
This commit is contained in:
idevakk
162
docs/laravel_webhook_handover.md
Normal file
162
docs/laravel_webhook_handover.md
Normal file
@@ -0,0 +1,162 @@
|
||||
# MailOps Webhook Handover Document
|
||||
|
||||
This document provides the exact specifications needed to implement the receiving end of the MailOps email synchronization system within the Laravel application.
|
||||
|
||||
## 1. Webhook Endpoint Specification
|
||||
|
||||
The MailOps worker will push new emails to this exact endpoint on your Laravel server:
|
||||
|
||||
* **URL:** `POST https://your-laravel-app.com/api/webhooks/incoming_email`
|
||||
* **Headers:**
|
||||
* `Content-Type: application/json`
|
||||
* `Authorization: Bearer <CONFIGURED_WEBHOOK_SECRET>` (You must configure this secret in both MailOps and Laravel).
|
||||
|
||||
### A. Expected JSON Payload (With Attachments)
|
||||
|
||||
```json
|
||||
{
|
||||
"hash": "a1b2c3d4e5f6g7h8i9j0...",
|
||||
"metadata": {
|
||||
"hash": "a1b2c3d4e5f6g7h8i9j0...",
|
||||
"recipientEmail": "user@example.com",
|
||||
"recipientName": "John Doe",
|
||||
"senderEmail": "alert@service.com",
|
||||
"senderName": "Service Alerts",
|
||||
"domain": "example.com",
|
||||
"subject": "Important Notification",
|
||||
"received_at": "2026-02-26T17:35:00Z",
|
||||
"attachments": [
|
||||
{
|
||||
"filename": "invoice.pdf",
|
||||
"mimeType": "application/pdf",
|
||||
"size": 102400,
|
||||
"s3_path": "mail-attachments/2026/02/26/hash_invoice.pdf"
|
||||
}
|
||||
],
|
||||
"attachmentSize": 102400
|
||||
},
|
||||
"bodyText": "Plain text content...",
|
||||
"bodyHtml": "<html>HTML content...</html>"
|
||||
}
|
||||
```
|
||||
*(Note: `received_at` is in ISO 8601 format ending with `Z` to explicitly denote UTC. `bodyHtml` and `bodyText` are completely separated from the metadata to optimize database payload sizes).*
|
||||
|
||||
### B. Expected JSON Payload (NO Attachments)
|
||||
When an email has no attachments, the `attachments` array will be empty and `attachmentSize` will be zero. Also, depending on the email client, `bodyHtml` or `bodyText` might be `null`.
|
||||
|
||||
```json
|
||||
{
|
||||
"hash": "b2c3d4e5f6g7h8i9j0a1...",
|
||||
"metadata": {
|
||||
"hash": "b2c3d4e5f6g7h8i9j0a1...",
|
||||
"recipientEmail": "user@example.com",
|
||||
"recipientName": "",
|
||||
"senderEmail": "friend@service.com",
|
||||
"senderName": "Friend",
|
||||
"domain": "example.com",
|
||||
"subject": "Quick Question",
|
||||
"received_at": "2026-02-26T17:38:12Z",
|
||||
"attachments": [],
|
||||
"attachmentSize": 0
|
||||
},
|
||||
"bodyText": "Hey, are we still fast approaching the deadline?",
|
||||
"bodyHtml": null
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Laravel Implementation Checklist
|
||||
|
||||
When you switch to the Laravel project, you need to build the following:
|
||||
|
||||
### Step 1: Route & Middleware
|
||||
Define the API route and protect it with a simple Bearer token check.
|
||||
```php
|
||||
// routes/api.php
|
||||
Route::post('/webhooks/incoming_email', [EmailWebhookController::class, 'handle'])
|
||||
->middleware('verify.webhook.secret');
|
||||
```
|
||||
|
||||
### Step 2: The Controller
|
||||
The controller persists the metadata to MariaDB and the heavy body to MongoDB. **Crucially**, it also checks if the MongoDB TTL index exists, and if not, automatically creates it using the value defined in your Laravel `.env` file (e.g., `EMAIL_BODY_TTL_SECONDS=259200`).
|
||||
|
||||
```php
|
||||
// app/Http/Controllers/EmailWebhookController.php
|
||||
use Illuminate\Support\Carbon;
|
||||
use Illuminate\Support\Facades\DB;
|
||||
use Illuminate\Support\Facades\Cache;
|
||||
|
||||
public function handle(Request $request)
|
||||
{
|
||||
$payload = $request->all();
|
||||
$meta = $payload['metadata'];
|
||||
$hash = $payload['hash'];
|
||||
|
||||
// 1. Auto-Setup MongoDB TTL Index (Executes only once via Cache)
|
||||
$this->ensureMongoTtlIndexExists();
|
||||
|
||||
// 2. MariaDB: Save Metadata
|
||||
Email::updateOrCreate(
|
||||
['unique_id_hash' => $hash],
|
||||
[
|
||||
'recipient_email' => $meta['recipientEmail'],
|
||||
'sender_email' => $meta['senderEmail'],
|
||||
'subject' => $meta['subject'] ?? '',
|
||||
'is_read' => false,
|
||||
// Parse the ISO 8601 UTC timestamp format explicitly for SQL
|
||||
'received_at' => Carbon::parse($meta['received_at'])->setTimezone('UTC')->toDateTimeString(),
|
||||
// Store attachments JSON. If empty, ensure it's saved as an empty array '[]'
|
||||
'attachments' => !empty($meta['attachments']) ? json_encode($meta['attachments']) : '[]',
|
||||
'attachment_size' => $meta['attachmentSize'] ?? 0
|
||||
]
|
||||
);
|
||||
|
||||
// 3. MongoDB: Save the heavy body with TTL
|
||||
// Assuming you have the jenssegers/mongodb package installed
|
||||
RecentEmailBody::updateOrCreate(
|
||||
['unique_id_hash' => $hash],
|
||||
[
|
||||
// Handle cases where the sender only sends Text or only HTML
|
||||
'body_text' => $payload['bodyText'] ?? '',
|
||||
'body_html' => $payload['bodyHtml'] ?? '',
|
||||
'created_at' => new \MongoDB\BSON\UTCDateTime(now()->timestamp * 1000), // BSON required for TTL
|
||||
]
|
||||
);
|
||||
|
||||
return response()->json(['status' => 'success'], 200);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensures the TTL index is created on the MongoDB collection.
|
||||
* Uses Laravel Cache to avoid checking the database on every single webhook.
|
||||
*/
|
||||
private function ensureMongoTtlIndexExists()
|
||||
{
|
||||
Cache::rememberForever('mongo_ttl_index_created', function () {
|
||||
// Fetch TTL from Laravel .env (Default: 72 hours / 259200 seconds)
|
||||
$ttlSeconds = (int) env('EMAIL_BODY_TTL_SECONDS', 259200);
|
||||
|
||||
$collection = DB::connection('mongodb')->getCollection('recent_email_bodies');
|
||||
|
||||
// Background creation prevents locking the database during webhook execution
|
||||
$collection->createIndex(
|
||||
['created_at' => 1],
|
||||
[
|
||||
'expireAfterSeconds' => $ttlSeconds,
|
||||
'background' => true,
|
||||
'name' => 'ttl_created_at_index' // Named index prevents duplicate recreation errors
|
||||
]
|
||||
);
|
||||
|
||||
return true;
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Resiliency Notes
|
||||
|
||||
* **Idempotency:** The MailOps worker might retry a webhook if a network timeout occurs even after Laravel successfully saved it. Your Laravel code MUST use `updateOrCreate` or `INSERT IGNORE` (like the example above) so it doesn't create duplicate emails if the same payload hash is received twice.
|
||||
* **Timeouts:** The MailOps worker expects a response within 5 to 10 seconds. Do not perform long-running synchronous tasks (like connecting to external APIs or sending heavy push notifications) inside the webhook controller. Dispatch those to a Laravel Queue instead.
|
||||
Reference in New Issue
Block a user