Cascading System

The cascading system automatically propagates status changes along the ITSM chain Problem → Incident → Ticket. When a problem or incident changes status, all linked downstream entities are updated: activity logs, notifications, SLA pause/resume and WebSocket updates — fully asynchronous, without blocking the triggering request.

🔗

Features

✓ Transactional outbox (single DB transaction)

✓ Asynchronous Redis queue (5 parallel, max 20/s)

✓ 2-hop propagation (problem → incident → ticket)

✓ Cascade entries authored by "System"

✓ Ticket SLA follows the incident (resume/pause)

✓ Idempotent (no duplicates on re-runs)

✓ 3 attempts with backoff, then dead-letter

✓ Crash recovery on startup

Architecture: Transactional Outbox + BullMQ

The status mutation and the cascading event are coupled atomically, while the processing itself is decoupled. This prevents both lost cascades (event committed but worker crashes) and blocked requests (the user does not wait for the cascade).

TRIGGER (synchronous, in one transaction)

  ProblemMutationService / IncidentStatusService
    │  BEGIN TX
    │   ├─ Change status (RESOLVED / CLOSED / …)
    │   └─ INSERT CascadingEvent (status = PENDING)   ← Outbox
    │  COMMIT
    │
    └─ cascadingQueue.enqueue(eventId)   ← after commit

PROCESSING (async, BullMQ worker in backend)

  Queue "cascading-events"  (Redis)
    │
    ▼
  processCascadingEvent(eventId)
    ├─ Claim: PENDING → PROCESSING (atomic)
    ├─ INCIDENT → linked tickets
    │   PROBLEM  → tickets + incidents + tickets-of-incidents (2-hop)
    ├─ per target: activity log (System) + SLA + notification + WebSocket
    └─ status → PROCESSED, audit CASCADING_PROCESSED

Runs in: Backend container. Processing starts with the backend, uses Redis as its queue and re-enqueues pending events on startup.

CascadingEvent (Outbox Table)

Field	Values	Description
`entityType`	`INCIDENT`, `PROBLEM`	Source entity of the cascade
`entityId`	ID	Source entity
`event`	`RESOLVED`, `CLOSED`, `MITIGATED`, `REOPENED`, `WORKAROUND_UPDATED`	What happened
`payload`	JSON	resolution, rootCause, mitigation, workaround, userId (for WebSocket echo suppression)
`status`	`PENDING` → `PROCESSING` → `PROCESSED` \| `FAILED`	Lifecycle
`attempts`	Int	Counter (dead-letter after 3)
`error` / `processedAt`	Text / DateTime	Error text resp. timestamp of successful processing

Lifecycle

PENDING ──claim──▶ PROCESSING ──ok──▶ PROCESSED
   ▲                   │
   │                   └──error──▶ attempts < 3 ──▶ PENDING  (retry with backoff)
   │                                attempts ≥ 3 ──▶ FAILED   (dead-letter + audit)
   │
   └─ Startup recovery: PENDING/PROCESSING events older than 10 s are re-enqueued

Propagation Direction

Source	Targets	Link
Incident	linked tickets (1-hop)	`ticketOnIncident`
Problem	directly linked tickets (1-hop)	`ticketOnProblem`
	linked incidents (1-hop)	`incidentOnProblem`
	tickets of those incidents (2-hop)	incident → ticketOnIncident

Double-counting avoidance: A ticket linked both directly to the problem and via an incident receives only the direct entry — the 2-hop path skips tickets already linked directly.

Event Types & Effect

Event	Source	Propagates to	Note
`RESOLVED`	Incident / Problem	tickets (+ incidents + 2-hop for problem)	SLA resume; customer notified
`CLOSED`	Incident / Problem	tickets (+ incidents + 2-hop for problem)	SLA resume (incident)
`REOPENED`	Incident / Problem	tickets (+ incidents + 2-hop for problem)	Incident REOPENED → SLA re-paused; "resolution invalidated"
`MITIGATED`	Incident	tickets	activity + notification only (no SLA resume)
`WORKAROUND_UPDATED`	Problem	incidents ONLY	no ticket entry, no 2-hop

Side Effects per Target

1. Activity Log (System Actor)

Ticket: TicketMessage type=ACTIVITY, activityData.type=CASCADING_UPDATE
Incident: IncidentActivity CASCADING_UPDATE
The author is "System". The triggering user does not appear as author; their client receives no duplicate live update.
Closed tickets (CLOSED/SPAM) and closed incidents (CLOSED) are skipped.

2. SLA Pause / Resume (Incident → Ticket)

RESOLVED / CLOSED → ticket SLA resumes
REOPENED → ticket SLA is paused again
This is the runtime counterpart of the SLA flag pauseOnIncidentLink (SLA).

3. Notifications

Notifications go to the agent (IN_APP) and the customer (email + IN_APP).
Customers are notified only for RESOLVED and REOPENED.
Notification types: LINKED_INCIDENT_{RESOLVED,CLOSED,MITIGATED,REOPENED}, LINKED_PROBLEM_{RESOLVED,CLOSED,REOPENED,WORKAROUND}.
Each notification has a dedupeKey (target + event + cascadingEventId) → no duplicate notifications.
WORKAROUND_UPDATED: incident agents only, no ticket notification.

4. WebSocket Broadcast

After saving, each affected entity receives a live update (channel "activities").
Echo suppression: the triggering client (userId from payload) does not receive a redundant update.

Reliability

Mechanism	Effect
Transactional outbox	Event and status change are stored together or not at all
Atomic claim	PENDING→PROCESSING in one atomic step — no event is processed twice in parallel
Activity dedup	Each target is checked against cascadingEventId — re-runs create no duplicates
Job-Dedup	BullMQ jobId = cascading-{eventId}
Retry	3 attempts, exponential backoff 1 s / 2 s / 4 s
Dead-Letter	After 3 failed attempts → status=FAILED + audit CASCADING_FAILED (ERROR)
Crash-Recovery	On startup: pending events > 10 s are re-enqueued (PROCESSING → PENDING reset)
Stale monitoring	CronJob stale_cascading_reminder flags stuck resolution chains (source resolved, child open)

BullMQ Queue Configuration

Queue:          "cascading-events"  (Redis)
attempts:       3
backoff:        exponential, 1000 ms   (1s, 2s, 4s)
concurrency:    5
limiter:        max 20 / 1000 ms
removeOnComplete: { count: 500, age: 24h }
removeOnFail:     { count: 200 }

Audit

Every cascade is recorded in the enterprise audit system (domain ENTITY, category CASCADING, actor SYSTEM):

CASCADING_PROCESSED (INFO) — successfully processed, with affectedCount
CASCADING_FAILED (ERROR) — after final failure (dead-letter)