BinkStream Back-Channel

BinktermPHP uses BinkStream as its browser-facing real-time channel. BinkStream is a shared command and event interface with multiple transports behind it:

• WebSocket when available
• SSE when WebSocket is unavailable or disabled
• short-window SSE as the degraded Apache-friendly fallback

The business logic is shared. GET /api/stream, POST /api/stream, and the standalone WebSocket server all use the same realtime core.

Table of Contents

1. Architecture Overview
2. Current Transport Model
3. Key Files
4. The sse_events Table
5. Event Targeting
6. Admin Daemon Role
7. The /api/stream Endpoint
8. Command Path
9. SharedWorker (binkstream-worker-v2.js)
10. Client Library (binkstream-client.js)
11. Subscribing to Events in JavaScript
12. Publishing a New Event
13. Connection Lifecycle
14. Debugging - Check the active transport mode - Diagnosing Apache SSE buffering - Reverse proxy for the realtime server - Inspect the SharedWorker - Dev mode source badge - Check recent stream rows

Architecture Overview

PostgreSQL triggers / PHP app code
      |
      | INSERT INTO sse_events (event_type, payload, user_id, admin_only)
      v
src/Realtime/StreamService.php
      |
      +--> GET /api/stream      -> SSE event delivery
      |
      +--> WebSocket server     -> WS event delivery
      |
      v
src/Realtime/CommandDispatcher.php
      |
      +--> POST /api/stream     -> UI -> BBS commands in SSE mode
      |
      +--> WebSocket server     -> UI -> BBS commands in WS mode
      v
SharedWorker (public_html/js/binkstream-worker-v2.js)
      |
      v
Client library (public_html/js/binkstream-client.js)
      |
      v
window.BinkStream.on(...) / off(...) / send(...)

The important design rule is that WebSocket is not a separate realtime subsystem. It is another transport over the same command/event core used by /api/stream.

BinkStream is a platform service, not a chat-only add-on. It is the common live-update path for notifications, chat, dashboards, admin tools, and other browser features that need near-real-time state.

Current Transport Model

The effective transport is controlled by BINKSTREAM_TRANSPORT_MODE.

Supported values today:

• `auto`
• `sse`
• `ws`

In auto mode:

• the server prefers WebSocket when the BinkStream daemon PID file exists
• the browser still confirms availability by successfully opening the socket and receiving the `connected` message
• if that handshake fails, the worker falls back to SSE
• while running on SSE fallback, the worker re-probes WebSocket every 30 seconds and switches back automatically if it becomes available

In sse mode:

• the worker skips WebSocket and uses `/api/stream` directly

In ws mode:

• the worker requires WebSocket and retries it with backoff if disconnected

Apache note

Testing has shown that Apache + PHP-FPM can buffer SSE responses, including short-window SSE. Short windows reduce the delay but do not guarantee event-by-event delivery. This makes short-window SSE acceptable for notifications and degraded chat behavior, but it is not equivalent to real streaming.

For that reason:

• Apache deployments should treat SSE as a degraded compatibility path
• Caddy and direct PHP-FPM testing have shown correct realtime streaming behavior

Key Files

| File | Purpose | |---|---| | database/migrations/v1.11.0.55_sse_events_table.php | Creates sse_events table and installs the initial DB trigger | | database/migrations/v1.11.0.57_sse_events_user_targeting.php | Adds user_id / admin_only targeting columns and chat fat payload delivery | | src/Admin/AdminDaemonServer.php | Periodic sse_events pruning | | src/Realtime/StreamService.php | Shared event fetch, cursor anchor, SSE window resolution | | src/Realtime/CommandDispatcher.php | Shared realtime command execution for HTTP and WebSocket | | src/Realtime/WebSocketServer.php | Standalone WebSocket server | | scripts/realtime_server.php | CLI daemon entrypoint for the WebSocket server | | routes/api-routes.php | GET /api/stream for events and POST /api/stream for commands | | public_html/js/binkstream-worker-v2.js | SharedWorker transport layer; chooses WS or SSE | | public_html/js/binkstream-client.js | Per-tab client; exposes window.BinkStream |

The sse_events Table

CREATE UNLOGGED TABLE sse_events (
    id          BIGSERIAL    PRIMARY KEY,
    event_type  VARCHAR(64)  NOT NULL,
    payload     JSONB        NOT NULL DEFAULT '{}',
    user_id     INTEGER      NULL REFERENCES users(id) ON DELETE CASCADE,
    admin_only  BOOLEAN      NOT NULL DEFAULT FALSE,
    created_at  TIMESTAMPTZ  NOT NULL DEFAULT NOW()
);

UNLOGGED means no WAL writes, so inserts are fast. The table is automatically truncated on a Postgres crash, which is acceptable because it is a transient delivery queue. Actual data lives in domain tables permanently.

Why a separate table instead of using domain IDs?

Using chat_messages.id as the stream cursor would break the moment a second event type is added. sse_events.id is a single monotonic sequence across all event types, so the cursor works correctly regardless of how many event types share the stream.

Fat payload pattern: triggers store all fields needed to render the event directly in the payload column. The hot delivery path reads sse_events alone and avoids joins.

Retention: the admin daemon deletes rows older than one hour from its main loop. Autovacuum handles dead tuples from those deletes.

Event Targeting

Each row in sse_events carries two targeting columns that control which users receive it:

| user_id | admin_only | Delivered to | |---|---|---| | NULL | FALSE | All authenticated users | | NULL | TRUE | Admin users only | | <id> | FALSE | Specific user only | | <id> | TRUE | Specific user, only if admin |

The delivery query enforces targeting with a simple WHERE filter:

SELECT id AS sse_id, event_type, payload::text AS event_data
FROM sse_events
WHERE id > :cursor
  AND (user_id IS NULL OR user_id = :user_id)
  AND (admin_only = FALSE OR admin_only = TRUE)
ORDER BY id ASC
LIMIT 200

The admin_only check uses an inlined SQL literal (TRUE or FALSE) rather than a bound parameter to avoid PostgreSQL boolean/text type issues with PDO.

For new events, no changes to the delivery query are needed. Set user_id and admin_only when inserting into sse_events and targeting is enforced automatically.

Admin Daemon Role

The admin daemon's only BinkStream-specific responsibility today is periodic cleanup:

DELETE FROM sse_events WHERE created_at < NOW() - INTERVAL '1 hour'

The daemon does not deliver events to browsers directly. Event delivery happens through /api/stream or the WebSocket server, both of which query sse_events directly.

The /api/stream Endpoint

GET /api/stream requires authentication and returns Content-Type: text/event-stream.

session_write_close() is called immediately after authentication to release the PHP session file lock. Without that, a long-lived stream would block other requests from the same browser session.

Cursor

The endpoint needs a starting position to know which events to deliver. On reconnect, the worker passes the last seen sse_events.id as the ?cursor= URL parameter. The endpoint also accepts the standard Last-Event-ID HTTP header, but the worker always supplies ?cursor= explicitly because browsers can clear EventSource.lastEventId when the connection is manually closed.

Priority:

1. `Last-Event-ID` header
2. `?cursor=` query parameter
3. `0` on first connect

First connection

On first connect, the endpoint anchors at the current max cursor without replaying historical events. Historical page state remains the responsibility of the page's normal load APIs.

Reconnect with known cursor

The endpoint immediately runs the catch-up query and emits any pending events, then stays open for the configured SSE window, polling every 200 ms and sending keepalive comments every 15 seconds. When the window expires, production mode emits event: reconnect and closes so the worker can reconnect immediately.

Polling window

SSE_WINDOW_SECONDS controls how long each SSE request stays open.

• normal default: `60`
• Apache + `BINKSTREAM_TRANSPORT_MODE=auto`: implicit default `2`
• built-in PHP dev server: forced `0`

An explicit SSE_WINDOW_SECONDS in .env always wins.

Apache caveat

On Apache, even a short window may still be buffered and delivered in clumps at window close. That is why the system treats short-window SSE as degraded compatibility behavior, not a true fix for Apache realtime streaming.

Command Path

Outbound events and inbound commands share the same interface but use different directions:

• `GET /api/stream` - BBS -> UI - SSE event delivery
• `POST /api/stream` - UI -> BBS - command submission when using SSE transport
• WebSocket - BBS -> UI events and UI -> BBS commands over one socket

From page code, the transport-specific details stay behind window.BinkStream:

window.BinkStream.on('chat_message', handler);
window.BinkStream.send('get_dashboard_stats', {});

In WebSocket mode, send() writes a command frame to the socket. In SSE mode, send() uses POST /api/stream and returns the command result over HTTP.

SharedWorker (binkstream-worker-v2.js)

The SharedWorker owns one active transport for the whole origin and fans events out to all tabs through MessagePort.

It is responsible for:

• maintaining the last seen `sse_events.id` cursor
• trying the preferred transport
• falling back from WebSocket to SSE when needed
• re-probing WebSocket every 30 seconds while running on SSE fallback
• reporting active transport changes back to tabs

Cursor tracking

The worker keeps a lastCursor value containing the most recent sse_events.id. On SSE reconnect it sends that cursor back to /api/stream. In WebSocket mode the same cursor is used for initial catch-up and reconnect continuity.

Connection lifecycle

auto mode
  -> try WebSocket first when server preference is ws
  -> if WS reaches connected: use ws
  -> if WS handshake fails: switch to sse
  -> while on sse fallback: re-probe ws every 30 s

Dynamic event type subscription

Tabs call BinkStream.on('some_type', fn), which causes the client to send {action: 'subscribe', type: 'some_type'} to the worker. The worker tracks subscribed types and re-registers them whenever the underlying transport reconnects.

Backoff

• forced `ws` mode retries WebSocket with exponential backoff from 1 second to 30 seconds
• `auto` mode falls back to SSE after WS handshake failure, then periodically re-probes WS
• SSE reconnects continue to follow the existing reconnect logic around `event: reconnect` and error backoff

Client Library (binkstream-client.js)

Loaded on authenticated pages and exposes window.BinkStream.

API

window.BinkStream.on('chat_message', function (payload) {
    console.log('New message:', payload);
});

window.BinkStream.off('chat_message', handler);

window.BinkStream.send('get_dashboard_stats', {}).then(function (result) {
    console.log(result);
});

window.BinkStream.getMode(); // "ws", "sse", or null until connected

The client talks to the SharedWorker and keeps the page API transport-agnostic.

Subscribing to Events in JavaScript

Example: chat notifications on every page

public_html/js/notifier.js can subscribe to chat_message without caring whether the underlying transport is WebSocket or SSE:

window.BinkStream.on('chat_message', function (payload) {
    if (payload.from_user_id === window.currentUserId) return;
    playNotificationSound();
});

Example: updating the chat thread in real time

public_html/js/chat-page.js subscribes on the chat page only:

window.BinkStream.on('chat_message', function (payload) {
    if (!payload || !payload.id) return;
    payload._source = window.BinkStream.getMode() || 'sse';
    handleIncoming(payload);
    if (payload.id > state.lastChatId) {
        state.lastChatId = payload.id;
        saveState();
    }
});

payload.id is the domain ID such as chat_messages.id, not the stream cursor. The cursor is internal to the worker.

Workflow: how realtime updates flow

1. A platform subsystem inserts a targeted event into `sse_events`.
2. `StreamService` reads the event and exposes it through SSE or WebSocket.
3. The SharedWorker keeps one active connection for the origin and tracks the stream cursor.
4. Open tabs subscribe to event types through `window.BinkStream.on(...)`.
5. UI code updates page state, notifications, or live widgets when the payload arrives.

Publishing a New Event

Step 1: insert a fat payload into sse_events

If the event is triggered by a DB change, add a trigger that inserts a fully renderable payload into sse_events:

INSERT INTO sse_events (event_type, payload, user_id, admin_only)
VALUES (
    'user_online',
    json_build_object(
        'user_id',  NEW.id,
        'username', NEW.username,
        'online',   NEW.is_online
    ),
    NULL,
    FALSE
);

If inserting from PHP:

$db = Database::getInstance()->getPdo();
$stmt = $db->prepare("
    INSERT INTO sse_events (event_type, payload, user_id, admin_only)
    VALUES ('user_online', :payload, :user_id, FALSE)
");
$stmt->execute([
    ':payload' => json_encode([
        'user_id' => $userId,
        'username' => $username,
        'online' => true,
    ]),
    ':user_id' => null,
]);

Step 2: delivery is automatic

No transport code changes are needed. The shared event fetch query delivers every row from sse_events regardless of its event_type value.

Step 3: subscribe in JavaScript

window.BinkStream.on('user_online', function (payload) {
    updateUserPresenceIndicator(payload.user_id, payload.online);
});

No changes to binkstream-worker-v2.js or binkstream-client.js are required.

Step 4: bump the service worker cache

When changing binkstream-worker-v2.js, binkstream-client.js, or i18n strings used by those scripts, increment CACHE_NAME in public_html/sw.js.

Connection Lifecycle

Page load
  |
  +-- binkstream-client.js -> SharedWorker('/js/binkstream-worker-v2.js')
  |       |
  |       +-- choose ws or sse based on config and runtime availability
  |
  +-- WebSocket path
  |       |
  |       +-- connect to configured WS URL
  |       +-- authenticate via existing session cookie
  |       +-- subscribe / send commands / receive events
  |
  +-- SSE path
  |       |
  |       +-- GET /api/stream?cursor=N
  |       +-- receive connected event
  |       +-- receive event batches until reconnect or close
  |       +-- POST /api/stream for commands
  |
  +-- database change
          |
          +-- trigger or PHP inserts row into sse_events
          +-- StreamService fetches it
          +-- transport delivers it to worker
          +-- worker fans it out to tabs

Debugging

Inspect the SharedWorker

SharedWorker network requests do not appear in the main frame's Network tab. To inspect them:

1. Open `chrome://inspect/#workers` or `edge://inspect/#workers`
2. Find `binkstream-worker-v2.js`
3. Click `inspect`

The worker console logs its transport decisions, including:

• configured and preferred mode at init
• trying WebSocket
• using WebSocket
• WebSocket failure and SSE fallback
• trying SSE
• using SSE
• periodic WebSocket re-probes from SSE fallback

Dev mode source badge

When IS_DEV=true, chat messages can show a small source badge:

• `ws`
• `sse`
• `poll`

This makes it easy to verify which delivery path actually delivered a message.

Check the active transport mode

The admin dashboard (Admin ? Dashboard, Service Status section) shows the active transport next to the Realtime Server entry: WebSocket, SSE, or poll. This is the live mode reported by the SharedWorker in your browser and updates automatically if the transport changes.

poll means the SharedWorker is not available (the browser does not support it, or it failed to load). SSE means the realtime server is not running or the WebSocket connection could not be established. WebSocket is the expected steady state when the realtime server is running.

Diagnosing Apache SSE buffering

If the realtime server is not running and you are behind Apache, SSE events may be buffered and delivered in clumps rather than one at a time. Signs of this:

• Chat messages appear in bursts rather than individually
• Notification sounds fire late or all at once
• The source badge (when `IS_DEV=true`) shows `sse` but messages still feel delayed

Confirming the problem via the SharedWorker inspector:

Because /api/stream is opened inside a SharedWorker, it does not appear in the main frame's Network tab. To watch the SSE stream directly:

1. Open `chrome://inspect/#workers` (or `edge://inspect/#workers`)
2. Find `binkstream-worker-v2.js` and click inspect
3. In the worker's Network tab, find the `/api/stream` request
4. Click it and watch the EventStream tab
5. Send a chat message from another session
6. If the event appears immediately: SSE is flushing correctly
7. If the event appears only after a delay or alongside several others: Apache is buffering

Fix options (in order of preference):

1. Run the realtime server behind a reverse proxy ? eliminates the problem entirely. See Reverse proxy setup below.

2. Disable output buffering for the SSE endpoint ? add to your Apache VirtualHost or `.htaccess`:
   <LocationMatch "^/api/stream">
       SetEnv no-gzip 1
       SetEnv dont-vary 1
   </LocationMatch>
   

   If using `mod_proxy_fcgi`, also add `flushpackets=on` to the ProxyPass directive:
   

   ProxyPass /api/stream fcgi://127.0.0.1:9000/path/to/public_html/index.php flushpackets=on
3. Lower SSE_WINDOW_SECONDS ? reduces the maximum delay before a buffered window is flushed. Set to `2` (or leave unset on Apache, where `auto` mode already defaults to `2`). This does not fix buffering but limits its impact. See `docs/CONFIGURATION.md` for details.

Reverse proxy for the realtime server

The realtime WebSocket daemon (scripts/realtime_server.php) binds to 127.0.0.1:6010 by default and must be exposed to browsers through a reverse proxy. The browser connects to the public /ws path; the proxy upgrades the connection and forwards it to the daemon.

.env settings:

BINKSTREAM_TRANSPORT_MODE=auto
BINKSTREAM_WS_BIND_HOST=127.0.0.1
BINKSTREAM_WS_PORT=6010
BINKSTREAM_WS_PUBLIC_URL=/ws

Caddy:

yourdomain.com {
    # ... your existing PHP / php-fpm config ...

    reverse_proxy /ws 127.0.0.1:6010 {
        header_up Host {host}
        header_up X-Real-IP {remote_host}
    }
}

Caddy handles the WebSocket upgrade automatically ? no extra directives are needed.

Nginx:

location /ws {
    proxy_pass         http://127.0.0.1:6010;
    proxy_http_version 1.1;
    proxy_set_header   Upgrade $http_upgrade;
    proxy_set_header   Connection "upgrade";
    proxy_set_header   Host $host;
    proxy_read_timeout 3600s;
}

Apache with mod_proxy_wstunnel:

WebSocket is a raw TCP tunnel after the upgrade handshake, so it should not be affected by the same response-body buffering that causes SSE problems under Apache ? though this has not been verified. Requires mod_proxy and mod_proxy_wstunnel to be enabled (a2enmod proxy proxy_wstunnel).

ProxyPass        /ws  ws://127.0.0.1:6010/
ProxyPassReverse /ws  ws://127.0.0.1:6010/

Check recent stream rows

SELECT id, event_type, user_id, admin_only, payload, created_at
FROM sse_events
ORDER BY id DESC
LIMIT 20;

SELECT event_type, count(*)
FROM sse_events
WHERE created_at > NOW() - INTERVAL '1 hour'
GROUP BY event_type;

Related Systems

• Architecture ? where BinkStream sits in the platform
• API Reference ? `/api/stream` and related HTTP APIs
• Admin Terminal ? one of the live consumers of realtime events
• WebDoors ? browser-native doors that can reuse live platform APIs