Streaming Interface Proposal

Draft generated by AI. This proposal may not have been reviewed for accuracy.

Summary

This proposal defines a unified real-time interface for BinktermPHP centered on a single endpoint:

• `GET /api/stream` for BBS -> UI event delivery
• `POST /api/stream` for UI -> BBS commands

The goal is to make window.BinkStream transport-agnostic so the same client API can operate over:

• WebSocket when available
• Server-Sent Events (SSE) when WebSocket is unavailable
• Short-window SSE reconnect loops when long-lived SSE is unsuitable

This proposal treats short-window SSE as the degraded form of the same streaming interface rather than as a separate polling subsystem.

Goals

• Keep one logical real-time interface for all transports
• Preserve a stable client API regardless of transport
• Support future WebSocket transport without rewriting page-level code
• Preserve Apache compatibility by allowing degraded short-window SSE behavior
• Avoid duplicating event formats and cursor logic across separate endpoints

Non-Goals

• Defining the full WebSocket server implementation
• Replacing all existing page-specific API routes immediately
• Removing the current `/api/stream` SSE behavior in the short term

Core Interface

GET /api/stream

Used for BBS -> UI delivery.

Behavior:

• Returns `Content-Type: text/event-stream`
• Delivers cursor-based real-time events
• Supports long-lived SSE windows
• Supports short-lived reconnect windows via `SSE_WINDOW_SECONDS`
• Remains the canonical event feed even when the client reconnect pattern is effectively polling

Current examples of delivered data:

• `connected`
• `reconnect`
• `chat_message`
• future event types such as dashboard stats, notifications, or system events

POST /api/stream

Used for UI -> BBS commands.

Behavior:

• Accepts JSON POST requests
• Executes a command on behalf of the authenticated user
• Returns JSON immediately
• Provides the non-WebSocket fallback for client-to-server messages

This allows the same logical client API to work in both modes:

• WebSocket: commands are sent over the socket
• SSE: commands are sent via `POST /api/stream`

Directionality Model

The endpoint becomes symmetric by HTTP method:

• `GET /api/stream`: server push path
• `POST /api/stream`: client command path

Conceptually:

• BBS -> UI uses event frames
• UI -> BBS uses command messages

This gives BinkStream a bidirectional abstraction even when the transport itself is not fully bidirectional.

Client API Shape

The browser-facing API should remain transport-neutral:

window.BinkStream.on(type, handler);
window.BinkStream.off(type, handler);
window.BinkStream.send(command, payload);

Transport behavior

WebSocket mode

• `on()` subscribes to incoming WS event messages
• `send()` writes command messages to the socket

SSE mode

• `on()` subscribes to incoming SSE events from `GET /api/stream`
• `send()` performs `POST /api/stream`

The page should not need to care which transport is active.

Configuration Model

Proposed runtime transport selection:

• `BINKSTREAM_TRANSPORT_MODE=auto|ws|sse`

Meaning:

• `auto`: prefer WebSocket when available, otherwise use SSE
• `ws`: require WebSocket transport
• `sse`: require SSE transport

SSE degradation behavior

SSE_WINDOW_SECONDS controls SSE connection behavior:

• larger value: long-lived streaming connection
• `0` or very small value: short-lived reconnect loop

This proposal explicitly treats short-window SSE as the degraded form of the same transport, not as a separate polling subsystem.

That means Apache support can degrade by changing SSE connection strategy rather than by replacing the event system entirely.

Important caveat: short-window SSE does not guarantee acceptable behavior on Apache. In tested Apache + mod_proxy_fcgi deployments, the entire SSE window may still be buffered and dumped on close. That may be tolerable for low-urgency notifications, but it is still poor for interactive chat. This proposal uses short-window SSE as an interface-preserving fallback, not as proof that Apache can deliver real-time SSE correctly.

Client Transport Selection

BINKSTREAM_TRANSPORT_MODE is a server-side setting, but BinkStream still needs to know the effective mode at runtime.

The proposed mechanism is:

• the server resolves the effective transport mode for the request
• the page exposes that value in bootstrapped page context
• `BinkStream` reads that value and selects the transport implementation

For example:

window.siteConfig = {
  realtimeTransportMode: "sse"
};

or:

window.siteConfig = {
  realtimeTransportMode: "ws"
};

This proposal assumes transport selection is server-informed rather than purely client-guessed.

Command Envelope

Suggested request body for POST /api/stream:

{
  "command": "get_dashboard_stats",
  "payload": {}
}

Suggested response body:

{
  "success": true,
  "result": {}
}

Optional future fields:

• `request_id`
• `error_code`
• `error`

Example commands

• `get_dashboard_stats`
• `mark_notification_seen`
• `subscribe`
• `unsubscribe`
• `ping`

Not every command must immediately emit an event. Some commands may return only a direct JSON result.

Event Envelope

The existing SSE event model should remain authoritative.

Example:

id: 44
event: chat_message
data: {"id":101,"type":"room","room_id":1}

WebSocket mode should mirror the same logical event types so subscribers do not care whether they arrived by SSE or WS.

Suggested WebSocket event shape:

{
  "type": "chat_message",
  "id": 44,
  "data": {
    "id": 101,
    "type": "room",
    "room_id": 1
  }
}

Authentication

Both GET /api/stream and POST /api/stream should require normal authenticated user context.

SSE

• existing session-cookie authentication remains valid

POST commands

• same authenticated session
• same CSRF protections as other state-changing API endpoints

WebSocket

Future WebSocket transport will need an authenticated handshake strategy consistent with the current web session model.

Why a Unified /api/stream

Benefits:

• clearer mental model
• fewer endpoint names to document
• one obvious place for real-time traffic
• easier transport abstraction inside `BinkStream`
• easier future parity between WS and SSE

Instead of teaching the client separate concepts such as:

• event endpoint
• command endpoint
• polling endpoint

the application exposes one logical streaming interface with method-based directionality.

Apache Compatibility

Testing has shown that some Apache + mod_proxy_fcgi deployments buffer SSE responses even when direct PHP-FPM delivery streams correctly.

Because of that, Apache support should be treated as degraded SSE mode:

• same `GET /api/stream` endpoint
• same cursor semantics
• same event format
• shorter or zero SSE windows, causing reconnect-driven polling behavior

This avoids building and maintaining a second event delivery mechanism just for Apache.

However, Apache should still be considered a degraded environment for real-time interactivity. Short-window SSE preserves endpoint and cursor consistency, but it does not guarantee chat-quality latency when Apache buffers the whole response window.

WebSocket Deployment Considerations

The future WebSocket server will need additional deployment support beyond the current PHP-FPM HTTP path.

At minimum, sysops will need:

• a dedicated WebSocket listener port or Unix socket
• a reverse-proxy rule in the frontend web server
• TLS-compatible upgrade handling
• a process supervisor entry for the WebSocket daemon

This is a real operational burden and should be treated as part of the design, not an implementation footnote.

Migration Path

Phase 1

• keep current `GET /api/stream`
• add `POST /api/stream`
• add `BinkStream.send()`
• continue using SSE-only delivery

Phase 2

• introduce WebSocket server
• teach `BinkStream` to prefer WS when configured
• preserve `GET /api/stream` and `POST /api/stream` fallback behavior

Phase 3

• move more page-level real-time requests onto the unified `BinkStream` interface
• reduce transport-specific logic in page scripts

Open Questions

• Which commands should return only direct JSON versus also emitting follow-up events?
• Should `subscribe` and `unsubscribe` be explicit commands, or remain client-local in SSE mode?
• Should command responses be normalized to a single envelope immediately, or support route-specific payloads during migration?
• Should `BINKSTREAM_TRANSPORT_MODE=auto` permit WS -> SSE fallback automatically, or should failed WS setup force an explicit config change?
• What public WebSocket URL, listener port, and reverse-proxy rules should be standardized for sysops?
• Should Apache be documented as notification-safe but chat-degraded even in short-window SSE mode?

WebSocket Server Runtime

This section describes the preferred server-side implementation strategy for a future WebSocket transport.

PHP vs Node.js

The preferred implementation language is PHP.

Reasons:

• reuses the existing application language
• makes it easier to share auth, config, command handlers, and event formatting
• keeps deployment and contributor workflows aligned with the rest of BinktermPHP
• makes it easier to keep WebSocket, `GET /api/stream`, and `POST /api/stream` behavior consistent

Node.js remains a viable option, but it introduces a second runtime and code stack.

Important constraint

A single-process event loop has the same core weakness in both PHP and Node.js:

• long-running work blocks processing of other clients

Node.js does not remove this problem. It mainly makes non-blocking I/O easier to structure. CPU-heavy or synchronous work still blocks the event loop there as well.

Because of that, the primary architectural decision is not PHP versus Node.js. It is:

• inline single-loop handling versus
• a main I/O loop plus worker isolation

Preferred architecture

Use a portable PHP design:

• one main process owns all client sockets
• a `stream_select()` loop handles network I/O
• slow or blocking commands are delegated to separate worker processes

This provides a portable baseline that works on both Unix and Windows.

Forking

Forking can be used as an optimization on Unix-like systems, but it should not be the core design.

Why:

• Windows does not support `pcntl_fork()`
• the project still needs a non-fork design for Windows
• the non-fork design should therefore be the primary architecture

That means:

• baseline: spawned worker pool
• optional Unix enhancement: fork-based worker model

Process model

Main process

Responsibilities:

• accept WebSocket connections
• perform handshake and authentication
• parse frames
• track subscriptions and per-connection state
• dispatch commands
• process worker responses
• send outbound frames to clients

The main process should not run expensive business logic inline unless it is known to be trivial and non-blocking.

Worker processes

Responsibilities:

• handle slow or blocking commands
• perform database-heavy work
• produce command results
• optionally emit follow-up events back to the main process

Workers should not own client sockets.

Why spawned worker processes

When fork() is unavailable, long-lived child processes can still be created with normal process spawning.

This is the preferred portability story for Windows:

• start a fixed worker pool at server boot
• keep those workers alive
• communicate with them over IPC

This avoids blocking the main socket loop without relying on Unix-specific process control.

Worker pool sizing

The worker pool should be explicitly configurable.

Suggested environment variables:

• `REALTIME_WORKER_COUNT`
• `REALTIME_WORKER_MAX_QUEUE`

Initial default recommendation:

• small installs: `2` workers
• moderate installs: `4` workers

The right count depends on:

• expected concurrent users
• command mix
• database latency
• whether commands are mostly lightweight or potentially blocking

Worker pool exhaustion

The proposal must define behavior when all workers are busy.

Suggested initial behavior:

• maintain a bounded in-memory queue in the main process
• when the queue is full, reject new slow commands immediately
• return a structured busy error rather than blocking the socket loop indefinitely

Example failure response:

{
  "success": false,
  "error_code": "errors.realtime.server_busy",
  "error": "Realtime worker pool is busy"
}

This is preferable to silently stalling all active clients.

IPC recommendation

Use local loopback TCP as the first IPC mechanism.

Reasons:

• cross-platform
• easy to inspect and debug
• works on both Unix and Windows
• integrates naturally with `stream_select()`

Suggested message framing:

• newline-delimited JSON

IPC envelope examples

Server -> worker:

{
  "type": "command",
  "job_id": "abc123",
  "conn_id": "c42",
  "user_id": 7,
  "command": "get_dashboard_stats",
  "payload": {}
}

Worker -> server command result:

{
  "type": "command_result",
  "job_id": "abc123",
  "conn_id": "c42",
  "success": true,
  "result": {
    "unread_netmail": 1
  }
}

Worker -> server event:

{
  "type": "event",
  "target": {
    "subscription": "chat_message"
  },
  "event": "chat_message",
  "data": {
    "id": 101
  }
}

Connection ownership

The main process should own:

• connection sockets
• authentication state
• subscription tables
• job-to-connection correlation

Workers should only handle background command execution and return structured results.

Suggested command routing

Safe to handle inline in the main loop:

• `ping`
• `subscribe`
• `unsubscribe`

Better delegated to workers:

• `get_dashboard_stats`
• database-heavy queries
• filesystem access
• any command that may block for noticeable time

Suggested code structure

• `src/Realtime/WebSocketServer.php`
• `src/Realtime/WebSocketConnection.php`
• `src/Realtime/WebSocketFrameParser.php`
• `src/Realtime/WebSocketFrameWriter.php`
• `src/Realtime/WorkerPool.php`
• `src/Realtime/WorkerClient.php`
• `src/Realtime/CommandDispatcher.php`
• `src/Realtime/Commands/`
• `src/Realtime/EventBus.php`
• `scripts/websocket_server.php`
• `scripts/realtime_worker.php`

Recommendation

Build the future WebSocket transport in PHP.

Use:

• a main `stream_select()` socket loop
• a spawned worker pool for slow commands
• shared command and event handlers reused by both WebSocket and `/api/stream`

Treat Unix forking as an optional optimization layer, not as the primary runtime model.

Recommendation

Adopt /api/stream as the unified real-time interface:

• `GET /api/stream` for event delivery
• `POST /api/stream` for commands

Keep BinkStream as the transport-neutral browser API.

Treat short-window SSE as degraded SSE rather than as a separate polling subsystem.

Add WebSocket later as a preferred transport without changing page-level consumers.