Downlink & Peer Echomail Distribution Proposal

DRAFT DOCUMENT

This proposal is a draft document generated by AI and may not have been reviewed for accuracy. The technical details, implementation decisions, and schema described herein should be validated by developers familiar with FidoNet protocols and the BinktermPHP codebase before implementation begins.

Overview

This document proposes adding downlink/peer distribution to BinktermPHP: the ability to bundle and deliver echomail (and route netmail) to subordinate nodes and peers, in both push (we poll them) and pull (they poll us) modes. The design targets up to ~100 peers.

This is distinct from the existing uplink model, where BinktermPHP receives mail from and sends mail up to a hub. Here we are the hub ? or at least a distribution point ? for nodes below or beside us in the network.

Table of Contents

• Concepts
• Requirements Summary
• Database Schema
• Echomail Fanout Engine
• Netmail Routing
• SEEN-BY and PATH Handling
• Binkp Integration
• Admin Interface
• Areafix (Future Phase)
• Implementation Plan
• Open Questions

Concepts

| Term | Meaning | |---|---| | Uplink | A node we receive echomail from and send our echomail up to. Already supported. | | Downlink | A node we distribute echomail to. They are subscribed to specific echo areas we carry. | | Peer | A node at the same level; mail exchange is bidirectional and symmetric. Modelled identically to a downlink ? the distinction is social/topological, not technical. | | Push | We poll the downlink to deliver their waiting mail. | | Pull | The downlink connects to our binkp server to collect their mail. | | Areafix | A netmail robot that lets downlinks self-manage their area subscriptions via netmail commands. Planned for a later phase. |

Requirements Summary

• Up to ~100 downlinks/peers
• Per-downlink area subscriptions (which echoes they receive)
• Both push (we poll) and pull (they poll us) delivery
• Echomail fanout with correct SEEN-BY/PATH handling
• Netmail passthrough to known downlinks
• Admin UI for managing downlinks and subscriptions
• No breaking changes to existing uplink behaviour
• Areafix robot deferred to a later phase but schema must accommodate it cleanly

Database Schema

downlinks table

Stores each peer or downlink node.

CREATE TABLE downlinks (
    id                  SERIAL PRIMARY KEY,
    node_address        VARCHAR(50)  NOT NULL UNIQUE,  -- e.g. 2:345/67
    name                VARCHAR(100),                  -- system name
    sysop_name          VARCHAR(100),
    session_password    VARCHAR(255),   -- binkp session password (they poll us)
    packet_password     VARCHAR(255),   -- .pkt password
    inet_host           VARCHAR(255),                      -- override host for push; falls back to nodelist
    port                INTEGER,                            -- override port for push; falls back to nodelist/default
    enabled             BOOLEAN     NOT NULL DEFAULT TRUE,
    allow_inbound       BOOLEAN     NOT NULL DEFAULT TRUE,  -- they may poll us
    allow_outbound      BOOLEAN     NOT NULL DEFAULT TRUE,  -- we poll them
    allow_inbound_echomail BOOLEAN  NOT NULL DEFAULT TRUE,  -- accept echomail from them (peer mode)
    max_packet_kb       INTEGER     NOT NULL DEFAULT 0,     -- 0 = unlimited
    hold_mail           BOOLEAN     NOT NULL DEFAULT FALSE, -- hold mail (paused)
    queue_retention_days INTEGER    NOT NULL DEFAULT 30,    -- days to keep sent outbound rows
    notes               TEXT,
    created_at          TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    last_session_at     TIMESTAMPTZ
);

CREATE INDEX idx_downlinks_address ON downlinks (node_address);
CREATE INDEX idx_downlinks_enabled  ON downlinks (enabled) WHERE enabled = TRUE;

downlink_areas table

Maps which echo areas a downlink receives.

CREATE TABLE downlink_areas (
    id           SERIAL PRIMARY KEY,
    downlink_id  INTEGER NOT NULL REFERENCES downlinks(id) ON DELETE CASCADE,
    echoarea_id  INTEGER NOT NULL REFERENCES echoareas(id) ON DELETE CASCADE,
    paused       BOOLEAN NOT NULL DEFAULT FALSE,  -- temporary hold on this area only
    subscribed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE (downlink_id, echoarea_id)
);

CREATE INDEX idx_downlink_areas_downlink  ON downlink_areas (downlink_id);
CREATE INDEX idx_downlink_areas_echoarea  ON downlink_areas (echoarea_id);

downlink_outbound table

Queue of bundled packets waiting for delivery to each downlink. Keeping per-downlink, per-packet rows allows reliable retry, status tracking, and per-downlink queue inspection.

CREATE TABLE downlink_outbound (
    id             SERIAL PRIMARY KEY,
    downlink_id    INTEGER     NOT NULL REFERENCES downlinks(id) ON DELETE CASCADE,
    message_type   VARCHAR(20) NOT NULL DEFAULT 'echomail',  -- echomail | netmail
    echoarea_id    INTEGER     REFERENCES echoareas(id) ON DELETE SET NULL,
    echomail_id    INTEGER     REFERENCES echomail(id)  ON DELETE SET NULL,
    netmail_id     INTEGER     REFERENCES netmail(id)   ON DELETE SET NULL,
    packet_data    BYTEA       NOT NULL,
    size_bytes     INTEGER     NOT NULL DEFAULT 0,
    priority       SMALLINT    NOT NULL DEFAULT 5,
    attempts       SMALLINT    NOT NULL DEFAULT 0,
    status         VARCHAR(20) NOT NULL DEFAULT 'pending',  -- pending | sent | failed | held
    created_at     TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    next_attempt_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    sent_at        TIMESTAMPTZ,
    error_message  TEXT
);

CREATE INDEX idx_dl_outbound_pending ON downlink_outbound (downlink_id, next_attempt_at)
    WHERE status = 'pending';
CREATE INDEX idx_dl_outbound_downlink ON downlink_outbound (downlink_id);

Echomail Fanout Engine

When fanout fires

Fanout runs whenever echomail is added to the system, from two sources:

1. Inbound packet processing ? after `PacketProcessor` stores an inbound echomail message
2. Local posting ? after a user or robot posts to an echo area

A new DownlinkFanout class handles the logic. It is called by the packet processor and the message posting path.

Fanout algorithm

For each new echomail message M in area A:
  1. Fetch all active downlinks subscribed to A (paused=false, hold_mail=false)
  2. Parse M's SEEN-BY kludge into a set of address integers
  3. For each downlink D:
       a. If D's node address is already in SEEN-BY ? skip (already has it)
       b. Build a copy of the message with:
            - SEEN-BY updated: add our address + D's address
            - PATH kludge updated: append our address
       c. Serialise to .pkt format addressed to D
       d. INSERT into downlink_outbound (downlink_id=D, status='pending')

SEEN-BY address format

FTN SEEN-BY lines use the compact net/node form (zone is implicit from packet header). The fanout engine must:

• Parse all `SEEN-BY:` lines in the message
• Expand them using the zone from the packet/area context
• Emit correctly formatted SEEN-BY lines in outbound packets

Batching

Rather than one downlink_outbound row per message per downlink (which could be large for high-volume areas), the delivery worker batches pending messages into a single .pkt file per downlink session, up to max_packet_kb. This is handled at delivery time, not at queue time. The queue rows act as a work list; the delivery worker assembles them into packets on the fly.

Netmail Routing

When a netmail arrives whose destination address matches a downlink:

1. `PacketProcessor` checks the destination against `downlinks.node_address`
2. If matched: serialise the message to a .pkt addressed to the downlink and insert into `downlink_outbound` with `message_type='netmail'`
3. If not matched and routing is not configured: leave existing behaviour (store locally or discard)

Netmail routing is opt-in and disabled by default. A config flag DOWNLINK_ROUTE_NETMAIL=true enables it.

SEEN-BY and PATH Handling

Correct SEEN-BY and PATH kludge management is essential to prevent mail loops.

Rules

• Before forwarding: check the downlink's address is not already in SEEN-BY
• When building outbound packet: - Merge and sort all SEEN-BY entries; add our own address and the downlink's address - Append our address to PATH (do not duplicate if already present)
• Loop detection: if our own address appears in PATH more than once, do not forward

Key class: EchomailSeenBy

class EchomailSeenBy {
    public static function parse(string $seenByLines): array;   // returns [zone => [net => [node, ...]]]
    public static function contains(array $seenBy, string $address): bool;
    public static function addAddress(array $seenBy, string $address): array;
    public static function format(array $seenBy): string;        // back to SEEN-BY: lines
}

Transport Abstraction

The fanout engine (deciding which downlinks get which messages and queuing them) is identical regardless of transport. What differs is how queued messages are packaged and delivered. The downlinks table carries a transport column:

| Value | Delivery method | |---|---| | binkp | FTN .pkt files exchanged over binkp sessions (push or pull) | | qwk | QWK ZIP packets downloaded/uploaded over HTTP |

All transport-specific logic lives in a DownlinkDelivery strategy class selected by transport type. The fanout engine writes to downlink_outbound the same way for both.

QWK transport

When transport='qwk', the downlink is another BBS system (not an individual user). Authentication is by session_password against a dedicated HTTP endpoint ? not a BBS user account. The QwkPacketGenerator class (defined in the QWK Proposal) must be refactored to accept either a user context or a downlink context, driving conference lists from downlink_areas instead of user subscriptions. SEEN-BY and PATH kludges are not included in QWK packets; BinktermPHP adds them when tossing inbound REP replies back into echomail.

QWK downlinks do not use inet_host/port ? delivery is via HTTP endpoints on this system, not outbound sessions.

See the QWK Proposal for packet format details.

Binkp Integration

Pull: downlink polls us

When a downlink connects to our binkp server:

1. Server looks up the connecting address against `downlinks.node_address`
2. Authenticates using `downlinks.session_password`
3. Queries `downlink_outbound WHERE downlink_id=? AND status='pending'` and streams those packets
4. On successful transfer, marks rows `status='sent'`, updates `downlinks.last_session_at`

The existing BinkPSession class needs to be extended to handle downlink sessions alongside uplink sessions. The main differentiator is the outbound queue source: uplinks use mrc_outbound, downlinks use downlink_outbound.

Push: we poll the downlink

The existing binkp poll script (binkp_poll.php) targets uplinks. A new --downlink=<address> flag (or --all-downlinks) will:

1. Look up the downlink address in the `downlinks` table
2. Open a binkp session to the downlink's inet host (from nodelist or manually configured)
3. Deliver all pending `downlink_outbound` packets for that downlink
4. Accept any inbound mail the downlink wants to send us

Push scheduling is handled by the existing binkp scheduler ? downlinks can be added to the schedule alongside uplinks.

Admin Interface

Downlinks list page (/admin/downlinks)

• Table of all configured downlinks with address, name, status, last session, queue depth
• Add / Edit / Delete downlink
• Quick toggle: enable/disable, hold mail

Downlink detail / area subscriptions

• Checklist of all echo areas with current subscription status per downlink
• Bulk subscribe/unsubscribe
• Per-area pause toggle
• Current queue depth and oldest pending item

Outbound queue view

• Filterable table: downlink, status, age, size
• Actions: retry failed items, purge old sent items, hold/release

Areafix (Future Phase)

Areafix is a netmail robot that allows remote sysops to manage their own echo subscriptions by sending netmail to a well-known address (typically Areafix or Echofix at your node address).

Commands (standard)

| Command | Effect | |---|---| | +GENERAL | Subscribe to GENERAL | | -COOKING | Unsubscribe from COOKING | | %LIST | Reply with available areas | | %QUERY | Reply with currently subscribed areas | | %HELP | Reply with command reference | | %PAUSE | Pause all areas (hold mail) | | %RESUME | Resume all areas |

Design notes for current phase

• The `downlink_areas.paused` and `downlinks.hold_mail` columns support Areafix PAUSE/RESUME already
• When adding area subscriptions via admin UI, the same `downlink_areas` table is used ? Areafix will just be an automated writer to the same table
• The Areafix robot will be implemented as a `PacketProcessor` hook that intercepts netmails addressed to a configurable robot name and dispatches commands

Implementation Plan

Phase 1 ? Core infrastructure

1. Database migrations: `downlinks`, `downlink_areas`, `downlink_outbound`
2. `DownlinkFanout` class ? fanout logic with SEEN-BY/PATH handling
3. `EchomailSeenBy` helper class
4. Hook fanout into packet processor (inbound echomail) and local post path
5. Admin: downlinks CRUD and area subscription UI

Phase 2 ? Delivery

6. Binkp server: authenticate downlinks from `downlinks` table; serve `downlink_outbound`
7. `binkp_poll.php --downlink` / `--all-downlinks` flags
8. Scheduler integration for push delivery

Phase 3 ? Netmail routing

9. Netmail passthrough routing to downlinks

Phase 4 ? Areafix (separate proposal)

12. Areafix robot and netmail command parser
13. Reply packet generation
14. Per-downlink access controls (which areas they're allowed to request)

New Files

| File | Purpose | |---|---| | src/Downlink/DownlinkManager.php | CRUD for downlinks and area subscriptions | | src/Downlink/DownlinkFanout.php | Echomail fanout engine | | src/Downlink/DownlinkDelivery.php | Packet assembly and outbound queue management | | src/Echomail/EchomailSeenBy.php | SEEN-BY / PATH parse, check, update, format | | database/migrations/v1.x.x_downlinks.sql | | | database/migrations/v1.x.x_downlink_areas.sql | | | database/migrations/v1.x.x_downlink_outbound.sql | | | templates/admin/downlinks.twig | Admin downlinks list | | templates/admin/downlink_edit.twig | Add/edit downlink + area subscriptions | | routes/admin-downlink-routes.php | Admin routes for downlink management |

Modified Files

| File | Change | |---|---| | src/BinkP/PacketProcessor.php | Call DownlinkFanout after storing inbound echomail; netmail routing check | | scripts/binkp_server.php | Authenticate downlinks; serve downlink_outbound during pull sessions | | scripts/binkp_poll.php | Add --downlink / --all-downlinks flags for push delivery | | scripts/binkp_scheduler.php | Add downlink push schedules | | routes/admin-routes.php | Register admin downlink routes | | Local post path | Call DownlinkFanout after locally posted echomail is stored |

Decisions

1. Inet host for push: the `downlinks` table includes `inet_host` and `port` fields. If set, push delivery uses them directly. If not set, falls back to nodelist lookup by node address.
2. Packet format: Type-2+ (FSP-1010) for all downlink packets. No per-downlink option needed.
3. Queue retention: configurable, default 30 days. Sent rows older than the retention period are purged by a maintenance task.
4. Flood protection: deferred ? requirements unclear at this time.
5. Bounce netmail: no bounce. Mail for a held or disabled downlink sits in the queue until the hold is lifted or the retention period expires. This matches standard FTN behaviour.
6. Inbound echomail from downlinks: configurable per downlink via `allow_inbound_echomail` boolean on the `downlinks` table (default `true`). When `true`, echomail received from the downlink is tossed into the local echoareas as with any peer. When `false`, only netmail is accepted from them (pure downlink/hub-spoke mode).

Document Status: Draft Proposal Last Updated: 2026-03-14 Author: AI-Generated (Requires Review)