A single event's payload can be arbitrarily large

Nobody has started this yet — be first.

Business impact

This service's entire memory-bounding guarantee (the README's "Backpressure, for real" section) rests on the assumption that every queued event record is roughly the same, small size. Right now EventIn.payload accepts a JSON object of any size at all, so a single client sending one event with a multi-megabyte payload can blow that assumption open — the per-event size limit the codebase already defines is currently not enforced.

Problem

EventIn.payload accepts any JSON object with no size check. app/schemas.py already defines MAX_PAYLOAD_BYTES = 16 * 1024 and a _enforce_payload_size field validator wired onto EventIn.payload, but the validator's body is a no-op stub that returns the value unchanged no matter how large it is.

Current behavior

EventIn._enforce_payload_size is a stub that returns value unchanged regardless of size; constructing an EventIn with a payload well over 16KB once serialized raises nothing at all.

Expected behavior

An event whose payload, once JSON-serialized, exceeds MAX_PAYLOAD_BYTES is rejected at the API boundary with a 422, before it ever reaches the queue; an empty payload is always accepted.

Steps to reproduce

cd fastapi/ingest_service && .venv/bin/python -c "from app.schemas import EventIn; EventIn(device_id='d1', event_type='t', payload={'blob': 'x' * 20000})"
Observe: constructs without error — should raise a ValidationError.

Why this matters

This whole service's headline design property is "memory is bounded by queue_max_size * bytes-per-record, full stop" — that arithmetic implicitly assumes each queued record is roughly the same, small size. A single client sending one event with a multi-megabyte payload blows that assumption open; the size bound has to be enforced per event, at the API boundary, for the queue-level bound to mean anything.

Suggested approach

Look at EventIn._enforce_payload_size in app/schemas.py — it's already registered as a @field_validator("payload"), it just needs a real body. You'll need some way to measure the serialized size of a dict (think about what payload becomes on the wire, and what standard library tool turns a dict into that representation so you can measure its length).

Acceptance criteria

A payload whose JSON-serialized form is <= MAX_PAYLOAD_BYTES is accepted unchanged
A payload whose JSON-serialized form exceeds MAX_PAYLOAD_BYTES raises a Pydantic validation error (surfaces as 422 from POST /events / POST /events/batch)
An empty payload ({}) is always accepted regardless of the limit

Verification

pytest practice_tickets/tests/test_ticket_06_payload_size_limit.py -v

Hints (0/2)

Try it without hints first — the reading is the exercise.

Working on this ticket

Work on a branch named for the ticket — that's what you'll submit.

Branch off your fork

$git checkout -b feat/payload-size-limit

Fix it and commit

Meet every acceptance criterion, and add a test that would have caught this.

Push the branch

$git push -u origin feat/payload-size-limit

Submit it below

Paste your fork URL and the branch name, with a short write-up of the root cause.

Submit your fix

Questions

Stuck on something?

Ask about anything unclear in the ticket — the maintainer and anyone who has solved it can answer. Please don't post full solutions.

No questions yet. If something in this ticket reads ambiguously, you are probably not the only one — ask.

The project

High-Throughput Event Ingestion Service

A backpressure-safe FastAPI ingestion API built to swallow ~50,000 events/sec without ever doing one INSERT per event or growing an unbounded buffer.

Fork on GitHub

Download

Setup

High-Throughput Event Ingestion Service

git clone <your-fork>
cd FastAPIDevTraining/ingest_service
docker compose up --build   # Postgres 16 + FastAPI (uvicorn --reload)

App reachable at ; Postgres at . Verify with (liveness only) and (checks a DB connection and that all flush workers are alive).