Adyrem
b25156a438
ci / verify (push) Successful in 45s
The real cause of the ship/bullet separation, which the previous commit only half-addressed. The server dropped inputs past a lead of 12 while the client only re-synced past 16, so a client whose lead drifted into 13-16 had every input silently rejected while believing its timing was fine. The server coasted on held_input and then stopped; the client kept predicting. The two separated permanently and the reconciler fought it every snapshot -- "shoved around". It needed two independent clocks to drift, hence "only after some time", and nothing in the loop could notice, hence "then persists". The listen-server diagnostic could never reproduce it: one process, one physics tick, lead constant by construction. Two defences: INPUT_MAX_LEAD (40) is now far wider than the client's correction band (3..20), asserted by tests/unit/test_input_lead.gd so narrowing it fails a test; and an ack-stall detector re-syncs when last_input_tick stops advancing, which catches the whole class regardless of cause -- lead alone cannot, because a wrong lead looks normal from the client. diag_prediction.gd now injects a +14 tick drift and exits non-zero unless the gap recovers. Also: - No invulnerability frames. Every bullet that touches a player lands; i-frames made dense patterns safer than sparse ones, which inverts the genre. Measured: a stationary player survives ~13.6s of the Warden's opening phase, ~17.5s drifting. spawn_grace remains the only invulnerable state. - Death is exited with a HUD button, disabled for the first 3s. The lockout is enforced in SimWorld, not just by graying the button -- a client that ignores its own UI still waits. The interact key no longer respawns. - Joining a server that is not there no longer drops the player into an empty lobby they cannot act in. Net.join() only creates an ENet object; the game scene now waits for the server to actually place us in an instance, with an 8s timeout, and headless runs exit non-zero instead of idling. Protocol 2 -> 3. 98 tests; check.sh, test.sh and smoke.sh all pass.
257 lines
13 KiB
Markdown
257 lines
13 KiB
Markdown
# Netcode
|
||
|
||
## Model
|
||
|
||
Authoritative dedicated server, clients send input only, with client-side
|
||
prediction and reconciliation for the local player.
|
||
|
||
```
|
||
client server (60 Hz)
|
||
| |
|
||
|-- InputFrame (60 Hz, x3) ---->| queue_input(): validate, drop, sort
|
||
| move, aim, 3 button bits | SimWorld.step(): move, AI, emitters,
|
||
| | integrate bullets, resolve hits
|
||
|<-- events (reliable, ch 3) ---| bullet spawns/despawns, hits, deaths
|
||
|<-- snapshot (unrel., ch 2) ---| players, enemies, boss @ 20 Hz
|
||
| |
|
||
predict locally, reconcile authoritative for everything
|
||
```
|
||
|
||
The client has no message that expresses a position, a hit, damage taken, or a
|
||
completed escape. This is deliberate and stronger than validating such messages
|
||
after the fact: there is no code path to exploit, only intent to interpret.
|
||
|
||
## Why bullets are not replicated as state
|
||
|
||
A bullet hell has hundreds of live bullets — this project peaks at ~350 with one
|
||
boss and four players. Sending a position per bullet per snapshot would be
|
||
roughly 350 × 8 bytes × 20 Hz ≈ 56 KB/s per client and would grow with the
|
||
pattern density, which is the one thing you want freedom to increase.
|
||
|
||
Instead the server sends **one spawn event per bullet, once**: uid, position,
|
||
velocity, radius, lifetime, kind, team, acceleration, turn rate (~36 bytes).
|
||
Both sides then run the identical integration in `BulletPool.step()`. A bullet
|
||
that expires or leaves the arena dies on both sides with no traffic at all.
|
||
|
||
The only bullets that need a despawn message are the ones killed early by a hit,
|
||
because that is the one outcome a client cannot derive.
|
||
|
||
Cost at the same peak: a few KB/s, dominated by whatever the patterns are
|
||
actually firing rather than by how much is in the air.
|
||
|
||
`tests/integration/test_replica_parity.gd` is the test that keeps this honest.
|
||
It runs the server and a replica side by side for 900 ticks of a boss fight —
|
||
once in-process, once with every event encoded and decoded through the real wire
|
||
format — and asserts that every live bullet matches within 0.01 px (1 px through
|
||
float32 on the wire). If that test fails, the bullets on screen are no longer
|
||
the bullets that can kill you.
|
||
|
||
Note what this does **not** require: lockstep determinism. Clients simulate
|
||
bullets for display only. The server never reads a client's bullet state, so
|
||
float drift between machines is a cosmetic concern, not a correctness one.
|
||
|
||
## Hit validation
|
||
|
||
All in `SimWorld._resolve_bullet_hits()` and `_resolve_contact_damage()`, server
|
||
only, circle-vs-circle:
|
||
|
||
- **Enemy bullet → player:** skipped entirely while the player has i-frames, so
|
||
a dense pattern costs one hit, not thirty. The bullet is consumed and a
|
||
`BULLET_DESPAWN` goes out.
|
||
- **Player bullet → boss, then enemies:** first overlap consumes the bullet.
|
||
Boss damage is scaled by the current phase's `damage_taken_mult`.
|
||
- **Enemy body → player:** contact damage, same i-frame gate.
|
||
|
||
There is no lag compensation and no rewind. For a bullet hell that is the right
|
||
call: what the player is dodging is their own position against a bullet field,
|
||
and rewinding the world to a shooter's view would mean a player who dodged on
|
||
their screen still gets hit. The cost is that a high-ping player's shots at a
|
||
moving enemy land where the enemy *was* on the server. Enemies are large and
|
||
slow relative to bullet speed, so this reads as latency rather than unfairness.
|
||
If it becomes a complaint, the fix is to lag-compensate **player bullets against
|
||
enemies only**, never enemy bullets against players.
|
||
|
||
## Input timing, and a bug worth remembering
|
||
|
||
The client numbers its input frames `INPUT_TARGET_LEAD` ticks ahead of the
|
||
server's tick; the server accepts anything within `INPUT_MAX_LEAD` and re-syncs
|
||
nothing itself. **These two numbers must be chosen together.**
|
||
|
||
They originally were not — the server dropped inputs past a lead of 12, while
|
||
the client only corrected itself past 16. A client whose lead drifted into 13–16
|
||
therefore had *every* input silently rejected, while believing its own timing
|
||
was fine. The server, receiving nothing, coasted on the last input it had
|
||
(`held_input`) and then stopped; the client kept predicting forward. The ship
|
||
and the authoritative position separated permanently, the reconciler fought it
|
||
every snapshot, and the player got shoved around. It needed two independent
|
||
clocks to drift apart, so it appeared only after minutes of play — and never
|
||
recovered, because nothing in the loop could notice.
|
||
|
||
Two defences now:
|
||
|
||
1. `INPUT_MAX_LEAD` (40) is far wider than the client's correction band
|
||
(`INPUT_LEAD_MIN` 3 … `INPUT_LEAD_MAX` 20), so the client always re-syncs
|
||
long before the server starts refusing anything. `test_input_lead.gd`
|
||
asserts that ordering directly, so narrowing the window fails a test rather
|
||
than shipping.
|
||
2. An **ack-stall detector**: if `last_input_tick` does not advance across
|
||
`INPUT_ACK_STALL_LIMIT` snapshots, the client concludes its numbering is
|
||
outside the window and hard re-syncs. Lead alone cannot detect this, because
|
||
a wrong lead looks perfectly normal from the client's side. This is the
|
||
backstop that makes the whole class of failure self-healing regardless of
|
||
cause.
|
||
|
||
`tools/diag_prediction.gd` injects a +14 tick drift mid-run and asserts the gap
|
||
returns to normal; it exits non-zero if it does not.
|
||
|
||
## No invulnerability frames
|
||
|
||
A hit grants no immunity — every bullet that touches you deals its damage. In a
|
||
bullet hell the wall *is* the threat, and i-frames invert that: you are punished
|
||
for the first bullet of a pattern and gifted the next thirty, which makes dense
|
||
patterns *safer* than sparse ones. Measured cost: a stationary player survives
|
||
~13.6s of the Warden's opening phase, ~17.5s while drifting aimlessly.
|
||
|
||
Arrival protection (`SPAWN_GRACE_TICKS`) is the sole exception, and it is a
|
||
transition, not a combat mechanic.
|
||
|
||
## Input validation
|
||
|
||
`SimWorld.queue_input()` is the single audit point. It drops:
|
||
|
||
| Case | Guard |
|
||
| --- | --- |
|
||
| Replay of a consumed tick | `f.tick <= p.last_input_tick` |
|
||
| Stale input | `f.tick < tick - INPUT_MAX_AGE` |
|
||
| Input claiming the future | `f.tick > tick + INPUT_MAX_LEAD` |
|
||
| Flood | queue capped at `INPUT_MAX_AGE`, oldest dropped |
|
||
| Leaving the hub early after death | `RESPAWN_LOCKOUT_TICKS`, server-side |
|
||
|
||
The respawn lockout is worth calling out: the HUD disables its button for the
|
||
same three seconds, but that is presentation. A client that ignores its own UI
|
||
and holds the bit down still waits, because the check lives in `SimWorld`.
|
||
|
||
Beyond that, the wire format itself constrains the cheat surface: the move
|
||
vector is two signed bytes at 1/100 precision, and `Movement.sanitize_move()`
|
||
clamps to the unit disc, so no packet can express extra speed. Fire rate is a
|
||
server-side cooldown; holding the button every tick fires at exactly the same
|
||
rate as tapping it correctly.
|
||
|
||
A client that stops sending coasts on its last input for `INPUT_MAX_AGE` ticks
|
||
and then stops, so a dropped connection does not leave a player sliding.
|
||
|
||
## Leaving a run, and why disconnecting is not an escape
|
||
|
||
The emergency escape is a **one-second server-owned channel** that damage does
|
||
*not* interrupt. That combination is deliberate and the two halves depend on
|
||
each other.
|
||
|
||
An interruptible channel sounds like the right kind of risk until you follow it
|
||
through: a player about to die under fire can never finish the channel, so
|
||
killing the game process becomes strictly better than using the button. The
|
||
escape hatch turns into the exploit.
|
||
|
||
So a dropped connection runs the same channel. On disconnect the peer is removed
|
||
from the send list but its player **stays in the world** (`SimPlayer.linkdead`),
|
||
treated as holding the escape button down, still fully killable, for the same
|
||
second everyone else pays. Only when that resolves is the peer forgotten. Pulling
|
||
the plug is therefore never cheaper than pressing the key, and there is no
|
||
timing window where it is.
|
||
|
||
Two details that are easy to get wrong and are covered by tests:
|
||
|
||
- An instance must not close while a linkdead body is still resolving
|
||
(`Instance.has_linkdead()`), or a solo player's drop would delete their body
|
||
on the next tick and hand back the exact exploit.
|
||
- Dying still ends the channel. That is the one thing the escape cannot beat,
|
||
and it is what keeps the second a real risk rather than a formality.
|
||
|
||
`tools/smoke.sh` SIGKILLs a bot mid-dungeon and asserts the server channels it
|
||
out rather than dropping it instantly.
|
||
|
||
## Arriving in a run
|
||
|
||
Entering a dungeon grants `SimConfig.SPAWN_GRACE_TICKS` (2s) of **arrival
|
||
protection**: invulnerable *and* unable to shoot. Both halves matter —
|
||
invulnerability alone would make the spawn point a free firing position. It is
|
||
flagged in the snapshot (`Protocol.F_SPAWN_GRACE`) so every client can draw it
|
||
on every protected ship, not just its own.
|
||
|
||
## Prediction and reconciliation
|
||
|
||
`ClientRuntime` keeps every unacknowledged `InputFrame`. Each snapshot echoes
|
||
`last_input_tick` per player; on receipt the client discards acknowledged
|
||
frames, takes the server's authoritative position, and replays the rest through
|
||
the same `Movement.step_player()` the server used. Divergence over 24 px snaps;
|
||
smaller errors lerp at 0.3 so ordinary jitter does not read as rubber-banding.
|
||
|
||
`ClientRuntime` sets `process_physics_priority = -10` so it samples and sends
|
||
input *before* `ServerRuntime` ticks. This matters only on a listen server,
|
||
where both live in one process: with the default tree order the server ran
|
||
first, so input sampled on frame N was not consumed until frame N+1, leaving the
|
||
drawn ship a permanent one tick (4px at 240 u/s) ahead of the authoritative one
|
||
— and bullets, which spawn at the authoritative position, visibly trailed behind
|
||
the ship. `tools/diag_prediction.gd` measures this gap; it is currently ~0.1px
|
||
mean, down from a constant 4.00px.
|
||
|
||
The other half of that fix is `SimConfig.PLAYER_MUZZLE_OFFSET`, derived from the
|
||
*visual* radius rather than the hitbox. A remote client still draws its ship a
|
||
tick or two ahead of the server, and that margin is what keeps bullets emerging
|
||
from the nose rather than the middle of the sprite.
|
||
|
||
Remote players, enemies and the boss are interpolated between the last two
|
||
snapshots. Bullets are not interpolated — they are simulated, and a spawn event
|
||
carries the server tick it was generated at so the client fast-forwards the
|
||
bullet by the packet's flight time (`BulletPool.advance_slot()`) instead of
|
||
popping it in at the muzzle a round-trip late.
|
||
|
||
## Channels
|
||
|
||
ENet channels keep a burst of reliable bullet events from head-of-line blocking
|
||
the snapshot stream:
|
||
|
||
| Channel | Traffic | Mode |
|
||
| --- | --- | --- |
|
||
| 1 | handshake, instance transitions | reliable |
|
||
| 2 | snapshots | unreliable, newest wins |
|
||
| 3 | bullet spawns/despawns, hits | reliable ordered |
|
||
| 4 | client input | unreliable ordered |
|
||
|
||
`Protocol.VERSION` is checked at handshake and mismatched clients are rejected
|
||
with a reason, rather than desyncing later.
|
||
|
||
## Instances
|
||
|
||
One server process hosts the lobby hub plus every concurrent dungeon. An
|
||
`Instance` is a `SimWorld` and a peer list; messages are addressed to the peers
|
||
of one instance, so a player never receives traffic for a world they are not in.
|
||
At ~0.24 ms/tick per instance, the limit is bandwidth and player count long
|
||
before it is CPU.
|
||
|
||
Transfers (portal in, escape out, dungeon cleared) are driven by server-only
|
||
events that `NetCodec` refuses to serialise, so the decision cannot leak to a
|
||
client as something it might try to send back.
|
||
|
||
## The listen server
|
||
|
||
"Host and play" runs a `ServerRuntime` and a `ClientRuntime` in one process. The
|
||
client is peer 1 and goes through the identical code path — it still sends input
|
||
and still learns outcomes from snapshots. Only the transport is short-circuited,
|
||
in `Net.send_*` / `Net.send_input`, which check for the local peer and call the
|
||
handler directly instead of going through ENet. No simulation code knows the
|
||
difference, so a bug cannot hide in a "host-only" branch.
|
||
|
||
## Known gaps
|
||
|
||
- No lag compensation (see above — deliberate, with a stated escape hatch).
|
||
- No snapshot delta compression. Full state at 20 Hz is fine at this actor
|
||
count; delta-encoding against the last acknowledged snapshot is the next step
|
||
if enemy counts grow.
|
||
- No encryption or authentication. `ENetMultiplayerPeer` supports DTLS; wire it
|
||
up before exposing a server to the internet.
|
||
- Bullet spawn events are per-bullet. Sending *pattern* events instead (emitter
|
||
id + tick + seed, clients regenerate) would cut bullet traffic by roughly the
|
||
count of each emitter's shot. Worth doing only if bandwidth becomes the
|
||
binding constraint — it couples the client to emitter behaviour, which the
|
||
current design deliberately avoids.
|