Repository navigation
Inbound and outbound packet-size checks differ, so a QoS 1 message just under MaxUserPayloadBytes is accepted but never delivered (blocks a persistent subscriber) #313
Description
Version: 4.0.0-incubating (standalone, default DevOnlySettingProvider)
Summary
MaxUserPayloadBytes (stored as TenantSettings.maxPacketSize) is enforced differently in the two directions:
- Inbound (publisher → broker): the replacement
MqttDecoder(maxPacketSize)compares the fixed header's remaining length R against the limit. A packet is accepted while R ≤ L. - Outbound (broker → subscriber):
MQTTPacketFilterdrops a packet whose full encoded size1 + varint(R) + Ris over the limit (MQTT3MessageSizer), reportingOversizePacketDroppedand never completing the write.
So a QoS 1 PUBLISH whose remaining length is within 3 bytes of the limit (R = L−2 … L for L = 16384) is accepted and PUBACKed from the publisher, stored in the subscriber's inbox, and then dropped on every delivery attempt. writeConfirmableSubMessage keeps it in the unconfirmed set and resend() retries it until MaxResendTimes. With a large MaxResendTimes, a persistent subscriber's in-flight window fills with undeliverable messages and no further QoS 1 is delivered to it until the session is reset.
For MQTT 5 subscribers this cannot be worked around from the client: MQTT5ConnectHandler rejects a client Maximum Packet Size above maxPacketSize (CONNACK 0x83), and MQTTPacketFilter uses min(clientMax, settings.maxPacketSize).
Reproduction
- Start the broker with
-DMaxUserPayloadBytes=16384(and a large-DMaxResendTimes). - Subscriber: MQTT 3.1.1,
cleanSession=false, QoS 1, manual acks. - Publisher (same tenant): QoS 1 publishes with remaining length 16382, 16383, 16384 (payload = R − 2 − topicLength − 2), followed by a small normal message.
- The publisher gets PUBACKs for all of them; the subscriber receives none (not even the normal message once ~8 undeliverable messages occupy the window).
Expected
Both directions use the same measure, e.g. the inbound decoder limit is maxPacketSize − 5 (the maximum fixed-header overhead), or the outbound filter compares the remaining length, so anything accepted inbound is deliverable to a subscriber with the same limit. Independently, MQTTSessionHandler.writeConfirmableSubMessage already drops-and-confirms a confirmable message for NoSubPermission, Duplicated and NoLocal; doing the same when the encoded size exceeds the subscriber's limit (report a drop event, confirm(packetId, false)) would stop an undeliverable message from occupying the in-flight window until MaxResendTimes.
Checked against main (2026-10-02, 1394131): still present. #279 ("Fix oversized packet mem leakage", merged 2026-08-31, not yet released) makes MQTTPacketFilter release the buffer and complete the write promise, but the message stays unconfirmed in the session, and the inbound MqttDecoder(maxUserPayloadSize) in ChannelAttrs.setMaxPayload still compares the remaining length.
Related: #258 (resend cancelled by PUBACK), #289 (resend chain stops after a MaxRetried drop), #279 (oversized-packet buffer leak).
Activity
abhiramaab commentedon Oct 2, 2026
Hi @DrEjc04 / maintainers, I would like to work on this issue.
I am looking into aligning the inbound payload check with the outbound packet sizer and ensuring dropped oversize confirmable messages are properly confirmed/cleared from the in-flight window in
MQTTSessionHandler.popduke commentedon Oct 8, 2026
@DrEjc04 Thanks for the report. We confirmed the mismatch, however you can work around it by raising the broker’s packet-size limit enough to cover the protocol overhead. For MQTT 5, make sure the subscriber’s Maximum Packet Size is large enough too, then reconnect. With that workaround available, we’ll keep this at a lower priority for now.
DrEjc04 commentedon Oct 8, 2026
Thanks for confirming. One note on the workaround: as far as we can tell,
MaxUserPayloadBytessets both the inbound decoder limit and the outbound filter limit, so raising it moves the gap rather than closing it. A publisher can still send a remaining length within 4 bytes of the new limit, and it's accepted but never delivered. That makes the workaround work only when every publisher is trusted to stay below the limit minus the fixed-header overhead, which doesn't hold for us (untrusted IoT devices). Is there a way to set the inbound limit separately from the outbound one?Until a fix lands we run a local patch that drops and confirms an outbound confirmable message larger than the subscriber's limit (as in #314). Could you say which event or messaging semantics that approach breaks, so we can check our patch against them?
popduke commentedon Oct 10, 2026
No. MaxUserPayloadBytes is intended as a protective ceiling shared by inbound and outbound MQTT packets. Your service should publish its own application message limit that leaves enough room for MQTT headers within that ceiling. That is the purpose of the workaround.
Out of curiosity, could you share more about your use case? Is there a reason your service would need to accept messages from untrusted devices that exceed that application message limit?
For your patch, just make sure confirming an oversized message early doesn’t break the delivery semantics in the MQTT spec.
DrEjc04 commentedon Oct 12, 2026
Thanks for the explanation. On the use case: the publishers are bike-locker controllers (ESP32) in public places. The firmware is ours and enforces our application limit, and our backend subscribes with a persistent QoS 1 session that serves every locker. "Untrusted" here doesn't mean hostile by design. It means we can't assume every device in the field always behaves: a firmware bug or a tampered device can send a packet in that last band, between the application limit and the ceiling. Today such a packet is accepted at ingress and then sits in the subscriber's in-flight window forever, which stalls delivery from all the other lockers. So one bad device becomes a fleet-wide outage, and nothing downstream can reject a message it never receives.
On the semantics: for MQTT 5, §3.1.2.11.4 (Maximum Packet Size) says "Where a Packet is too large to send, the Server MUST discard it without sending it and then behave as if it had completed sending that Application Message" [MQTT-3.1.2-25]. Our patch does exactly that for QoS 1/2: it discards the oversized outbound message and completes it as if it had been sent, so the in-flight window moves on. As far as we can see, that is the behaviour the spec requires rather than a deviation from it. The current behaviour, keeping it in flight and never sending it, is what blocks the session. For MQTT 3.1.1 there's no client-announced maximum, but the same reasoning applies to the broker's own outbound limit. Happy to turn the patch into a PR (#314 goes in that direction) if you'd accept the fix on that basis.
popduke commentedon Oct 13, 2026
Thanks for clarifying the use case. The fix is already in main via #315. This issue overlaps with two other known issues, so the fix needed coordinated changes across several files. Please try it and let us know if you still see the issue.
DrEjc04 commentedon Oct 13, 2026
Thanks, I tried #315. I built
mainat f19c557 and ran the scenario from this issue against it and against v4.0.0-incubating:MaxUserPayloadBytes=16384, 12 QoS 1 publishes with remaining length 16383, then 6 normal ones, subscriber acking normally, MQTT 3.1.1 and 5, persistent and transient, QoS 1 and 2.So #313 is fixed for us. One note for others: a device now sees a closed connection rather than a PUBACK for these packets, which is fine, but worth a line in the release notes. We'll drop our patch when this lands in a release.