Ancestors

Written by Jepsen on 2024-11-12 at 14:15

A new #Jepsen report! We worked with Buf to analyze the safety of Bufstream, a Kafka-compatible streaming system. We found three safety and two liveness issues in Bufstream 0.1.0, including the loss of acknowledged writes in healthy clusters. These problems were resolved by version 0.1.3.

https://jepsen.io/analyses/bufstream-0.1.0

=> More informations about this toot | More toots from jepsen@jepsen.io

Written by Jepsen on 2024-11-12 at 14:23

One of the surprising things we found during this collaboration was that the #Kafka transaction protocol assumes messages are delivered in order, but allows those messages to be delivered to different nodes over different TCP connections. This means that a client's commit or abort message could be applied to a completely different transaction, or even in the middle of a transaction.

=> More informations about this toot | More toots from jepsen@jepsen.io

Toot

Written by Jepsen on 2024-11-12 at 14:26

We reproduced lost writes, aborted reads, and torn transactions in Bufstream and Kafka. We believe every Kafka-compatible system is likely vulnerable to these issues. Some clients may be able to mitigate them, but the official Java client presently does not.

=> More informations about this toot | More toots from jepsen@jepsen.io

Descendants

Written by Jepsen on 2024-11-13 at 16:59

Thanks to everyone who wrote in objecting to the report's description of data loss due to auto-commit. Some experiments this morning suggest that we got it wrong (at least for the official Java client). We've published an update to the report: https://jepsen.io/analyses/bufstream-0.1.0#updates

=> More informations about this toot | More toots from jepsen@jepsen.io