Reliable log ingestion: queues, retries, and duplicates
Define delivery guarantees, bound your buffers, and make duplicate records and pipeline failures visible.
TOPIC COLLECTION
Log ingestion
Follow an event through buffers, retries, acknowledgments, and durable storage. Define measurable limits and test duplicate delivery, overload, replay, and shutdown rather than assuming the happy path is sufficient.
Read it. Then test it.
Use this collection to choose a focused design question. Each guide includes a practical investigation or fixture exercise, a primary technical reference, and links to the next relevant pattern. Start with the event you need to understand, then validate the full path from producer to reader.