NifFi Integration for a DCP Streaming Client

Exactly-once delivery is a hard problem. The Couchbase Kafka connector currently does not guarantee “exactly once” delivery due to a limitation of the Kafka Connect framework. Since you’re already consuming DCP streams directly*, perhaps you could experiment with Idempotent Kafka Producers, but that has limitations as well.

The best advice I can offer would be to architect the system in such as way that “at least once” is a sufficient guarantee. Which, to be fair, can also be a hard problem.

* Obligatory disclaimer: DCP is not officially supported.