Real time events if flux finished after all elements are retrieved

Hello all guys,

I have a question related to Couchbase eventing. In Firebase/Firestore DB you can use a nice feature where you can be registered through the API to be notified when for example a new document is created or updated. I need to do the same thing with Couchbase but the Couchbase Event system is more like the old PLSQL or similar language to trigger some actions from and to the DB and executing some action according to that, but I want a more reactive/real-time approach.

Do you know if is it possible to do that with just Couchbase (something like a listener without curl function)?

PS1: Also, I tried to do a test with SpringDataCouchbase using a simple findAll. That operation returns a Flux but it’s a finite flux that finishes when all items are retrieved.
PS2: I found something similar in MongoDB (https://docs.mongodb.com/manual/core/tailable-cursors/), does Couchbase provide something similar?

Thx for your help.

Hello Hernán,

If you don’t want to use the Eventing service, my first recommendation would be to use the Couchbase Kafka connector to publish the database change events to a Kafka topic. Then you can subscribe to the topic and process the changes asynchronously.

If you don’t want to use Kafka either, you could use the Java Database Change Protocol (DCP) client. The client is not officially supported, but it’s a well-maintained library used by the official Couchbase Spark, Kafka, and Elasticsearch connectors.

There’s some example code in the repo. I’d probably start by looking at the High-Level API example. The DCP protocol itself is well-documented, but it’s not officially supported for external use.

Be warned that a heavily loaded multi-node Couchbase cluster can produce so many change events that a single DCP client process might not be able to keep up with all the changes. For a truly scalable solution, you might need to have several clients running on different machines, with each client handling a different subset of the Couchbase partitions.

Thanks,
David

Thanks for your advice @david.nault,
I just want to notify through Flux/ServerSentEvents in near realtime to the clients when a document is created/updated/deleted, so
I created a PoC using RabbitMQ that is similar what you propose with Kafka but at application level, but would be great if the DB provides that kind of listeners.

Regards,
Hernan.