Hello
Is there an ability in the java sdk or somewhere else in couchbase API to monitor changes in a document data? Even and event that there was a change will be a good start
Thanks
There is not directly at the moment, no. We are looking at this though.
It’d be hugely useful to @tyler.mitchell and I if you could describe how you’d use this.
For instance, is it from a web app? Would you watch a set of documents based on the key? Or would you specify it based on attributes within the docs? Would best effort delivery be sufficient, or do you need an at-least-once guarantee?
The actual solution we are thinking to build will be more part of the cluster and scale well, but we do have one experiment that could be modified to do this.
We are also looking for this kind of solution where we would want an at-least-once guarantee. So that we can combine several documents and post to other database. We need it for more of an Integration point.
In your case @pankaj.sharma, do you need to filter on any attributes? And when you say combine several documents, does that mean you’ll want to receive an update on a document and then fetch other documents to further denormalize before putting it into an external system?
The Kafka Connector may be useful for your case today, depending.
Thanks in advance for any info to help us refine what we’re looking to build!
All the documents will be changed on the mobile. And when they send this to sync gateway, I want to get that change, And do multi-get to get all other documents which should be combined. After combining the documents I will give it to a web service which will call other services to put data in legacy databases. From the legacy databases selected data with calculations will be taken and put into a cluster of couch base and made available for other applications.
I think either a Kafka connecter will be good or a Daemon which can watch and process. But then this Daemon need to be a client of the sync gateway. And it can communicate using web socket connection and feeds base on last_seq number, so that no data fails to reach this Daemon.
I think that the basic use case is to monitor a document data changes, even a boolean to indicate that the document has changed. In case that a document is marked as changed, I can compare it with for example my local cache (or something like it) and notify my consumers.
Thanks for the additional info. When you say “consumers” do you mean something like web browsers connected to an application server or more like service integration, meaning there are other components to your architecture that need updates on a message queue kind of system?
Any more concrete details you can share are hugely useful since it impacts the tradeoffs we’ll have to address in design. Thanks!
Hi @ingenthr,
Sorry to revive an old thread. Was this feature ever developed, as I’m interested in this as well? Use case would be more service integration related - having a concrete way to inform data consumers that certain data has now been updated, and that they should get new data, invalidate their caches, etc. Ideally this would be modeled as an event. We would use a pub-sub model for data consumers to react accordingly.
Many thanks,
C