Watch for Change Streams


#1

MongoDb has a https://docs.mongodb.com/manual/changeStreams/ that let you watch changes of documents

In MongoDb I have a config collection that I watch to changes , In my nodejs application I cache configs and when ever receive changes I stale cache

I want to migrate to Couchbase , Is there similar solution for Couchbase and nodejs driver?
I have 100s million documents in my bucket but only 10s config documents , And I only want to watch do config documents changes


#2

In Couchbase 5.5, we released Couchbase Eventing [1]. Now you can write server side functions that get triggered when data changes. Couchbase Functions affords you OnUpdate and OnDelete handler(yes, you write Javascript!) which is much easy, as you just have to concentrate on business logic than infrastructure.

As you have just 10 documents that have to be watched(but we scale for millions and billions), Couchbase Functions would be a much better fit, as you do not have to maintain middleware code that keeps track of the stream. Your infrastructure would be much simpler.

Do take it for a spin and let us know what you think :slight_smile:

[1] https://blog.couchbase.com/eventing/


#3

@venkat Couchbase Eventing is a nice tool but I think It cannot help me

Consider the following pseudo code

const cache = {};
function getConfig(id){
    if(!cache[id]){
        cache[id] = bucket.get(id).result
    }
    return cache[id];     
}

When document is updated the cache is stale and I want to delete cache

on('Config changed',function(doc){
     delete cache[doc.id]
})

Assume I have a isMaintenance in my config and it cached as false When I admin panel I set it true I want the cached value in memory deleted and fetch it again


#4

Hi @socketman2016, are you sure you need the client-side cache in the first place? Couchbase is in part a very high performance key-value store (part of its history is memcached, after all). Perhaps your performance would be fine if you just do a KV get() on the doc each time you need it.


#5

I’m not sure , But it is obviously , If I cache configs (less than 30 docs) , my load on couchbase decreases by 50% , my chached configs not changed frequently , even If couchbase cannot notify me , I prefer create a redis pub/sub

In a simple benchmark I see that my nodejs application can send 30K hello world http response persecond
When I do a simple KV get() --tiny doc-- , it decreases to 24K/s , What you think?