Asynchronously triggering a view update after inserting data

.NET SDK
1

I’m using the .NET SDK 2.1.
I have an application that is sometimes performing poorly because of queries issuing the stale=false option.
The Couchbase documentation is very clear, thanks for that.
But I would like to try working the other way around, triggering a view update after inserting data, instead of using a query parameter, as I always need up to date data. I have to try to confirm which option is best in my use case.
I’m storing documents with revision numbers and I always need the latest revision. Nothing is updated, nothing is deleted. I’m always inserting new data. The view is here to find all revisions of the documents. The client is selecting the one that it needs in the result set.

Can I trigger a view update after inserting a new document with the .NET SDK ?

2

@omatrot

Based on your explanation, I’m going to make the following assumptions about your use case. Please correct me if I misunderstand.

  1. You are inserting/updating data often
  2. You are only occasionally querying the data using a view
  3. Because there may have been a lot of updates since the last view request, using stale=false is taking too long to update the view data before returning the results

So, to the best of my knowledge (note: I’m a user, not an expert) there is no way to automatically trigger view generation when data is updated. This is done for performance reasons, since the MapReduce view generation can be an expensive process to repeat constantly.

As I see it, you have two options to address this concern.

  1. Automatically issue an async view request with stale=false after every update or on a regular interval. This will trigger the view to update.

  2. Assuming you don’t need the MapReduce function of views (i.e. summarization of data in advance of the query), using Couchbase 4.0 Global Secondary Indexes instead of views may be a better use case. Unlike views, indexes are kept up to date regularly instead of waiting for a query.

You can then choose to run the query with an almost-up-to-date state (the default) or force it to get the latest changes first. Forcing the latest changes would be kind of like stale=false, but would be potentially faster since the indexes will be closer to up-to-date when the query is started.

3

Thanks for your prompt answer.
I will sum up my use case. Points of interest are in bold.

  1. I am inserting data often. Data is never updated.
  2. I am querying the view often.
  3. Because there may have been a lot of inserts since the last view requests, using stale=false is taking too long.
  4. I am not using the map/reduce function of views.

The documentation says the following:

•Updates can be triggered in two ways:
◦At the point of access or query by using the stale parameter.
◦Automatically by Couchbase Server based on the number of updated documents, or the period since the last update. Automatic updates can be controlled either globally, or individually on each design document.

I’m afraid that “updated documents” do not include inserted documents, which would be bad news for me…
I also want to be sure that stale=false does nothing if everything is already up to date. based on my testing, it seems to be the case.

I have only 3 views in one design document, I will try to have 3 design documents with one view to see the performance improvements.

4

@omatrot

Ah, you are correct. There is a way to configure automatic view updates. But it looks like you have to do it in the JSON of the view design document. I found some details about it here:

http://developer.couchbase.com/documentation/server/4.0/indexes/mapreduce-view-operation.html

You should be able to trigger the updates on a shorter time interval or after fewer document changes.

updateInterval - How often it checks to see if an update should be performed, defaults to every 5 seconds.
updateMinChanges - How many documents must have been changed to trigger an update. This should include inserts and updates.

So, for example, you could set this to:

{
“updateInterval”:1000,
“updateMinChanges”:1,
}

And it will poll every 1000 milliseconds, and if there has been at least one update then trigger the update.

However, be careful with these settings. They could easily degrade your cluster performance if you get the numbers too low. I’d monitor your performance under load carefully.

1 Like