Document stored by not exist in View

Hi all,

I have very strange situation.

I’m creating and storing document to couchbase bucket via ExecuteStore() method (.Net SDK).
in result Success = true and no errors. When I getting documents throught View new document doesn’t exist in result. So, I’m opening CouchbaseConsole and try finding out document via Document ID - but no result, no document. Next I just click on Lookup Id button and see(!) the document. Why it’s happens?

The scenario you are describing happens quite often when first learning about Map/Reduce. I recommend checking out the webinars on Map/Reduce Views at www.couchbase.com/webinars so you understand it better. Also, understanding the Couchbase architecture itself can help as well, there are many presentations on this as well.

First, please make sure that the document doesn’t have an expiration that has expired already when querying your View! This happens quite often to people who are new. They set an expiration of 5 seconds, but then forget they did that, so they are removed from their View index by the time they query! Now, the Couchbase admin interface data explorer might still show it! because it is not intended to be updated as frequently as the production data/views nor compete with actual Views you author (meaning resources are limited for it). It can occasionally show stale data, instead it’s better to use GET operations to see data existence.

Ok, so if that’s not the case, let’s go into an explanation, the Indexers (Design Documents equate to Indexers), index documents in batches. The documents they index are the documents that are on disk therefore there can be a time gap between your ExecuteStore() and getting it on disk, if you query within that time gap and the Indexer hasn’t run, it won’t show in the result set.

What triggers the Indexer to index the new data that has been created or mutated?

There are three triggers: time, mutation count, and user initiated. You can configure the first two, and the default is 5 seconds, 5000 set/replace/add operations. User initiated is by using the stale=false or stale=update_after view query parameters. See: http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-views-operation-autoupdate.html for configuration of defaults.

stale=false actually will wait for the Indexers to finish indexing new data since it’s last run before returning query results, which means there can be increased result latency. stale=update_after will first return the query results and then trigger the Indexer to run to update the Index with new data since the last run. stale=true basically will only return the current result set but not trigger the Indexers manually.

It is still possible to do an ExecuteStore(), immediately query the View with stale=false and still not see the result in the query results if the time gap between the ExecuteStore() and View Query is less than the time required to get the document on disk.

One technique would be to use the observe parameter on ExecuteStore() so that you get a callback when it has gotten on disk, and query the View after with stale=false. If that is not practical, you may need to think about using key-value patterns for immediate consistency if that is needed.

Do your documents have an expiration? please double check that they don’t. If they are expiring, they won’t show up. Restarting the server is really unnecessary, that won’t change anything. Make sure you are using a Couchbase bucket, not a memcached bucket. Especially if you are restarting the server, memcached bucket documents are not persisted to disk nor can you run Map/Reduce Views on it.

scalabl3, thanks, I know about all you said and understand how is couchbase work.
anyway I do not understand why saved document not saved to disk during the day, after restart server all saved documents that did not exist in view result are missing (obvious why).

Documents definitely without expiration; it’s implemented project which works approximately year and no problem before like now.
Yeah, it’s not memcached bucket, and restarting server helped resolve my problem.
When this situation happens again I’ll investigate problem more carefully.