Silent failure of upsert in couchbase .net client?


#1

I have now seen several times, a problem with the latest 2.1.3 couchbase .net client failing silently on upsert, against a single node Couchbase 4 node.

I have my own Writer class which I wrote, which I create like this:

var writer = new CouchbaseExportWriter("http://hostname:8091/pools"); 
writer.openBucket("default");

The complete code of the class is shown here, including how I am invoking upsert:

On a COMPLETELY EMPTY default bucket, where NO DOCUMENTS YET EXIST, I get silent failures on my first upserts. I do 8000 upserts and a row, and they all quickly, silently fail. This is very troubling.

After I have created some documents in my empty single node cluster, in the default bucket, the C# client upsert method now succeeds.

What is going on?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using Newtonsoft.Json;

using Couchbase;
using Couchbase.Core;
using Couchbase.Configuration.Client;
using Couchbase.N1QL;

//using Couchbase.IO;
//using Couchbase.Management;
//using Couchbase.N1QL;
//using Couchbase.Views;
//using Couchbase.Utils;

using JsonDictionary = System.Collections.Generic.Dictionary<string, object>;
using JsonTableDictionary = System.Collections.Generic.Dictionary<string, System.Collections.Generic.Dictionary<string, object>>;
using JsonDictionaryItem = System.Collections.Generic.KeyValuePair<string, System.Collections.Generic.Dictionary<string, object>>;


namespace FirebirdTest1
{
    class CouchbaseExportWriter
    {
        private ClientConfiguration _config;
        private Cluster _cluster;
        private IBucket _bucket;

        public ClientConfiguration config { get; set; }
        public Cluster cluster { get; set; }
        public IBucket bucket { get; set; }

        // serverurl = "http://hostname:8091/pools" 
        public CouchbaseExportWriter( string serverUrl)
        {
            _config = new ClientConfiguration();
            _config.Servers.Add(new Uri(serverUrl));
            _cluster = new Cluster(_config);


        }
        public int getBucketItemCount()
        {
            if (_bucket != null)
            {
                string qryText = "SELECT COUNT(*) FROM `" + _bucket.Name + "`";
                var queryRequest = new QueryRequest(qryText);
                var result = _bucket.Query<dynamic>(queryRequest);
                var row = result.Rows[0];
                return row["$1"]; // Get value for default $1 result
            }
            else
            {
                return 0;
            }

        }

        public void openBucket(string bucketName)
        {
            _bucket = _cluster.OpenBucket(bucketName);
            
        }

        public void upsert( Couchbase.Document<dynamic> document)
        {
            _bucket.Upsert(document);
        }

        public void upsert(string id, JsonDictionary documentDictionary )
        {
            var couchDoc = new Couchbase.Document<dynamic> 
            { Id = id,
              Content = documentDictionary
            };

            _bucket.Upsert(couchDoc);
        }

        public void delete(string id)
        {
            bucket.Remove(id);
        }
           
    }
}

#2

@Warren_Postma -

The Upsert method returns an OperationResult object which contains properties Status, Message and Exception which will give you insight into why the operation failed or succeeded. The Exception field will contain an exception and stacktrace if an exception was thrown and caught internally.

Additionally, your getBucketItemCount method using the N1QL query may be giving you stale results because indexing is “eventual consistent” unless you use a stronger scan consistency (at the cost of performance) such as REQUEST_PLUS. You can confirm this by checking the count within the Admin Console to see if indeed all 8k records were inserted. This is something that is still being worked on for N1QL GA; expect better documentation and examples of how scan consistency impacts query results as Couchbase Server 4.0 gets closer to GA.

-Jeff


#3

Oh I see. Since I expect failures NOT to be silent in my own style, I would simply add something that checks that the Status is not Success, and raise an exception myself from my Writer class.


#4

@Warren_Postma -

Yes, the rational behind this is that the SDK’s are fairly fault tolerant and will try to fix themselves if an error or exception occurs. The exception is when the SDK cannot bootstrap; in this case it has no recourse but to fail hard and throw an Exception.

-Jeff