Random err while upserting a document using .net SDK

.NET SDK
1

Hi,

Lately, one of our long running service started to throw randomly the following exception…
We are using Couchbase server version 6 and .net SDK v 2.7.12

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List1.Enumerator.MoveNextRare() at System.Collections.Generic.List1.Enumerator.MoveNext()
at System.Linq.Enumerable.Any[TSource](IEnumerable1 source) at Couchbase.Configuration.Server.Serialization.VBucketServerMap.EnsureIPEndPointsAreLoaded() at Couchbase.Core.VBucket.LocatePrimary() at Couchbase.Core.Buckets.CouchbaseRequestExecuter.GetServer(String key, UInt32 revision, IVBucket& vBucket) at Couchbase.Core.Buckets.CouchbaseRequestExecuter.SendWithRetry[T](IOperation1 operation)
at Couchbase.CouchbaseBucket.Upsert[T](String key, T value)

Any suggestion?

Thanks,

2

Hi @shaike.marc,

Would you mind posting the snippet of code that causes this exception? If you can provide a complete sample (including configuration) to reproduce the issue that would be even better.

Have you changed anything recently? Also, have you considered updating to the latest version of the SDK?

3

@matthew.groves

thanks for the quick reply…

The actual upsert and the configuration code seems to be pretty simple.
We recently changed the server version to be 6. It was 4.5 and the very same code worked OK.
If there is a chance of a fixed bug at the sdk code, I don’t mind upgrading it but while looking at the release notes of the most updated versions i didn’t see any reference to that issue, so…

// config - no special parameters except servers and auth
var cbConf = new cb.Configuration.Client.ClientConfiguration
                    {
                        Servers = new List<Uri>()
                    };

                    foreach (var server in servers)
                    {
                        cbConf.Servers.Add(new Uri(server));
                    }

                    _cluster = new cb.Cluster(cbConf);

// upsert

using (var b = OpenBucket(_usersBucketName))
            { 
....

        res = b.Upsert<dynamic>(cxt.EntityKey, entity);

}
4

Can you provide more detail about the OpenBucket method? (Possibly just paste the code)

5

sure, here it is (omitted try/catch/logging)

return _cluster.OpenBucket(bucketName);

Thanks

6

Hi,

Any update on that one? started to get this err on our dev env.
I suspect that it happens when one of the master is replaced and the sdk fails to deal with it.
Do i need to change the code, connect options or the way i open the bucket?

Thanks

7

@shaike.marc,

The code looks reasonable to me. I’m slightly suspicious of cxt, but otherwise the code looks fine.

Some suggestions that might help you suss out the problem:

  • Try running sdk-doctor to see if it identifies any connectivity/network issues
  • Try to enable tracing and look into open tracing, although this route seems unlikely to tell you much since you aren’t dealing with a timeout issue. I would say to check out the client and server logs in general (if you haven’t already)
  • Is your Couchbase cluster stable enough? Are nodes going down and being spun back up often? The error message indicates that something along these lines might be causing a problem.
  • Definitely try upgrading your .NET client and see if that has any effect.
8

@matthew.groves

Thanks. Will step along that path.

Shaike

9

This could be related to a topic I just started EnsureIPEndPointsAreLoaded() is not thread-safe