System.InvalidOperationException exception in 2.6 while waiting on GetDocumentsAsync<...>(keys);


#1

I’m trying to track down an issue that has appeared since I updated to the latest version of CouchBase 5.5, and the .NET client 2.6.

var task = bucket.GetDocumentsAsync(keys);
task.Wait();

{System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List1.Enumerator.MoveNextRare() at Couchbase.Tracing.SpanSummary.PopulateSummary(IEnumerable1 spans)
at Couchbase.Tracing.SpanSummary…ctor(Span span)
at Couchbase.Tracing.ThresholdLoggingTracer.ReportSpan(Span span)
at Couchbase.Tracing.Span.Finish()
at Couchbase.Core.Buckets.CouchbaseRequestExecuter.d__121.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Couchbase.CouchbaseBucket.<GetDocumentAsync>d__761.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Couchbase.CouchbaseBucket.d__76`1.MoveNext()}

I’m in the process of rolling back the SDK back to 2.5.12 to verify it’s specific to the update, but wanted to see if anyone else has come across this.

Edit 1: Rolling back to 2.5.12 resolves the issue.


.NET SDK Performance
#2

I was able to eliminate the issue by disabling tracing in .NET 2.6 SDK:

var configuration = new ClientConfiguration(configurationSection);
configuration.OperationTracingEnabled = false;
Cluster cluster = new Cluster(configuration);


#3

@xulane -

I created a Jira ticket for tracking the bug: https://issues.couchbase.com/browse/NCBC-1743

-Jeff


#4

Hi @xulane - I haven’t been able to reproduce, even with a very similar test here. FYI, your code example doesn’t compile because GetDocumentsAsync requires the target type.

However, upon reflection, a guess at the root cause could be task synchronisation where the task awaiters are not setup correctly on the sub tasks. I’ll work on creating a fix for GetDocumentsAsync that sets the task awaiter for each inner get task.

You can bypass the GetDocumentsAsync helper method by creating the get tasks yourself. Something like this should work:

var result = Task.WhenAll(keys.Select(key =>
{
    var task = bucket.GetDocumentAsync<object>(key);
    task.ConfigureAwait(false);
    return task;
}));

I am curious why you’re using the Async method in a synchronous environment, are you using it simply for the mult-get utility or something else?


#5

After looking further into it, I can see PopulateSpan takes an IEnumberable which is being enumerated directly. I’ve created a fix for this here.

I still think the task awaiters need to be reviewed, which I’ll look into too.


#6

Yes, it was mainly for the simplified multi-get.


#7

Hi @xulane

The example code I had above will do a good job of requesting multiple documents efficiently without relying on GetDocumentsAsync.

Thanks