Potential bug in handling of dropped collections

Hi all,
since we have updated to 3.4.3 we have this strange issue.
Our requirement is that after a collection was dropped e.g. by a administrator a new one should be created and the data should be reloaded by our software.

But after dropping a custom collection the SDK seems not to realize that the collection does not exist anymore. Instead if we add new values they are added to the standard collection silently.
Before we got this working by catching the exception during insertion and then checking if the collection does not exist. Now it seems that it hides this error from us, so we can not react on it.

Here is a test that reproduces the issue:

        [Test]
        [Explicit]
        public async Task SetValueAsync_ReproduceDefaultCollectionHandling_CollectionAndElementAdded()
        {
            // arrange
            string collectionName = "Custom";
            string id = "ID1";
            string content = "content";

            var clusterOptions = new ClusterOptions();
            clusterOptions.UserName = "aaa";
            clusterOptions.Password = "bbbb";

            var cluster = await Cluster.ConnectAsync("couchbase://localhost", clusterOptions);
            await cluster.WaitUntilReadyAsync(new TimeSpan(0, 0, 30));
            var bucket = await cluster.BucketAsync("test");

          IScope scope = await bucket.DefaultScopeAsync();
            var collection = await scope.CollectionAsync(collectionName);
            await scope.Bucket.Collections.CreateCollectionAsync(new CollectionSpec(scope.Name, collectionName));
            await collection.InsertAsync(id, content);

            // verify it was added properly
            var result = await collection.TryGetAsync(id);
            result.ContentAs<string>().Should().Be(content);

            // verify its not in the default collection
            var defaultCollection = await scope.CollectionAsync("_default");
            result = await defaultCollection.TryGetAsync(id);
            result.Exists.Should().BeFalse();

            // remove the custom collection
            var spec = new CollectionSpec(scope.Name, collectionName);
            await scope.Bucket.Collections.DropCollectionAsync(spec);
            await Task.Delay(1000); // fun fact if we don't wait the data still exists... -> bug?

            // add again but no collection exists 
            await collection.InsertAsync(id, content);

            // it simulates that its coming out of our custom collection
            result = await collection.TryGetAsync(id);
            result.ContentAs<string>().Should().Be(content);

            result = await defaultCollection.TryGetAsync(id);
            result.Exists.Should().BeFalse(); // this fails because the data was inserted to _default !!!
        }

Hi @p.klein,

I am a bit confused about this line:

Earlier in this test you create a cluster object and open a bucket:

But then your using a class level _couchbaseService instance to drop the collection? Why not use the local Cluster instance? I am asking because I am trying to debug the code you posted and cannot resolve that dependency. Perhaps the test itself is not correct?

Jeff

You are right this local reference shoul not be there. Never the less could you also add a breakpoint and drop the collection manually to see the behavior. Let me check if i can fix the test…

Think i have fixed the test still behaves wrong on my side. Have editied the initial post

1 Like

@p.klein -

A couple of things I found:

  • TryGetAsync: this method was added in 3.4.3, so this code could not have existed in 3.4.2.

  • By design TryGetAsync does not throw the DocumentNotFoundException.

  • I tested both TryGetAsync on 3.4.3 as you used here and GetAsync in 3.4.2, but caught the DocumentNotFoundException instead to simulate what’s going on here. The end behavior in both cases is the same (assuming you catch the DocumentNotFoundException; the _default collection ends up with “ID1”.

I do know that by default, every bucket on the server has both a default scope and collection (_default). If the either is omitted and you add a key to a bucket, then the key will end up in the default scope and collection. That being said, I would think the operation would fail if the collection did not exist by the server, so I am going to dig a bit deeper into the SDK and see what I find.

I was just using TryGetAsync to demonstrate the behavior. Our production code looks different…
My point is more about the behavior of InsertAsync. There should be a way to find out that the collection does not exist. Before it was possible by catching the exception…

Thank you :slight_smile:

Seems like the bahavioral change was introduced with 3.4.1. I can get the old behavior with 3.4.0 OperationCanceledException with reason System.OperationCanceledException with: (CollectionNotFound,CollectionNotFound,CollectionNotFound,CollectionNotFound,CollectionNotFound,CollectionNotFound,CollectionNotFound)’

@p.klein -

Ok thanks, that helps tremendously. I created a Jira ticket for tracking.

Jeff

@p.klein -

I pushed a commit and you can follow the Gerrit code review link in the ticket and checkout the changes.

Jeff

Great News! Thank you for the fast response! :smiley:

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.