I can see the debug log keep retrying on NotMyVBucketException
Debug 2021-07-05 04:03:54.571 +00:00 - Receiving new map revision 25315
Debug 2021-07-05 04:03:54.572 +00:00 - Op failed: Couchbase.Core.IO.Operations.Set`1[System.String]: Couchbase.KeyValue.NotMyVBucketException: Exception of type 'Couchbase.KeyValue.NotMyVBucketException' was thrown.
at Couchbase.Core.ClusterNode.ExecuteOp(Func`4 sender, IOperation op, Object state, CancellationToken token)
-----------------------Context Info---------------------------
{"DispatchedFrom":null,"DispatchedTo":null,"DocumentKey":"sh:1178129","ClientContextId":"730","Cas":0,"Status":7,"BucketName":"mvmx_hk_trunk","CollectionName":"_default","ScopeName":"_default","Message":"KV Error: {Name=\"NOT_MY_VBUCKET\", Description=\"Server does not know about this vBucket\", Attributes=\"fetch-config,invalid-input\"}"}
Debug 2021-07-05 04:03:54.573 +00:00 - Retrying op 730/"sh:1178129" because KvNotMyVBucket and always retry.
Debug 2021-07-05 04:03:54.635 +00:00 - Updating new map revision 25315
Debug 2021-07-05 04:03:54.638 +00:00 - Publishing config revision 25317 to subscribers for processing.
Debug 2021-07-05 04:03:54.641 +00:00 - Current revision for "mvmx_hk_trunk" is rev#25317
Debug 2021-07-05 04:03:54.644 +00:00 - Receiving new map revision 25315
Debug 2021-07-05 04:03:54.647 +00:00 - Updating new map revision 25315
Debug 2021-07-05 04:03:54.650 +00:00 - Publishing config revision 25317 to subscribers for processing.
Debug 2021-07-05 04:03:54.652 +00:00 - Current revision for "mvmx_hk_trunk" is rev#25317
Debug 2021-07-05 04:03:54.655 +00:00 - Receiving new map revision 25315
Debug 2021-07-05 04:03:54.658 +00:00 - Updating new map revision 25315
Debug 2021-07-05 04:03:54.659 +00:00 - Executing op Set on "10.112.64.109:11210" with key "sh:1178370" and opaque 732.
Debug 2021-07-05 04:03:54.661 +00:00 - Publishing config revision 25317 to subscribers for processing.
Debug 2021-07-05 04:03:54.662 +00:00 - Server "10.112.64.109:11210" returned VBucketBelongsToAnotherServer for op Set with key "sh:1178370" and opaque 732.
Debug 2021-07-05 04:03:54.664 +00:00 - Current revision for "mvmx_hk_trunk" is rev#25317
Debug 2021-07-05 04:03:54.667 +00:00 - {"NetworkResolution":"auto","SurrogateHost":null,"rev":25315,"name":"mvmx_hk_trunk","uri":"/pools/default/buckets/mvmx_hk_trunk?bucket_uuid=e3f065a1c1d63567ee19f21ba8bedcde","streamingUri":"/pools/default/bucketsStreaming/mvmx_hk_trunk?bucket_uuid=e3f065a1c1d63567ee19f21ba8bedcde","nodes":[{"couchApiBase":"http://10.112.64.108:8092/mvmx_hk_trunk%2Be3f065a1c1d63567ee19f21ba8bedcde","couchApiBaseHttps":"https://$HOST:18092/default","hostname":"10.112.64.108:8091","ports":{"direct":11210,"proxy":11211,"sslDirect":11207,"httpsCAPI":18092,"httpsMgmt":18091},"services":null,"version":null},{"couchApiBase":"http://10.112.64.109:8092/mvmx_hk_trunk%2Be3f065a1c1d63567ee19f21ba8bedcde","couchApiBaseHttps":"https://$HOST:18092/default","hostname":"10.112.64.109:8091","ports":{"direct":11210,"proxy":11211,"sslDirect":11207,"httpsCAPI":18092,"httpsMgmt":18091},"services":null,"version":null},{"couchApiBase":"http://10.112.64.110:8092/mvmx_hk_trunk%2Be3f065a1c1d63567ee19f21ba8bedcde","couchApiBaseHttps":"https://$HOST:18092/default","hostname":"10.112.64.110:8091","ports":{"direct":11210,"proxy":11211,"sslDirect":11207,"httpsCAPI":18092,"httpsMgmt":18091},"services":null,"version":null}],"nodesExt":[{"thisNode":false,"services":{"mgmt":8091,"mgmtSSL":18091,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"kv":11210,"kvSSL":11207,"capi":8092,"capiSSL":18092,"projector":9999,"n1ql":8093,"n1qlSSL":18093,"cbas":8095,"cbasSSL":18095,"fts":8094,"ftsSSL":18094,"moxi":11211},"hostname":"10.112.64.108","alternateAddresses":null,"HasAlternateAddress":false},{"thisNode":true,"services":{"mgmt":8091,"mgmtSSL":18091,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"kv":11210,"kvSSL":11207,"capi":8092,"capiSSL":18092,"projector":9999,"n1ql":8093,"n1qlSSL":18093,"cbas":8095,"cbasSSL":18095,"fts":8094,"ftsSSL":18094,"moxi":11211},"hostname":"10.112.64.109","alternateAddresses":null,"HasAlternateAddress":false},{"thisNode":false,"services":{"mgmt":8091,"mgmtSSL":18091,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"indexHttps":19102,"kv":11210,"kvSSL":11207,"capi":8092,"capiSSL":18092,"projector":9999,"n1ql":8093,"n1qlSSL":18093,"cbas":8095,"cbasSSL":18095,"fts":8094,"ftsSSL":18094,"moxi":11211},"hostname":"10.112.64.110","alternateAddresses":null,"HasAlternateAddress":false}],"nodeLocator":"vbucket","uuid":"e3f065a1c1d63567ee19f21ba8bedcde","ddocs":{"uri":"/pools/default/buckets/mvmx_hk_trunk/ddocs"},"vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":2,"serverList":["10.112.64.108:11210","10.112.64.109:11210","10.112.64.110:11210"],"vBucketMap":[[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,1,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[1,2,0],[1,2,0],[2,0,1],[2,0,1],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,2,1],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,1,2],[0,2,1],[2,1,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,2,0],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,1,0],[2,1,0],[2,1,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[2,1,0],[2,1,0],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,1,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,1,0],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[1,0,2],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[1,2,0],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[0,1,2],[0,1,2],[0,2,1],[0,1,2],[0,1,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,1,2],[1,0,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[1,0,2],[0,2,1],[0,2,1],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[2,1,0],[2,1,0],[2,1,0],[0,2,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,0,2],[1,0,2],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[0,2,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,1,0],[2,1,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,1,0],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,2,1],[0,2,1],[1,0,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,0,2],[1,0,2],[1,0,2],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,2,0],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[2,0,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,0,1],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[2,0,1],[2,0,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[1,0,2],[0,1,2],[0,1,2],[0,1,2],[0,1,2],[1,2,0],[1,2,0],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,2,1],[1,2,0],[1,2,0],[1,2,0],[2,1,0],[1,0,2],[1,0,2],[1,0,2],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[0,2,1],[2,0,1],[2,0,1],[2,1,0],[0,2,1],[2,0,1],[2,0,1],[2,0,1],[0,1,2],[0,1,2],[0,1,2],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[1,2,0],[0,1,2]],"vBucketMapForward":[]},"bucketCapabilitiesVer":"","bucketCapabilities":["couchapi","xattr","dcp","cbhello","touch","cccp","xdcrCheckpointing","nodesExt"],"clusterCapabilitiesVer":null,"clusterCapabilities":null,"VBucketMapChanged":false,"ClusterNodesChanged":false}
Upgrade to SDK 3.1.7 cause this problem in this issue for me: "Only the default Scope is supported by Memcached Buckets" Error
This is the code I’m using, I don’t think bucket/cluster is dispose anywhere
var bucket = await _playerTagsBucketProvider.GetBucketAsync(); //INamedBucketProvider
var collection = bucket.DefaultCollection();
await collection.UpsertAsync(...);
And finally, SDK version is 3.1.0 and server version is Enterprise Edition 6.0.3 build 2895