Bulk upsert data into Couchbase

Couchbase Server
2

Hi,

This is an issue in the Go SDK (https://issues.couchbase.com/projects/GOCBC/issues/GOCBC-231). The issue is that too many requests are being queued and the dispatcher queue is becoming full. You can see this issue is happening by adding something like this before your bucket close, after Do:

for _, op := range batch {
    upsertOp := op.(*gocb.UpsertOp)
    if upsertOp.Err != nil {
		fmt.Println(upsertOp.Err)
	}
}

You can see that the error is “queue overflowed”, you can check for this error explicitly by comparing to gocb.ErrOverload. You could try using more, smaller batches or adding failed ops to a new batch and repeatedly doing that until none have failed.

1 Like