I’m currently evaluating Couchbase for my organization and have ran into a snag that for the life of me I have not been able to resolve. The use case is mostly straightforward so I’ll try to explain. I am currently running a two node cluster of Couchbase. I have created a single couchbase bucket enabling 1 replica and am trying to populate it with RawJsonDocuments.
I am doing a bulk load of RawJsonDocuments utilizing the java client version 2.2.8 and the rx observable. Here is snippet of the code used to load the documents.
private Func1<RawJsonDocument, Observable<RawJsonDocument>> upsertFunction = new Func1<RawJsonDocument, Observable<RawJsonDocument>>() {
public Observable<RawJsonDocument> call(final RawJsonDocument docToInsert) {
return bucket.async().insert(docToInsert);
}
};
Observable
.from(documentList)
.flatMap(upsertFunction)
.last()
.toBlocking()
.single();
I’m iterating over this code thousands of times loading a thousand documents at a time. This code works with no problems as long as the bucket I’m inserting into has no views defined. I’ve loaded 30 million documents within the same batch operation without issue. However, as soon as I add a view to the bucket, I start seeing the following error in our logs after about 500,000 upserts:
OnErrorThrowable.OnNextValue: OnError while emitting onNext value: com.couchbase.client.core.message.kv.UpsertResponse.class
rx.internal.operators.OperatorMerge$MergeSubscriber:242
rx.internal.operators.OperatorMerge$MergeSubscriber:776
rx.internal.operators.OperatorMerge$MergeSubscriber:537
rx.internal.operators.OperatorMerge$MergeSubscriber:526
rx.internal.operators.OperatorMerge$InnerSubscriber:810
rx.internal.operators.OperatorMap$1:48
rx.exceptions.Exceptions:188
]
The mapping function within the bucket is trivial:
function (doc, meta) {
if(doc.documentType == "FooDocument") {
emit(meta.id, doc);
}
}
The error seems to occur as the indexer is running on the view but this may just be coincidence. Has anyone else ran into this issue? Is there something foolish I’m doing here? Any help here would be much appreciated!