Retrieve statistics after batch async upsert

n1ql

#1

Hello good people from couchbase!
I am currently trying to create some mechanism to validate that all my documents made it to the DB, I am using the java SDK for the insertions, something along the lines of:

            protected void bulkLoad(List<JsonDocument> documents, long batchSize) {
                Observable<JsonDocument> largeBulk =
                    Observable.from(documents)
                        .flatMap(doc -> couchbaseBucket.async().upsert(doc))
                        .retryWhen(
                            RetryBuilder.anyOf(Exception.class)
                                .delay(Delay.exponential(TimeUnit.MILLISECONDS, 1000))
                                .max(20)
                                .build());

                largeBulk.subscribe(createSubscriber(documents, batchSize));
              }

Now, this function is called several times inside a loop, and I am saving the total amount of documents I have been inserting. The question here is: How can I verify that all of my documents made it to couchbase? As of now I see no way of getting this info programatically, is there of call I can make through the SDK to get this info from couchbae using N1QL or even using the couchbase REST API? Any feedback is more than welcome.

Thanks!


#2

If there aren’t any errors, it will be in the bucket. One thing you could do is also run a query when done to count.