Here is my code snippet. if my batch size is 30K or more then exceptions are thrown com.couchbase.client.java.error.TemporaryFailureException. Is there we can tune this async operation?
flatMap takes a maxConcurrent parameter that lets you control how many operations are in-flight at once.
TemporarilyFailure usually indicates the server is temporarily overloaded, it’s effectively backpressure. If you’re going for maximum throughput you will see a few of these, so it’s worth adding error handling for this and monitoring how many. (One advantage of reactive programming is it makes it easy to log & retry on specific error codes.)
Also you may want to consider tuning the kvEndpoints parameter in your CouchbaseEnvironment. This controls the number of connections created by the SDK to each KV process (there is one per node). That KV process maintains one internal event loop for each core on that node. You’ll probably achieve maximum throughput if you have one connection from the SDK for each of those event loops - so in short if you set your kvEndpoint parameter to your cluster per-node core count, that’s likely to be close to optimal.
Or you can just try different values (2, 4, 8, 16 etc.) and see what gives best throughput while minimising TemporaryFailures.
(All of this advice assumes only one application is running, which is possibly the case since you seem to be doing a batch loader. If you have multiple applications running, then I’d recommend being more conservative with the kvEndpoints setting.)