New documents are not being inserted

Java SDK
1

Hi,
We’re new to Couchbase in our organization, and we encountered a weird issue.
When trying to set multiple rows through the following code:

This is how our “set” method looks like at the moment:
public void setCampaign(Campaign camp){
client.set(camp.getId(), gson.toJson(camp, Campaign.class),PersistTo.ONE);
System.out.println(“Campaign with key " + camp.getId() + " was successfully inserted!”);
}

this part is a loop we use to test multiple insert statements:
for (int i = 0; i < 10000; i ++) {
Campaign c = new Campaign(“Chukumuku::” + i,new Date());
r.setCampaign©;
}

We’ve made the connection like in the samples of the website, using couchbase client 1.3.2 (gradle)
The Couchbase server’s version is 2.1.1 and it is deployed on Amazon on CentOS.
We’re using a single node cluster, for the test.

The issue is that the following code, did not insert all records but only 60-80%. even when we added PersistTo.One it helped, but did not ensure 100% of the data is inserted.

What am I missing here?Is couchbase supposed to ensure persistennce somehow?
Thanks,
Arthur

2

Could you check the return value of the set call?

OperationFuture res = client.set(camp.getId(), …);
if (!res.get()) {
System.out.println("Failed for " + camp.getId());
}

It’ll be interesting to know if the same keys fail all the time, or if it is random.

Couchbase first keeps all the keys in Memory at first PersistTo.ONE makes sure the key is persisted on its Master. You can read more about the persistence model here. So even without this added all keys should be stored.

3

Thanks for the info!
This code significantly slowed down the insert performance, but helped.
When I added thread sleep for 1 ms it helped too.

From the docs which did not exist in the bucket after running the loop without modifications,
I could see that it is totally random. Maybe it has something to do with the driver, or some buffer sizing?

4

I asked about this sometime ago: http://www.couchbase.com/communities/q-and-a/couchbase-do-not-store-all-documents

5

As explained by tgrall as mentioned above the slowdown is due to make the operation synchronous and therefor it has to wait before it can insert the next document. What happens if you run the code as in this gist by tgral? Do you also see failures?