Maintain order in array append

Java SDK
1

Hi, I am new to couchbase. I am appending an element to an array inside a json document. What I notice is that the order in which the appends are issued is not the order in which the elements occur in the array. For example, if the json document is {x:1, arrayx: [a, b, c]}, the append for c may have occurred before the append for b, but c occurs after b in the array. How can I ensure that the elements are ordered in the sequence that the appends were issued? I am using the following to append (in java): bucket.mutateIn(id).arrayAppend(“arrayx”, data).execute();

2

N1QL ARRAY_APPEND() function appends at the end of existing array. One in java might be using different approach directly with KV. You can try N1QL query

UPDATE  bucket  AS b USE KEYS "docid"
SET b.arrayx = ARRAY_APEND(b.arrayx, "new element")
3

mutateIn() uses the KV subdoc API.

@sparky1, from your code sample, I can’t quite tell what data is. Are you issuing multiple mutateIn().arrayAppend() statements and that’s when you see differing order, or is it a single statement? If you don’t mind, could you post a small code snippet that demonstrates the issue?

Note that the N1QL ARRAY_APPEND() retrieves the full document, modifies it at the query server, then does a CAS set/upsert of the entire document.

4

data is another element in the array and is actually a JsonObject (not a simple element like a,b, c as in my example). I do something like the following :

public void insertData(MData mData) {
Bucket bucket = CouchDBConnectionFactory.CONNECTION.bucket(“XX”);
String id = mData.getId();
JsonObject data = JsonObject.fromJson(mData.toJson());

	if (!bucket.exists(id)) {
		JsonObject o = JsonObject.create()
				.put(“cId", mesgData.getcId())
				.put("_type", “YY”);
		JsonDocument doc = JsonDocument.create(id, o);
		bucket.insert(doc);
	}
          bucket.mutateIn(id).arrayAppend(“arrayx”, data).execute();

}

insertData is called repeatedly from the upper layers in the application and so, yes, I am (in effect) issuing multiple mutateIn().arrayAppend() statements from the application. The calls for a particular document are sequenced in the application layer and are timestamp-based. However, occasionally, I see out of order messages in the array. Everything else seems to be working fine. Wondering if the array append operations are not guaranteed to be sequenced for performance reasons and if there is a way I can guarantee them. Again, I am new to couchbase, so trying to find my way here. Thanks for your help.

5

Thanks, I will try this.

6

Sorry, I missed a line in the code to create the jsonarray. Here it is again:

public void insertData(MData mData) {
Bucket bucket = CouchDBConnectionFactory.CONNECTION.bucket(“XX”);
String id = mData.getId();
JsonObject data = JsonObject.fromJson(mData.toJson());

 if (!bucket.exists(id)) {
	JsonObject o = JsonObject.create()
			.put(“cId", mData.getcId())
                            .put("arrayx", JsonArray.create())
			.put("_type", “YY”);
	JsonDocument doc = JsonDocument.create(id, o);
	bucket.insert(doc);
}

bucket.mutateIn(id).arrayAppend(“arrayx”, data).execute();
}

7

Hi Sparky,

If you’re calling insertData repeatedly in the same thread then the append operations are expected to be sequenced. If you’re calling it from different threads then there are no guarantees about which request will reach Couchbase Server first.

If you’re calling from multiple threads and you need to the timestamp and array order to be consistent, you could try serializing the process that generates the timestamp and issues the append request, so that these to operations are never interleaved with other requests.

Thanks,
David

8

Hi David,

Ok, got it. The requests are coming from different threads and they are guaranteed to be generated in order from the app, but they are not guaranteed to arrive at the couchbase server in order.

Thanks!