Array Indexation consistency in sub document mutations

Java SDK
1

Hello,
I try to perform sub document mutations on arrays but I’m affraid the paths containing fixed index can lead to unwanted changes.

I have such structure:
{
“age”: “101”,
“name”: “testName”,
“testArray”: [
{
“id”: “x0”,
“obj1Field”: “obj1FieldValue”,
“obj1Field2”: {
“field1”: “fv1”,
“field2”: “fv2”
}
},
{
“id”: “x2”,
“obj1Field”: “testValue”,
“obj1Field2”: {
“field1”: “fv3”,
“field2”: “fv4”
}
},
{
“id”: “x1”,
“obj1Field”: “testValue2”,
“obj1Field2”: {
“field1”: “fv5”,
“field2”: “fv6”
}
}
],
“testInnerObj”: {
“field1”: “value1”,
“field2”: 4,
“innerArray”: [
“newValue”,
“newValue2”,
“newValue3”
]
}
}

I want to use MutateIn Java SDK to remove elements in the arrays (testArray for example).
I can use path like this: “testArray[1]” in remove or upsert MutateInSpec.

How can I be sure it always will point to the same object in array?

Is there any way to filter array elements by id?

Let’s say I will get object like this:
{
“operation”: REMOVE,
“objectId”: “x1”,
“path”: “testArray”
}

Can I retrieve sub document part like “testArray” and operate on it like it would be a separate document and modify it freely, so the changes will be applied in proper objects and sub inserted in document as sub document operation?

Please advise me how can I utilise sub document operations in such case to assure proper removal?

I saw I could use ARRAY (CASE WHEN… THEN…) FOR … END operation of N1ql.
But it will require use of UPDATE statement which I read will always use full docment operation, right?
Can I combine those two approaches somehow?

2

Hi @DominikS
Is it possible to adjust your data model so that instead of an array you have a map, something like this:

{
  "age": "101",
  "name": "testName",
  "test": {
    "x0": {
      "obj1Field": "obj1FieldValue",
      "obj1Field2": {
        "field1": "fv1",
        "field2": "fv2"
      }
    }
  }
}

Now you can delete “test.x0”.

1 Like
3

Thank you @graham.pople for a quick reply.
This looks great.
Only thing I worry about is how to use and translate such structure to graphql and mupstruct.

Do you have any suggestion on that?

4

Hey @DominikS
Sorry, I don’t have enough experience with those technologies to speak to them.

5

HI @graham.pople after a while I wanted to give it a try and change the structure using the maps.
But instead of using maps in java entities I would like to have regular lists.

I thought about using the writing and reading convertes to convert from list of objects to the structure you proposed as Document (for me it is a map).

I don’t know how to convert the CouchbaseDocument when it is more complex and can have nested similar objects:

"test": {
    "x0": {
      "obj1Field": "obj1FieldValue",
      "obj1Field2": {
        "field1": {
                "fv1": {},
                "fv2": {}
                ...
         }
        "field2": {
                "nt1": {},
                "nt2": {}
                ...
         }
      }
    }

My approch is using the ObjectMapper:

    ObjectMapper mapper = new ObjectMapper();
    mapper.registerModule(new JavaTimeModule());
    for (Object o : source.getContent().values()){
        Class clz = Class.forName((String) source.get("_class"));
        Object entity = mapper.readValue(source.toString(), clz);
        list.add((Identifiable) entity);
    }

but then I end up transforming the highest level up to the bottom at first read not relying on the couchbase converters mechanism.

What is more I got exception:
“Unrecognized token ‘CouchbaseDocument’: was expecting (JSON String, Number, Array, Object or token ‘null’, ‘true’ or ‘false’)” which is quite obvious :slight_smile:
Do you have maybe some idea what approach would be good in this case?

I tried having java Map instead of List in all my model but it requires a lot of changes and if possible I would like to avoid them.

6

What I want to achieve is:
Java:

class Org{
  List<Person> employees;
}
class Person {
  List<Address> addresses;
}

but in Couchbase database have:

{
  "employees": {
     "Person::1": {
        "addresses": {
           "Address::1": {},
           "Address::2": {}
         },
     "Person::2": {
        "addresses": {
           "Address::3": {},
           "Address::4": {}
         }
    }
}

Is it possible using Couchbase? How can I achieve it?

7

Hi @DominikS
I’m not sure if it’s directly possible to get Jackson (or another JSON library) to do an advanced mapping like that directly. I would probably do it more simply (albeit more inefficiently), by initially converting it as a JsonObject. And then walking over those, asking Jackson to convert the objects into your model classes. Something like this incomplete and untested example:

GetResult doc = collection.get("docId");
JsonObject json = doc.contentAsObject().getObject("employees");

List<Person> employees = new ArrayList<>();

json.getNames().forEach(employeeName -> {
    /*
     * employeeJson contains:
     *  "Person::1": {
     *      "addresses": {
     *          "Address::1": {},
     *          "Address::2": {}
     *      }
     *  }
     */
    JsonObject employeeJson = json.getObject(employeeName);

    // Note - this won't actually work, since your addresses are also in a map
    Person person = Mapper.convertValue(employeeJson.toBytes(), Person.class);
    
    employees.add(person);
});

But as the comment notes, the actual work would be more complex, since you’d have to do similar logic to walk over and decode the address objects.

Some other ideas to explore:

  1. Jackson would be able to decode directly into these alternative classes:
class Org{
  Map<String, Person> employees;
}

class Person {
  Map<String, Address> addresses;
}

You could then flatten these out in code e.g. List<Person> people = employees.values()

  1. Do you need map-like access to every part of the document? Maybe you could reduce complexity by just having the bits that you absolutely do need to access through sub-doc in a map format.

  2. Perhaps this whole approach isn’t worth the complexity. Sub-doc is an optimisation, and it’s always easiest to simply fetch the full document. While sometimes it can make sense to change the data model to support sub-doc for performance reasons, it does add complexity, and it’s really down to your business requirements whether it’s worth it.

1 Like
8

@graham.pople thank you for the support here.
I succeeded transforming above example into map style document in DB.
From my perspective we need direct access for each level of nesting.

Right now I wonder how can I query this type of documents using UNNEST statement, which is not supporting such structure. Or at least when using it like this is not the right way:

select adrs.* from person
unnest addresses adrs