Document.getProperties() for POJO values

When implementing a Couchbase Light database as local database for one of my Android apps, I put a POJO object in one of the properties of a document.

When retrieving the object back with Document.getProperties() I got back a LinkedHashMap (with the POJO variables as key/value pairs) when accessing the document after the creation of the first revision or after opening the database but got the POJO object back after the document had been updated.

The following code shows this behaviour:

private static String testDocumentId;

class A {

  public String value;
  public A (String value){
  	this.value = value;
  }

}

public void updateTest() throws CouchbaseLiteException{

  Document document = database.createDocument();
  HashMap <String, Object> properties = new HashMap <String, Object>();
  properties.put("key", new A("myValue"));
  document.putProperties(properties);
  Object retrievedValue = document.getProperties().get("key");
  Log.d("Couchbase Update Test", "retrievedValue.getClass() = " + retrievedValue.getClass().getSimpleName());
  properties.put("_rev", document.getCurrentRevisionId());
  document.putProperties(properties);
  Object retrievedValue2 = document.getProperties().get("key");
  Log.d("Couchbase Update Test", "retrievedValue2.getClass() = " + retrievedValue2.getClass().getSimpleName());
  testDocumentId = (String) document.getProperty("_id");

}

public void updateTest2(){

  Document document = database.getDocument(testDocumentId);
  Object retrievedValue = document.getProperties().get("key");
  Log.d("Couchbase Update Test 2", "retrievedValue.getClass() = " + retrievedValue.getClass().getSimpleName());

}

Calling updateTest and updateTest2 one after the other results in

03-22 19:43:25.870: D/Couchbase Update Test(29376): retrievedValue.getClass() = LinkedHashMap
03-22 19:43:25.900: D/Couchbase Update Test(29376): retrievedValue2.getClass() = A
03-22 19:43:25.900: D/Couchbase Update Test 2(29376): retrievedValue.getClass() = A

while closing the database after Update Test 1 and reopening it before Update Test 2 results in

03-22 19:39:29.710: D/Couchbase Update Test(29171): retrievedValue.getClass() = LinkedHashMap
03-22 19:39:29.730: D/Couchbase Update Test(29171): retrievedValue2.getClass() = A
03-22 19:39:29.750: D/Couchbase Update Test 2(29171): retrievedValue.getClass() = LinkedHashMap

Analyzing the source code a bit I found out that Couchbase Light stores the body of the document either in JSON format (during creation or when reading the document from the database) or as key/value hashmap after an update of the document properties. Document.getProperties() seems to just return what is currently available.

My questions are:

  • Is this the expected behaviour of Couchbase Light or is this a bug?
  • Is there any workaround to get independent of the document state
    always the same object type back for properties containing objects?
  • Especially, is there any way to apply Jackson annotations so that the
    objects get properly reconstructed by Jackson’s object mapper? I
    tried the “usual things” using @JsonTypeInfo etc. but did not have
    much luck so far.

Any help highly appreciated.

Thanks

Joerg

Hey @Joerg_Richter,

-Where is your updateTest1() method body?
-If you are creating the objects on the fly then you will be getting new objects each time unless you save the objects that you created.
-Have you tried creating Getter/Setters after aligning the structure of your objects with the structure of your JSON?

Hi @sweetiewill,

ups sorry, slight typo. The line before the second test should have been

… while closing the database after updateTest and reopening it before updateTest2 …

So there are only these two methods for the test which are both described in the source example.

Not sure what you exactly mean by “creating objects on the fly”. My point is not that I get new objects from document.getProperties() every time I call the method but that the object types differ depending on when the document.getProperties() function is called.

I also don’t understand what you mean with “the structure of your JSON”. As far as I understand Couchbase Light I can only put objects as values into the properties map of a document and not JSON strings. Or did I miss anything here?

Joerg

Hey @Joerg_Richter, we were wondering the same thing, seems like cblite always returns a linkedhashmap. One option is to use the Jackson ObjectMapper (very slow when you have many rows) like that:

final ObjectMapper mapper = new ObjectMapper(); // jackson's objectmapper
final MyPojo pojo = mapper.convertValue(map, MyPojo.class);

The other solution that we found performs really well, is to build your model using the LazyJson object. An example that worked for us:

....
        List<MyPojo> result = new ArrayList<>();
        // iterate through the query enumerator
        for (Iterator<QueryRow> it = rowEnum; it.hasNext();) {
            QueryRow row = it.next();
            result.add(new MyPojo((LazyJsonObject) row.getValue()));
        }
.... 
class MyPojo {
    private LazyJsonObject mLazy;

    public MyPojo(LazyJsonObject lazy) {
       mLazy = lazy;
    }

    public String getMyField() {
        return (String)mLazy.get("myField");
    }
}

Hope this helps

Regards,
Vlad

Hi @vladoatanasov,

thank you, interesting approaches, I will try them out.

But one warning from my side: cblite does not always return a LinkedHashMap! If you read the document from the database, update the document properties and save them in a new revision you will get the original property object and not the LinkedHashMap back next time you call getProperties() on the updated document as long as you don’t close the database.

Which means you have to find out what type of property you received using instanceof.

Joerg

I am new to CB and to CB Lite. I have the same problem with the current lite sdk in that I am getting a LinkedHashMap back instead of my Pojo.
My questions are:

  1. What is currently (1.5 years later) the best approach to solve this (problem)? I don’t like the LazyJsonObject approach, since it requires the modification of my pojo classes.

  2. Does CouchBase SERVER (4.5) also behave like this? If so, same solution?

  3. Would the use of the Sync Gateway server force the use of a particular approach in solving this?

Thanks,
nat

hi @natgross,
CB Lite stores the documents as text in SQLite/ForestDB, so when it deserializes it back it doesn’t know about your POJO schema.

  1. You can try the jackson ObjectMapper as described in my previous post, but I found this method to be pretty slow
  2. Couchbase server and couchbase lite are two different things. If you are building a backend Java application, you are better off using the Java SDK, which talks directly to the server. I haven’t done anything with the sdk, so I am not sure about the serialization/deserialization process there. If you are building a client application, like android or some other java app, running on an end device, your best bet is couchbase lite, which is essentially a standalone database.
  3. The sync gateway is a service that “glues” couchbase server and couchbase lite, maintaining revisions between the two endpoints - the server and the client. Let’s say you have data that needs to be stored in the cloud and accessed on the client app, you will have two copies of this document - the server “backup” and the local copy, sync gateway makes sure the document is up to date on both endpoints and helps you resolve any possible conflicts, that can emerge if your client goes offline. So to the point, sync gateway won’t enforce an approach when it comes to serialization/deserialization of data

I hope this helps, maybe someone from the couchbase team, like @hideki, can correct me, my information might be out of date

Regards,
Vlad

I have a solution! But wait…I need some professional input if it is faster the jackson Mapper.
Based on a post in this thread, cb lite does return the pojo and not the LinkedHashMap if the document has been updated.
So, why not always update right after creating the new document? “No way”, you say. Double overhead to create a document??
No! No overhead. How? The original NEW document gets an EMPTY map. Minimal overhead. The update that immediately follows, inserts the data intended for the new document.
Of course it takes longer to insert new (even empty data) and then update, but the question is if it beats the jackson Mapper approach, performance wise. Definitely cleaner code, though.
(I don’t mind that the revision field starts at ‘2’, for the initial data.)

Here is some code I tested, and proven to return the pojo and not a LinkedMap:
public void newDocumentNullValues(Object id) { Document document = db .getDocument(id.toString());// Also creates a new doc. Map<String, Object> mapWithNullValues = new HashMap<String, Object>(); //mapWithNullValues.put(DateObjectKey, null); //mapWithNullValues.put(Pojo1Key, null); try { document.putProperties(mapWithNullValues); updateDocument(id); } catch (CouchbaseLiteException e) { Log.e("DBCrud", "Cannot save document", e); } } public void updateDocument(Object id){ //id might be a Long or a String. if (id==null) return; Document doc = db.getDocument(id.toString()); try { doc.update(new Document.DocumentUpdater() { @Override public boolean update(UnsavedRevision newRevision) { Map<String, Object> map = newRevision.getUserProperties(); map.put(DateObjectKey, new Long(System.currentTimeMillis()));//Test Update Time versus insert time map.put(Pojo1Key, new Pojo1()); //Test Pojo with properties of various types. newRevision.setUserProperties(map); return true; } }); } catch (CouchbaseLiteException e) { e.printStackTrace(); } }
Thanks
nat

Incredibly cbLite behaves differently after app closed and re-opened.
This entire premise that if document was updated, the pojo is returned from the properties, is false - after app has closed and re-opened! In other words, although cbLite behaves as described in previous posts in this thread, that is only as long as app wasn’t closed! Once closed and relaunched, it will always return a LinkedHashMap and not the pojo.
This, with the default db.
I would like to test this with ForestDB, but having general problems with that, as I posted in another thread.
fwiw,
nat

@hideki While we are at it, how about calling a spade - a spade; on this issue, ie a bug.
Summary:
CB Lite inconsistently returns pojos stored in the document properties map. Depending on undocumented -and [somewhat] unknown- factors, a pojo might be returned as a LinkedHashMap or as the same pojo type inserted. Although ideally the database query should always return the pojo itself, this report merely requests that the query should consistently return the same type. If it returns a map, it should always return a map.
This is important because currently we are forced to check the type with instanceof, incurring additional overhead per row.

Hopefully this can still make it into 1.3 and will result in cleaner code as well.
Thanks
nat

Hi @natgross,

Thank you for your suggestions.

  • CBL Android/Java uses Jackson to serialize JSON. LinkedHashMap is returned by Jackson JSON parser. Jackson does not original type, so it always uses LinkedHashMap for Map implementation. If you could observe other Map implementation as the returned instance, I believe it is returned from the cache which is not loaded from JSON. As CBL can not guarantee implementation class of Map and List, CBL can guarantee that the returned instance is implemented Map or List interfaces.

  • CBL Android/Java does not support Model, although CBL iOS does. We have been discussing this, but it could take a time.

Hope this helps you.

Thanks!
Hideki

@hideki This is not a suggestion/enhancement request. This is a problem/bug report.

Wrong! Exactly my point. CBL’s guarantee is flawed! There is no functional guarantee. CBL sometimes does not return a Map nor a List. Sometimes (as detailed in this thread) CBL will return the original pojo, not a Map or List.

Thanks
nat

I believe original POJO is returned from the cache. We will look into this.

I filed the ticket: https://github.com/couchbase/couchbase-lite-java-core/issues/1329

Ok. I commented on github.
Thanks.

@hideki,

I was wondering what’s the status of this issue now? I am facing the same problem now. one of the properties of my document is an array of custom objects. I got an ArrayList of LinkedHashMaps when calling Document.getProperty(KeyOfThisProperty) — so each object is a LinkedHashMap. What’s the best way to get this array of custom objects back and (at least reasonably) performant? Any sample code you can provide? Thanks.

I’m facing this issue as well. I could see it was on v1.4 backlog with High Priority but it was removed, any update on this issue? It’s really critical this kind of bug, I lost several hours trying to figuring it out.

It’s not a bug; we never claimed to support encoding/decoding of custom classes. Document properties are JSON objects like Arrays and Maps. Any conversion to custom classes should be done by your own code.

Document properties are JSON objects like Arrays and Maps. Any conversion to custom classes should be done by your own code.

Is this still true in Couchbase lite 2.x?

Yes. They’re custom classes in 2.0, but they still represent JSON objects.
We’d like to add support for custom classes in a future version, but there’s no timeline for that yet.

1 Like

I am currently converting the old project where I got these issues to Kotlin and it unfortunately turned out, that the approach from @vladoatanasov does not work with Kotlin, if your Kotlin properties have names like mMyValue.

The Problem seems to come from the fact that the Jackson library cannot handle properties that have an uppercase letter within the first two letters of the name, which is documented here. As it seems that the Couchbase Light client (I am using a 1.4 Version) still uses Jackson, even manually converting the document read from the database does not work as the first two Digits of the property names are turned into lower case letters during serialization of the objects.

Luckily Gson does not seem to have this Problem so you can use Gson to convert the object into a JSON compatible format before saving it in the document properties and converting it back afterward.

Thanks to the reified feature of Kotlin generics this can be done easily with the follwing two functions

val gson = Gson()
fun objectToCouchMap(obj: Any) = gson.fromJson(gson.toJson(listOf(obj)), ArrayList::class.java)[0]
inline fun <reified T> couchMapToObject(map: Any?) = gson.fromJson<T>(gson.toJson(map), object : TypeToken<T>() {}.type)

The conversion then works like this:

var yourObject: yourType
...
documentProperties[key] = objectToCouchMap(yourObject)
...
yourObject = couchMapToObject<yourType>(documentProperties[key])

There may be caveats with these functions, so use them at your own risk. :slight_smile: For example, Gson turns integer literals into doubles during deserialization if it has no other information available. So integers are stored as doubles with this tranformation. Nevertheless the doubles reappear as integers if your target class has specified Int and not Any for the corresponding property.

I hope this helps someone else who faces a similar challenge when converting a couchbase project from Java to Kotlin.

Joerg

1 Like