Query over Serializable objects


#1

Hi there,

I loaded some java objects(POJO) into bucket by using SerializableDocument, they are stored as bytecode (not human readable), is there anyway to query over these serializable documents?

thanks!
Jeff


#2

Probably not. Can you store a second JSON document that contains a key to the serialized POJO?


#3

Hi Gerald,

Thank you for replying!

I did store JSON document contains some of those POJO’s attributes on bucket.
Just wondering how to store and query POJO directly.
Do you have any example about using N1QL on IDE?

many thanks!


#4

hi @Jeff_Chen

Is there a particular reason for storing the POJOs in serialized form?

Since N1QL works only on JSON documents, maybe a solution would be to use a library like Jackson to serialize/deserialize to/from JSON? You can then use the RawJsonDocument type instead of SerializedDocument to store the string Jackson will give you…
Anyway, that’s one possibility if you want to store and query the POJOs directly.

Concerning examples on IDE, there’s a blog post on the usage of the latest developer preview (but watch for Java SDK 2.1-dp-2 next week, because there’s been a bug identified :frowning: ).

Simon


#5

Hi Simon,

Thank you for responding!
(1)
I did customize the Transcoder using Jackson library and loaded those POJOs into bucket.
the output is human readable, but with redundant slash and make it difficult to query, example:
“1_1200000”: "{“category”:null,“ID”:1,“key”:“1_1200000”}"
any suggestion to fix this?

(2)
my viewResult doesn’t support the doOnNext() method, do I miss any dependencies?

thank you again for helping!
Jeff


#6

Hi @Jeff_Chen

(1) is that the output in the couchbase console? Did you use the RawJsonDocument?
The idea would be to encode POJO into JSON String using Jackson’t ObjectMapper.writeValueAsString(pojo). Then you store the resulting String using a RawJsonDocument.
You should obtain simple valid JSON in couchbase console and be able to query it using N1QL.

(2) What do you mean it doesn’t support it? I think you are using the sync API, the one on Bucket (which is the one most easily accessed). If that’s the case, call async() on your bucket to get an AsyncBucket reference, and the query will return an Observable<AsyncQueryResult>. In turn, this result will have Observable-returning methods, and Observable offers a doOnNext method…

On a side node, doOnNext, doOnError and doOnCompleted are made to implement side-effects, like logging. The “correct” way of working with items is to subscribe an Observer or an Action1


#7

Hi Simon,

Still can’t find SDK 2.1 -dp-2 from official site, same issue occurred. can you help checking the code:

            System.setProperty("com.couchbase.queryEnabled", "true");
	CouchbaseCluster cluster = CouchbaseCluster.create("127.0.0.1");
	Bucket trackBucket = cluster.openBucket("tracks", "");

	Observable<AsyncQueryResult> nkquery = trackBucket.async().query(
			"SELECT trackID FROM tracks WHERE trackType = 'RUN' LIMIT 100");

(1)
List list = nkquery.flatMap(
new Func1<AsyncQueryResult, Observable>() {

						public Observable<AsyncQueryRow> call(AsyncQueryResult t1) {							
							return t1.rows();
						}
					}).toList().toBlocking().single();

              for (AsyncQueryRow row : list)
		         System.out.println(row.value());

Program hangs in flatMap() forever with no output.

(2)

                             nkquery.flatMap(
					new Func1<AsyncQueryResult, Observable<AsyncQueryRow>>() {

						public Observable<AsyncQueryRow> call(AsyncQueryResult t1) {

							return t1.rows();
		}
			}).subscribe(new Action1<AsyncQueryRow>() {

		public void call(AsyncQueryRow t1) {
			System.out.println(t1.toString());

		}
	});

program finished execution with no output.

I am using Java SDK 2.0.3 and N1QL DP4.
thank you!
Jeff


#8

hi @Jeff_Chen, Java SDK 2.0.3 is not compatible with N1QL DP4 so it may be that.

I had pointed you to a blog post containing pom.xml instructions to get at least 2.1.0-dp, preferably 2.1.0-dp2 (some bugs had been identified in the first developer preview). But now we have released the 2.1.0 version proper :smile: (note: rather we are currently releasing, could be up to 2 hours before Central is synchronized)

So just replace the java-client version string in your pom.xml, 2.0.3 to 2.1.0, and it should pick it up from Maven Central.

From a code point of view, I see nothing particularly wrong with your code. If you really want to go the async way, snippet (2) is better IMO. Just a side note: to enable query feature programmatically, you can use the CouchbaseEnvironment rather than setting the system property, by doing:

CouchbaseEnvironment env = DefaultCouchbaseEnvironment.builder()
                .queryEnabled(true)
                .build();
Cluster cluster = CouchbaseCluster.create(env, "127.0.0.1");

#9

Thank you Simon for all your help, I really appreciate it!

Now I can query over bucket using N1QL DP4. but the subscribe() method still not working, don’t know why.


#10

Two things to check:

  • if you’re calling the async code directly in a main, put a Thread.sleep(5000) at the end to ensure the JVM doesn’t exits before the query has been processed (one never knows)
  • since you were on dp3 apparently (or at least an “old” version of N1QL server), maybe the N1QL index needs to be rebuilt

You should have access to the cbq command line tool. Try to execute your query and see if it returns with a positive result count. If not, maybe try to issue the following two commands to rebuild the index:

DROP PRIMARY INDEX ON tracks;
CREATE PRIMARY INDEX ON tracks;

I was able to execute the following code (I adapted the bucket and put documents with trackID and trackType “RUN” and “JUMP” in my default bucket):

public static void main(String[] args) throws InterruptedException {
    System.setProperty("com.couchbase.queryEnabled", "true");
    CouchbaseCluster cluster = CouchbaseCluster.create("127.0.0.1");
    Bucket trackBucket = cluster.openBucket("default", "");

    Query q = Query.simple("SELECT trackID FROM default WHERE trackType = 'RUN' LIMIT 100");
    Observable<AsyncQueryResult> nkquery = trackBucket.async().query(q);
    nkquery.flatMap(
            new Func1<AsyncQueryResult, Observable<AsyncQueryRow>>() {
                public Observable<AsyncQueryRow> call(AsyncQueryResult t1) {
                    return t1.rows();
                }
            }).subscribe(new Action1<AsyncQueryRow>() {

        public void call(AsyncQueryRow t1) {
            System.out.println(t1.toString());

        }
    });

    Thread.sleep(2000);
    cluster.disconnect();
}

I have 3 documents (2 with RUN, 1 with JUMP), output on console is:

{"trackID":"track1"}
{"trackID":"track2"}