Not able to get all rows returned from Couchbase View

Thanks Daschl for reply to my post.

In database I have more than 1000 records are there, if I am not providing any filter it’s returning 66 rows and if I provide filter it is only returning 2 records.

If I run the view on Couchbase Console, it is showing me all records.

Do I need to configure some thing to pull all rows from a view?

did you make sure to publish the view and then access the published one from both SDK and UI? Development views can vary greatly in the data they return, only a published view will give you the right output on your complete dataset.

2 Likes

@daschl, When I am running view from console with ‘full_set=true’ , it’s giving all rows. But I am not able to pass this option from Java Client SDK…

I need some workaround to pass this value.

as noted in the other forum post, it seems you are running on development views. Publish your view into production, then the data set will be accurately returned.

@daschl, Thanks for solution, now I am able to get full set data after publishing view to ‘Production’.

Here I am facing another issue, now I am not able to filter data based on ‘keys’.

/default/_design/couchbase_service/_view/getUtilizationBasedOnPractice?stale=false&inclusive_end=false&keys=%5B%22DAS%22%5D

If I use mentioned URL with ‘keys’, it’s returning

{“total_rows”:73342,“rows”:[
]
}

If I remove ‘keys’ attribute it’s giving me all records in row section.

NOTE: I copied above mentioned URL from *couchdb.log*

ViewQuery.from(documentDesignName, viewName).keys(JsonArray.fromJson(keysJson)).stale(Stale.FALSE)

You can use it in this way.

@shiv4nsh, I am using below mentioned code to create ViewQuery…

 String[] params -- keys values which I am passing to view to filter result set.. 
    JsonArray jsonArray = JsonArray.create();
    for (String param : params) {
        jsonArray.add(param);
    }
    ViewQuery query = ViewQuery.from(design, view);
    query.inclusiveEnd(false);
    query.stale(Stale.FALSE);
    query.keys(jsonArray); 

But it is not working… Do I need to pass some encoding parameter or set some configuration parameter?

What is the error that you are getting.?

@vinod how do the keys look like you are emitting and how does the view query generated look like? You can run toString() on it to see the http query URI and compare them.

@shiv4nsh, I am not getting any error, it’s not returning any rows.

@daschl,

toString() is not returning ‘keys’ value.

ViewQuery.toString() : stale=false&inclusive_end=false   

URL captured from ‘couchdb.log

/default/_design/couchbase_service/_view/getUtilizationBasedOnPractice?stale=false&inclusive_end=false&keys=%5B%22DAS%22%5D

Note: If paste above mentioned URL in browser a " symbol will comes automatically.

Am I missing anything here?

@vinod

I meant after that toString on “query”, so that you can see what is actually written to the keys() method!

@daschl, I am calling toString() on VIewQuery object. In toString() implementation it is not adding ‘keys’ value.

LOGGER.debug("$$ Query == " + query.toString());

@vinod maybe your params array is just empty?

@daschl & @shiv4nsh … I have found the problem, in query I was setting in inclusive_end=false, if I make it inclusive_end=true, it’ working.

Thanks a lot for the time you guys put to help me out…

2 Likes

Super glad you found it!

@vinod you are right though, the toString() method on ViewQuery doesn’t output the keys…

Since the keys is the only parameter that can be large enough to make a switch from GET to POST necessary, we didn’t include them in toString to not risk a too large string dump.

However, I think we can at least show a boolean flag in toString that says if the keys is set or not. I created JCBC-838 to track that.

Note that we don’t really keep the list around, but rather directly store is as a JSON string to pass it to the view engine as is, so we can’t conditionally dump it depending on its number of keys…

To more exhaustively dump a ViewQuery, you’d have to call both toString() and getKeys().

:thumbsup: glad you found how to make your query work :slight_smile:

@simonbasle, Thanks Simon for tracking this.

Now I am using both toString() and getKeys() to display my all input parameters.

Having a very similar issue here… inclusiveEnd() seems to help, but I’m basically trying to delete every key that a view returns in a batched process.

I always get the correct value for totalRows(), but I am certainly not getting data back in a consistent manner.

Very often, the rows are empty when data is expected. Making the same query from the command line (taking what the debugger shows for the parameters), of course the data shows up…

ViewQuery query = ViewQuery.from(DD_NAME, VIEW_NAME);
query.development(false);
query.limit(batchSize);
query.inclusiveEnd();
query.stale(Stale.TRUE);
bucket.query(query);

I have tried using skip() to skip the # I’ve deleted and that seemed to help, but that does not reliably account for data that gets inserted while this operation is going on. I’ve also experimented with Stale.UPDATE_AFTER (expecting to not need to use skip), but again, odd behaviors.

I suspect that my issues are mainly around the fact that I’m doing deletions…

Using Stale.FALSE of course makes it all work properly. Certainly not ideal, however.

Had a similar problem - forgot to publish the view and wasted half a day. Thanks for your comment - that sorted it for me!

1 Like