Inconsistent View Results

I am running into an issue where query results from a view are inconsistent when using the Java Couchbase Client 1.1. I run the same query directly in a browser and I get consistent results. I have run the query on each of the three nodes in the cluster; debugging shows that each node has returned inconsistent results through the java client, but in a browser the results are consistent for each node.

Here is the map:

function (doc, meta) {
var data = doc.data;
if(data) {
emit([data.company_number, data.status], {
applicationNumber: data.application_number,
formId: data.id,
status: data.status});
}
}

And the reduce:

function(keys, values, rereduce) {
var index;
var maxApp = {applicationNumber:""};
for (index in values) {
if (values[index].applicationNumber > maxApp.applicationNumber) {
maxApp.applicationNumber = values[index].applicationNumber;
maxApp.formId = values[index].formId;
maxApp.status = values[index].status;
}
}
return maxApp;
}

The unreduced results are:

{“total_rows”:89,“rows”:[
{“id”:“2b4325a8-0d12-11e3-9728-180373e4f256”,“key”:[30951,“APPROVED”],“value”:{“applicationNumber”:“30951-00001”,“formId”:“2b4325a8-0d12-11e3-9728-180373e4f256”,“status”:“APPROVED”}},
{“id”:“460_848934560”,“key”:[30951,“APPROVED”],“value”:{“applicationNumber”:“30951-00002”,“formId”:“460_848934560”,“status”:“APPROVED”}}
]
}

Sometimes the java client returns the “30951-00001” application number incorrectly, and other times the “30951-00002” correctly - as if sometimes it doesn’t know about the “30951-00002” row.

Any ideas?

The execution path is slightly different for the two, but mostly common. This is a bit surprising. I would recommend adding the debug parameter to each path and compare what the cluster logs on each node during execution of the view if through the browser and through the client library to get more info.

Hello,

Were you able to identify the source of the problem?
Have you tested with debug enabled?

Regards
Tug
@tgrall

No, I never figured it out or ran it in debug. Because of this problem and because client.getView is slow to open connections, I instead created my own client for getting data from views.

Hmm, surprised by this do you mind sharing the code you have written with us?

Sure. It is basically a client to access the REST API that is available from couchbase.

public class ViewClient {
private static final String PROTOCOL = “HTTP”;
private static final String METHOD = “GET”;
private static final int PORT = 8092;
private static final int RESPONSE_RANGE = 100;
private static final int OK_RANGE = 2;
private static final String ROWS = “rows”;
private final SerializerFactory serializerFactory;
private final SystemProperties systemProperties;
@Logger
private Log log;

public ViewClient(SerializerFactory serializerFactory, SystemProperties systemProperties) {
this.serializerFactory = serializerFactory;
this.systemProperties = systemProperties;
}

public List getResultList(View view, Query query, Class returnClass) {
List urls = loadUrls(view, query);
try {
HttpURLConnection connection = loadConnection(urls);
Iterator results = getJsonResults(connection);
return deserializeResults(results, returnClass);
} catch (IOException e) {
throw new HcfRuntimeException(“Error connecting to couchbase”, e);
}
}

private List deserializeResults(Iterator results, Class returnClass) {
List deserializedResults = new LinkedList();
Deserializer deserializer = serializerFactory.getDeserializer(returnClass);
while (results.hasNext()) {
String rowValue = results.next().getAsJsonObject().get(“value”).toString();
T result = deserializer.deserialize(rowValue);
deserializedResults.add(result);
}
return deserializedResults;
}

private Iterator getJsonResults(HttpURLConnection connection) throws IOException {
JsonObject jsonResult = new JsonParser().parse(getResult(connection)).getAsJsonObject();
JsonArray rows = jsonResult.get(ROWS).getAsJsonArray();
return rows.iterator();
}

private List loadUrls(View view, Query query) {
List urls = new LinkedList();
for (String host : loadHosts()) {
String uri = view.generateUri();
try {
urls.add(new URL(PROTOCOL, host, PORT, uri + query.toString()));
} catch (MalformedURLException e) {
log.warn(“Problem loading URL”, e);
}
}
return urls;
}

private List loadHosts() {
List hosts = new LinkedList();
for (URI uri : systemProperties.getCouchbaseUris()) {
hosts.add(uri.getHost());
}
return hosts;
}

private HttpURLConnection loadConnection(List urls) throws IOException {
HttpURLConnection connection = null;
for (URL url : urls) {
try {
connection = makeConnection(url);
break;
} catch (SocketTimeoutException e) {
log.warn(“Connection to couchbase could not be made. Retrying…”, e);
continue;
}
}
return connection;
}

private HttpURLConnection makeConnection(URL url) throws IOException {
HttpURLConnection connection = null;
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod(METHOD);
String unencryptedCredentials =
systemProperties.getCouchbaseUsername() + “:” + systemProperties.getCouchbasePassword();
String credentials = new String(new Base64().encode(unencryptedCredentials.getBytes()));
connection.setRequestProperty(“Authorization”, "Basic " + credentials);
connection.connect();
return connection;
}

private String getResult(HttpURLConnection connection) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = “”;
StringBuilder sb = new StringBuilder();
while ((line = reader.readLine()) != null) {
sb.append(line).append("\r");
}
if (responseIsValid(connection.getResponseCode())) {
return sb.toString();
} else {
throw new HcfRuntimeException(“Invalid response: [” + connection.getResponseCode() + "] - " + sb.toString());
}
}

private boolean responseIsValid(int responseCode) {
return (responseCode / RESPONSE_RANGE) == OK_RANGE;
}
}

It should be noted that View is not the CouchbaseClient View class. It just holds the name of the document and view in order to build the path to the view REST resource. I don’t think SerializerFactory and SystemProperties need an explanation but if anything does need an explanation let me know.

I bet it has something to do with the fact that you didn’t distinguish between rereduce false and true situations