Node SDK - Streaming Rows - Error Stops Streaming

Node.js SDK
1

Greetings …

I am working on a Node batch process that uses a N1QL query to stream row data to a SQL database and a CSV (for back-up purposes). According to the Node 2.6 SDK Documentation, “If you are expecting a large result set from the server you can omit passing the callback to the query() function and instead register to the returned object’s row , end , and error events.”

However, based on my local testing, if an error event is received no further rows are processed. What is the best to either restart or continue processing row information? One other note: the current code is using the N1QL example shown in the documentation:

req.on('row', function(row) {
  console.log('Got a row');
});
req.on('error', function(err) {
  console.error('Got error %j', err);
});
req.on('end', function(meta) {
  console.log('All rows received. Metadata is %j:', meta);
});
2

Hello @TerrenceMM2 I tried the code and I couldnt simulate an error to test your scenario.
How did you simulate an error ? I even did with 10k rows but was never successful in error-ing out.


var couchbase = require('couchbase')
var cluster = new couchbase.Cluster('couchbase://localhost/');
cluster.authenticate('Administrator', 'password');
var bucket = cluster.openBucket('travel-sample');
var N1qlQuery = couchbase.N1qlQuery;
var q = N1qlQuery.fromString('SELECT * FROM `travel-sample` LIMIT 100');
var req = bucket.query(q);
req.on('row', function(row) {console.log('Got a row');});
req.on('error', function(err) {console.error('Got error %j', err);});
req.on('end', function(meta) {console.log('All rows received. Metadata is %j:', meta);});

Regardless I would also like to say that 2.x has reached end of full maintenance so would recommend you to upgrade to 3.x as soon as you can.

1 Like
3

Thank you, @AV25242,

I noticed that later versions of the Node.js SDK were available. I am currently working to refactor the current (legacy) code to use SDK v3.1.3.

Do you know if streaming data is available in SDK v3+ as noted in the 2.6 documentation? I couldn’t find similar documentation/functionality.

4

This feature will be available in NodeJS SDK 3.2 which will be the next minor release of our SDK. Within a month. Will work exactly the same as SDK 2x

I am seeing what I can do in order to get this feature documented as it is in our SDK 2x docs. You can use the new SDK 3.2 but you would have to build the binaries locally until we release version 3.2 I can get you instructions on how to do that.

Can you let me know what OS you use on your dev machine, this will assist me in providing you the right instructions unless you would like to wait for the SDK 3.2 release instead.