Retrieve all bucket documents from Couchbase server


#1

I am trying to create view on couch base server and retrieve all the bucket document keys from the server for my bucket. I am able to create a view but failed to get the document keys from the server.

Have you any inputs?

Thanks
Amit


#2

Some code would be helpful. Can you paste the code you are working with?


#3

I am new to couchbase. I am using following code to create document view and get the document keys from couchbase server. I am using C SDK 2.5 and couchbase server 2.2

bool getBucketKeysFromCouchbase(lcb_t instance, char* bucketname)
{
lcb_error_t err;
struct lcb_create_st create_options;

(void)lcb_set_error_callback(instance, error_callback);

(void)lcb_set_http_complete_callback(instance, complete_callback);
(void)lcb_set_http_data_callback(instance, data_callback);
/* Run the event loop and wait until we've connected */
lcb_wait(instance);

{
    const char viewname[] = "_design/getdocumentid";
    const char view[] =   "{\"views\":{\"getdocumentid\":{\"map\":\"function (doc, meta) { emit(meta.id, meta.id); }\"}}}";

    lcb_http_cmd_t cmd;
    lcb_http_request_t req;
    lcb_error_t err;

    cmd.version = 0;
    cmd.v.v0.path = viewname;
    cmd.v.v0.npath = strlen(viewname);
    cmd.v.v0.body = view;
    cmd.v.v0.nbody = strlen(view);
    cmd.v.v0.method = LCB_HTTP_METHOD_PUT;
    cmd.v.v0.content_type = "application/json";
    err = lcb_make_http_request(instance, NULL, LCB_HTTP_TYPE_VIEW, &cmd, &req);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to schedule HTTP request: %s\n", lcb_strerror(instance, err));
        lcb_destroy(instance);
        return 1;
    }

    lcb_wait(instance);
}

lcb_CMDVIEWQUERY vq = { 0 };

//vq.cmdflags |= LCB_CMDVIEWQUERY_F_INCLUDE_DOCS;

lcb_view_query_initcmd(&vq, "getdocumentid", "_design/getdocumentid ", NULL, viewCallback);
    lcb_error_t rc = lcb_view_query(instance, NULL, &vq);
if (rc != LCB_SUCCESS) {
    // Handle error
}

lcb_wait(instance);
lcb_destroy(instance);

}

void error_callback(lcb_t instance, lcb_error_t error, const char *errinfo)
{
fprintf(stderr, “ERROR: %s (0x%x), %s\n”, lcb_strerror(instance, error), error, errinfo);
exit(EXIT_FAILURE);
}

void complete_callback(lcb_http_request_t req, lcb_t instance,
const void *cookie, lcb_error_t error,
const lcb_http_resp_t *resp)
{
size_t ii;

if (resp->v.v0.headers) {
    const char *const *headers = resp->v.v0.headers;
    for (ii = 1; *headers != NULL; ++ii, ++headers) {
        fprintf(stderr, "%s%s", *headers, (ii % 2 == 0) ? "\n" : ": ");
    }
}
fprintf(stderr, "\"");
fwrite(resp->v.v0.path, sizeof(char), resp->v.v0.npath, stderr);
fprintf(stderr, "\": ");
if (error == LCB_SUCCESS) {
    fprintf(stderr, "OK Size: %d", (int)resp->v.v0.nbytes);
    fwrite(resp->v.v0.bytes, sizeof(char), resp->v.v0.nbytes, stdout);
} else {
    fprintf(stderr, "FAIL(%d) %s Status:%d Size:%d\n",
            error, lcb_strerror(instance, error),
            resp->v.v0.status, (int)resp->v.v0.nbytes);
}
fwrite(resp->v.v0.bytes, sizeof(char), resp->v.v0.nbytes, stdout);
fprintf(stderr, "\n");
(void)req;
(void)cookie;

}

void viewCallback(lcb_t instance, int ign, const lcb_RESPVIEWQUERY response)
{
if (response->rflags & LCB_RESP_F_FINAL) {
printf("
** META FROM VIEWS ***\n");
fprintf(stderr, “%.*s\n”, (int)response->nvalue, response->value);
return;
}

printf("Got row callback from LCB: RC=0x%X, DOCID=%.*s. KEY=%.*s\n", response->rc,
    (int)response->ndocid, response->docid,
    (int)response->nkey, response->key);

if (response->docresp) {
    printf(" Document for response. RC=0x%X. CAS=0x%lx\n",
        response->docresp->rc, response->docresp->cas);
}

}


#4

In lcb_view_query_initcmd(), the view should be the simple name of the view, and the design should be the name of the design document without the _design/ prefix, in your case they should both be getdocumentid.

I know you’re just experimenting, but you don’t need to emit the document ID since that is always returned in the view result anyway. See the docid field http://docs.couchbase.com/sdk-api/couchbase-c-client-2.5.2/group__lcb-view-api.html#a8556878012feffc9e0beb86cd78f424d


#5

In lcb_view_query_initcmd(), I updated design document without the _design/ prefix but it didnt worked. I received following error
{“error”:“method_not_allowed”,“reason”:“Only GET,POST,HEAD allowed”}.ids got a smint?)…

Also I looked into the url you provided. By the way I have a question that Document ID (i.e. memcached key) associated with this row is as a input field? Because I want all the document id’s for my bucket documents. Also is this work only with memcached keys?


#6

Ok, so here’s the idea. lcb_make_http_request() is a fairly low-level API, and you are required to use the full endpoint you wish to contact. It’s pretty much a dumb http client like curl, etc. Because the C library doesn’t have a high level api (yet?) to create design documents, you need to specify the design there with the _design prefix.

When you query the view however, you can use the higher level lcb_view_query() which abstracts the various paths for the design document and the view. So while you need the _design/ prefix when creating the view, you don’t need it (and in fact, should omit it) when querying the view.

Of course you’re free to use the raw lcb_make_http_request() to issue the view query yourself, and then you can use the full path API, however I would recommend sticking to the higher level API instead.


#7

Thanks @mnunberg I resolved the issue. By connecting to port 8092 I am able to retrieve the data.

But I have a quick question, if I want to delete some documents on which port I have to connect port 8091 or 8092?

Thanks
Amit Kulkarni


#8

I am unsure what you mean here. By design the specific port applications should connect to are hidden by the SDK. You shouldn’t specify explicit ports or hosts anywhere in your code. You should only specify an explicit host (or set of hosts) during the initial bootstrap when you pass them in the connection string. These hosts are the bootstrap hosts, and any one of these hosts will be used during the initial lcb_connect() call to gather information about the rest of the cluster. This will give the SDK the information about which services are available on the cluster, and what their locations (host and port) are.


#9

You are right @mnunberg, means when I connected on port 8092 during the initial bootstrap. I am able to get data all the document keys.

I have one more question. I am creating document in couch base using lcb_store and retrieving the data using lcb_get. While setting I am setting expiration time (ttl) .

lcb_store_cmd_t scmd;
const lcb_store_cmd_t *scmdlist = &scmd;
scmd.v.v0.key = bucketID;
scmd.v.v0.nkey = strlen(bucketID);
scmd.v.v0.bytes = data;
scmd.v.v0.nbytes = 9;
scmd.v.v0.operation = LCB_SET;
scmd.v.v0.exptime = 300; 

but when I am retriving data using lcb_get I am able to get the expiration date which I set for the document.
The code to retrieve data is

lcb_get_cmd_t gcmd = NULL;
const lcb_get_cmd_t *gcmdlist = &gcmd;
gcmd.v.v0.key = bucketID;
gcmd.v.v0.nkey = strlen(bucketID);
err = lcb_get(instance, NULL, 1, &gcmdlist);
if (err == LCB_SUCCESS) {
returnStatus = true;
} else {
printf(“Couldn’t schedule get operation!\n”);
returnStatus = false;
}

Have you know how to retrieve document expiration information?


#10

I’m glad you have it working out for you. Note it is not possible to “bootstrap” off port 8092. Your connection string should typically be couchbase://host/bucket, using couchbase://host:port/bucket only for specific scenarios (yours not being one of them), however since you have it working, I would consider this explanation academic for the time being.

You cannot directly retrieve the document expiration. There is something called keystats (see the LCB_CMDSTATS_F_KV flag in lcb_CMDSTATS and lcb_stats3() - however that API has some limitations. If possible, design your application in a way where you do not need to know the expiration time beforehand, or embed it into your document.