Libevent io plugin on windows


#1

Can I use libevent io plugin on windows. If yes how to enable it?


#2

Are you referring to use with libcouchbase? Could you describe a bit more what you’d like to do?


#3

Yes @ingenthr I am using libcouchbase C SDK client with couch base server 3.0. I would like to achieve callback using events and not lcb_wait on windows platform. By default libevent plugin is not present on windows and even throwing error while tried to add the configuration (./configure enable-embedded-libevent-plugin). So please tell me which is the best way to achieve this?


#4

I believe not using the lcb_wait() and the choice of event backend are orthogonal. You’ll still need an event backend.

To solve your compilation problem, I think you’ll need to get a source distribution of libevent if that’s the backend you want to use. On windows, I believe it will use IOCP by default and you don’t need to pass anything to configure. @mnunberg may be able to confirm.

Generally speaking, you’ll want to implement the interface in iops.h. That’s a volatile interface, as covered in the docs.


#5

On Windows libcouchbase does indeed use IOCP by default, while an option is also available for using select().

The libevent plugin has not been tested on Windows. It may or may not work on Windows, likely you would have to change some #include s and #ifdefs, but it should otherwise work.

I am unclear on what type of “events” you are expecting to receive. Do you mean Windows events (as in the Windows Event API (https://msdn.microsoft.com/en-us/library/windows/desktop/ms686915(v=vs.85).aspx) or do you simply mean your application is non-blocking and is already using libevent?


#6

I just want my application non-blocking and I am free to use any io plugin. But I noticed that there is no code for libevent and libuv comes with libcouchbase. Is libcouchbase supports libevent and libuv on Windows? Or select is the only option ?

Also is there any samples for libuv and select calls?


#7

Of the I/O plugins in libcouchbase, the select and IOCP plugins are self-contained, meaning they are not intended for integration with other applications in a non-blocking manner.

The libev, libevent and libuv plugins are intended to allow integration with other applications performing I/O other than libcouchbase.

libuv is indeed supported and tested on Windows. A small example of its usage (which isn’t really an example per-se): https://github.com/couchbase/libcouchbase/blob/master/plugins/io/libuv/libuv_io_opts.h#L54 and another one: https://github.com/couchbase/couchnode/blob/master/src/binding.cc#L140

It is also possible to write your own I/O backend (that is what the Python SDK does, for example)


#8

Thanks @mnunberg for the quick reply. I am using libuv as a io plugin and able to create an instance, and connect to bucket. I am using following code for that.

options.v.v0.loop = uv_default_loop();
options.v.v0.startsop_noop = 1;

lcb_error_t error = lcb_create_libuv_io_opts(0, &ioops, &options);
if (error != LCB_SUCCESS) {
    return false;
}

char connectionString[BUFFLEN_LARGE] = {'\0'};
memset( connectionString, 0, sizeof(connectionString) );
GLSsnprintf(connectionString, sizeof(connectionString), "couchbase://%s/%s%s", hostname, bucketname, "?bootstrap_on=cccp");
       
struct lcb_create_st creatOptions;
memset(&creatOptions, 0, sizeof (creatOptions));
creatOptions.version = 3;
creatOptions.v.v3.connstr = connectionString;
creatOptions.v.v3.username = bucketUser;
creatOptions.v.v3.passwd = bucketPassword;
creatOptions.v.v3.io = ioops;
creatOptions.v.v3.type = LCB_TYPE_BUCKET;

error = lcb_create(&this->instance, &creatOptions);
if (error != LCB_SUCCESS) {
    return false;
}

// Register us as the thread callback
threadObject.Register(this);

// Start the thread
threadObject.StartThread(); 

lcb_set_configuration_callback(instance, onConfigCallback);
lcb_set_bootstrap_callback(instance, onBootstrapCallback);
lcb_set_error_callback(instance, onErrorCallback);
error = lcb_connect(instance);
if (error != LCB_SUCCESS ) {
    return false;
}

void CouchbaseClientWrapper::onErrorCallback(lcb_t instance, lcb_error_t error, const char *errinfo)
{
    if (errinfo) {
        cout << "Error: " << errinfo << endl;
    }
    return;
}

void CouchbaseClientWrapper::onConfigCallback(lcb_t instance, lcb_configuration_t config)
{
    if(config != LCB_CONFIGURATION_NEW) {
        cout << "Error: Failed to receive configurations" << endl;
    }else {
        cout << "Success: Received new configurations" << endl;
    }

    return;
}

void CouchbaseClientWrapper::onBootstrapCallback(lcb_t instance, lcb_error_t error)
{
    if(error != LCB_SUCCESS) {
        cout << "Error: " << error << endl;
    } else {
        cout << "Success: Received bootstrap callback" << endl;
    }

    return;
}

Once the connection is done I application calls lcb_get to get data from the couch base cluster.

lcb_set_get_callback(instance, onGetCallback);
lcb_get_cmd_t cmd = NULL;
const lcb_get_cmd_t *cmdlist[] = { &cmd };
cmd.v.v0.key = documentKey.c_str();
cmd.v.v0.nkey = documentKey.length();
cmd.v.v0.lock = true;

lcb_error_t error = lcb_get(instance, &clientData, 1, cmdlist);

But lcb_get fails with error as LCB_CLIENT_ETMPFAIL.

Have you any inputs why it is failing even after successful connection?


#9

Also I am using following function to run default uv loop in a separate thread

void CouchbaseClientWrapper::Run()
{
bool exisThread = false;
if ( !isShuttingDown() )
{
uv_run(uv_default_loop(), UV_RUN_DEFAULT);
}

return;

}


#10

Do not use lcb_set_error_callback or lcb_set_configuration_callback. These two are older deprecated APIs and are not particularly useful. (Can you tell me where you found out about them, since they have been deprecated for almost two years).

The connection is done only when the bootstrap callback (lcb_set_bootstrap_callback) indicates that it was successful. lcb_connect() merely schedules the connection.


#11

lcb_set_error_callback and lcb_set_configuration_callback are get it from one of the example in couchbase client C SDK.

Anyway I will remove these callbacks But In my case bootstrap callback (lcb_set_bootstrap_callback) indicates it is successful. andstill lcb_get fails with error as LCB_CLIENT_ETMPFAIL.

Have you any inputs?


#12

What is the sequence of events that you’re using? Note that before the bootstrap callback is invoked, you cannot schedule any operations


#13

As mentioned in the code in my previous post I am using a following sequence of code

  1. lcb_create_libuv_io_opts
  2. lcb_create
  3. lcb_set_bootstrap_callback
  4. lcb_connect

Once the lcb_connect is complete & bootstrap callback (lcb_set_bootstrap_callback) indicates that it was successful

  1. lcb_get

if you need complete code I can send it to you.


#14

Yes, some code (or better yet, some code I can actually reproduce and run) would be good.

Have you looked into our libevent example? while this does use libevent and not libuv, the use concept should be the same:

(note, you need libcouchbase >= 2.5.6 to make this example work without any modifications).

(Looking at your code agian, I also hope and assume that CouchbaseClientWrapper::onBootstrapCallback is a static function!)


#15

Yes @mnunberg I am using onBootstrapCallback as a static function.

Also I tried with latest 2.5.6 version of C SDK library but response is same. Also I observed that some times there is no response from the server not sure why?

here i am sharing a link of my code used as a wrapper classes to call C SDK library functionality for your reference.

https://drive.google.com/file/d/0B1gOZhJ_E8TLOS1GZkdHS2JaYXM/view?usp=sharing

if you have any inputs please let me know.


#16

For this I am using C SDK client 2.5.6 and couchbase server 3.1.3 on windows platform


#17

Thank you for sharing your code. It would seem you’re still using the synchronous programming model; furthermore it seems you’re never actually waiting anywhere for the bootstrap process to complete - more to the point, if you were running a non-blocking callback driven model, your bootstrap callback (I thought it was just stub code, but now I see the callback indeed does nothing) would signal to somewhere else in your application that the client is connected, and it is therefore now safe to start scheduling operations.

If you need to use synchronous code, then the actual I/O plugin does not matter (because the event loop is self-contained) and you should be using lcb_wait.


#18

Actually I am sending request for any scheduled operation only after lcb_connect is successful. which means my another application send an request to get document or store document only after I am seeing a successful connetion. In my case which I am logging in a log file for temporary purpose but planned to set an event in call function and check for set event in main function for each operation. But my problem is even I am sending get document request after seeing successful message in a log file I am getting error on lcb_get call which is LCB_CLIENT_ETMPFAIL


#19

What exactly are you seeing in the logs?


#20

I was get an message as “In onBootstrapCallback Success” which is printed in onBootstrapCallback of the library callback function.

Anyway I developed a small executable which basically connect to the cluster and get the document data from the cluster. I am hereby sharing a code and built binary for the same.

https://drive.google.com/open?id=0B1gOZhJ_E8TLUkpXc0p2VWh5QWs

I am using same io plugin (libuv) and interested only in asynchronous mechanism.

In this case I never able to connect to cluster as callback never gets called and hence failed to connect.
Not sure why?
I am able to connect to the same cluster using another library (which is in-house developed by a team) but failed using libcouchbase C SDK with libuv io plugin.