Error Using ElasticSearch Connector 4.0

Hi,

I have been tried to use ES Connector 4.0, as you know it was changed from plugin to a standalone process,
My version 3 plugin works perfectly, but i have a huge trouble with ES 4.0 connector. I configured it exactly what is said on Couchbase Website but i am getting the error below everytime i tried to start ./cbes. i tried it even default bucket, my bucket size 30-40 mb (for testing) but when i check ES indices, only 1.2kb of my data is syncing. Nothing more. And my error is ;

ERROR c.c.c.e.i.ElasticsearchWriter - Failed to index rejection document for event MUT:1016/76404134638339@1[0-22]=xxx; status code: BAD_REQUEST ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=Rejecting mapping update to [cbes-rejects] as the final mapping would have more than 1 type: [_doc, couchbaseCheckpoint]]]

There are hundres of the same error. How can i get rid of this ?

Thank you for your help.

Regards.

Hi Oguzhan,

Short answer: Delete the Elasticsearch indexes, then re-stream everything from Couchbase by running cbes-checkpoint-clear (make sure the connector is stopped what you run that command).

If you want to preserve the contents of the indexes, or you’re interested in the details, here’s the long answer:

In Elasticsearch 6 and later only one document type is allowed per index. In Elasticsearch 5.6 you can preview this behavior by setting index.mapping.single_type to true for an index.

Your cbes-rejects index already has a mapping type called couchbaseCheckpoint, so the connector is unable to create a document of type _doc. The same may be true of the other indexes where you want the actual Couchbase documents to go.

The couchbaseCheckpoint type is metadata used by the old version of the connector. You don’t need it anymore. The old connector also used a couchbaseDocument mapping type for the documents themselves.

Unfortunately, it’s not possible to delete a mapping type from an index. The Elasticsearch documentation recommends deleting the index and recreating it without the offending mapping type. It may be possible to use the reindex API to preserve any documents you want to keep, and to assign a new mapping type to those documents. For forward compatibility, the recommended type name is _doc.

If you’re happy to let the new connector re-stream everything from Couchbase, just delete the cbes-rejects index (and any other indexes targeted by the new connector) to get rid of the obsolete couchbaseCheckpoint and couchbaseDocument mappings types.

If you’re upgrading to Elasticsearch 6 and connector version 4 at the same time and you want to preserve the contents of your indexes, the best path is probably to upgrade the connector first, then reindex so you have one mapping type per index, and finally upgrade to Elasticsearch 6.

I’m afraid the connector’s migration guide glosses over these details. That’s something we should address.

Thanks,
David