XDCR failed for replication from 2.1.1 to 2.2.0

I’m trying to replicate my 2.1.1 cluster data bucket to different cluster with 2.2.0 that I setup for development. When I tried to XDCR the data, this is the constant error I got.

Error replicating vbucket 107: {badmatch, {error, all_nodes_failed, <<“Failed to grab remote bucket info from any of known nodes”>>}}

Centos 6.4 64 Bit

All necessary ports are open. I even tried to turn off the firewall and I’m still having a problem. Please advise on where I should start to debug this issue? Many thanks.

Here are the ports that I opened.

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8091 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8092 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 11209:11211 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 4369 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 21100:21199 -j ACCEPT

Few questions that can help us answer you better -

  1. Was the replication between source and destination bucket established immediately after bucket creation? Ideally, waiting for a minute after bucket creation is recommended before setting up XDCR. If you are not sure, deleting the replication and recreating it might help.

  2. Is the replication still happening despite the error?

  3. Can you confirm that the replication is only uni-directional from the cluster running 2.1.1 to the one running 2.2.0? If you also have a replication set up between 2.2.0 to 2.1.1, please make sure you chose version1 against XDCR protocol at the time of setting up replication. We changed the default protocol(from version1 to version2) in 2.2.0 so 2.1.1 which still uses version1 protocol for replication does not understand version2. So the source needs to replicate using version1. You cannot change this setting once the replication is created but you can delete and recreate the replication if this is the case. However if you are not replicating from 2.2.0 to 2.1.1, you do not have to worry about this.

If 1 and 3 don’t help, we will need your logs to diagnose the problem. This link will help help you through collecting and uploading logs using cbcollect_info tool that can be found in the bin directory -

http://www.couchbase.com/docs//couchbase-manual-2.0/couchbase-admin-cmdline-cbcollect_info.html

For diagnosis, logs from both source and destination cluster nodes are required. Thanks!