In Android Couchbase Lite mobile push replication is working but pull replication is not working

sunilkumarnitdgp91 · August 13, 2018, 10:50am

Couchbase Lite Mobile version - couchbase-lite-android:2.1.0-DB001.
Syngateway version - 2.0.0-832
Couchbase Server version - 5.5.0-2958
Ubuntu version - 16.04 lts

In Couchbase lite while pull replication this api is getting called internally in log files.

Request URL:http://192.168.1.109:4984/demobucket/_blipsync
whose log traces are given below.

V/WS: WebSocketListener.onOpen() response -> Response{protocol=http/1.1, code=101, message=Switching Protocols, url=http://192.168.1.109:4984/demobucket/_blipsync}
V/WS: receivedHTTPResponse() httpStatus -> 101
I/LiteCore [Sync]: {Repl#1} activityLevel=connecting: connectionState=1
I/WS: CBLWebSocket CONNECTED!
I/LiteCore [BLIP]: {N8litecore4blip10ConnectionE#1} Connected!
I/LiteCore [Sync]: {Repl#1} Connected!
{Repl#1} activityLevel=busy: connectionState=2
{Repl#1} now busy
I/LiteCoreJNI: [NATIVE] C4Replicator.statusChangedCallback() repl -> 0x0xa514a600 status -> 4
I/Sync: C4ReplicatorListener.statusChanged() status -> C4ReplicatorStatus{activityLevel=4, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
I/LiteCore [Sync]: {Repl#1} No local checkpoint ‘cp-uqVQw0XaCQL/giezAhygTpUlIBQ=’
{Repl#1} activityLevel=busy: connectionState=2
I/LiteCore [Sync]: {Pull#2}==> N8litecore4repl6PullerE ->ws:192.168.1.109:4984/demobucket/_blipsync
{Pull#2} Starting pull from remote seq
I/LiteCore [Sync]: {Pull#2} activityLevel=busy: pendingResponseCount=1, _caughtUp=0, _waitingForChangesCallback=0, _pendingRevMessages=0, _activeIncomingRevs=0
{Repl#1} activityLevel=busy: connectionState=2
I/LiteCore [Sync]: {DBWorker#3}==> N8litecore4repl8DBWorkerE ->ws:192.168.1.109:4984/demobucket/_blipsync
{DBWorker#3} activityLevel=idle: pendingResponseCount=0, eventCount=1
{Repl#1} pushStatus=stopped, pullStatus=busy, dbStatus=stopped, progress=0/0
{Repl#1} activityLevel=busy: connectionState=2
{Pull#2} activityLevel=busy: pendingResponseCount=1, _caughtUp=0, _waitingForChangesCallback=0, _pendingRevMessages=0, _activeIncomingRevs=0
{Repl#1} pushStatus=stopped, pullStatus=busy, dbStatus=idle, progress=0/0
{Repl#1} activityLevel=busy: connectionState=2
I/LiteCoreJNI: [NATIVE] C4Replicator.getResponseHeaders() repl -> 0xa514a600
I/Sync: statusChanged() c4Status -> C4ReplicatorStatus{activityLevel=4, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
I/Sync: Replicator[<* Database@7c73201{name=‘couchbasemobile’} URLEndpoint{url=ws://192.168.1.109:4984/demobucket}] is busy, progress 0/0, error: null
V/WS: WebSocketListener.onMessage() bytes -> 0102214572726f722d436f646500343034004572726f722d446f6d61696e0048545450006d697373696e67ec76bdf0
I/LiteCore [Sync]: {Repl#1} No remote checkpoint
{Repl#1} activityLevel=busy: connectionState=2
V/WS: WebSocketListener.onMessage() bytes -> 0201006f536b2c
I/LiteCore [Sync]: {Pull#2} activityLevel=busy: pendingResponseCount=0, _caughtUp=0, _waitingForChangesCallback=0, _pendingRevMessages=0, _activeIncomingRevs=0
V/WS: WebSocketListener.onMessage() bytes -> 0128d20b28ca4fcbcc496548ce48cc4b4f2d6670cecf2b49cd2bd10da92c4865482c28c8c94c4e2cc9cccfd3cf2acecf63c82bcdc9010028b79478
I/LiteCore [Sync]: {Pull#2} Caught up with remote changes
{Pull#2} activityLevel=idle: pendingResponseCount=0, _caughtUp=1, _waitingForChangesCallback=0, _pendingRevMessages=0, _activeIncomingRevs=0
{Repl#1} pushStatus=stopped, pullStatus=idle, dbStatus=idle, progress=0/0
I/LiteCore [Sync]: {Repl#1} activityLevel=idle: connectionState=2
{Repl#1} now idle
I/LiteCoreJNI: [NATIVE] C4Replicator.statusChangedCallback() repl -> 0x0xa514a600 status -> 3
I/Sync: C4ReplicatorListener.statusChanged() status -> C4ReplicatorStatus{activityLevel=3, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
I/Sync: statusChanged() c4Status -> C4ReplicatorStatus{activityLevel=3, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
I/Sync: Replicator[<* Database@7c73201{name=‘couchbasemobile’} URLEndpoint{url=ws://192.168.1.109:4984/demobucket}] is idle, progress 0/0, error: null

and If I try to hit same url ie ://192.168.1.109:4984/demobucket/_blipsync in webbrowser.
Then output is

Request URL:://192.168.1.109:4984/demobucket/_blipsync
Request Method:GET
Status Code:400 Bad Request
Remote Address:192.168.1.109:4984
Referrer Policy:no-referrer-when-downgrade.

And my sync_gateway.json config file is given below

{
“log”:[“CRUD+”, “REST+”, “Changes+”, “Attach+”],
“interface”:“0.0.0.0:4984”,
“adminInterface”:“0.0.0.0:4985”,
“compressResponses”:false,
“databases”: {
“demobucket”: {
“server”:"://127.0.0.1:8091",
“sync”:function (doc) { if (doc) { channel('sumit'); } },
“username”:“sync_gateway”,
“password”:“dost1234”,
“users”: {
“GUEST”: {
“disabled”: true,
“admin_channels”: [“public”]
},
“sync_gateway”:{
“password”:“dost1234”,
“admin_channels”: ["*"]
}
},
“unsupported”: {
“replicator_2”: true
},
“enable_shared_bucket_access”:true,
“import_docs”:“continuous”,
“shadow”: {
“server”: “://127.0.0.1:8091”,
“bucket”: “demobucket”

}

Could someone please help me in pulling data from Couchbase server via syngateway in android app.

borrrden · August 13, 2018, 9:12pm

You cannot access the _blipsync endpoint from a browser, as it needs to start a websocket connection (a browser is not capable of doing that). I don’t see a description of the problem here.

sunilkumarnitdgp91 · August 14, 2018, 10:31am

Hello Borrrden thanks for response,

Here is my android code for pull replication,

try {

Log.i(TAG, " PULLING DATA FROM SERVER *************************************STARTED :: ");
try {
Endpoint targetEndpoint = new URLEndpoint(new URI(“ws://192.168.1.109:4984/demobucket”));
ReplicatorConfiguration replConfigPull = new ReplicatorConfiguration(database, targetEndpoint);
replConfigPull.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PULL);

// Add authentication.
replConfigPull.setAuthenticator(new BasicAuthenticator(“sync_gateway”, “dost1234”));
replConfigPull.setContinuous(true);
// Create replicator.
Replicator replicatorPull = new Replicator(replConfigPull);
replConfigPull.setContinuous(true);
// Listen to replicator change events.
replicatorPull.addChangeListener(new ReplicatorChangeListener() {
@Override
public void changed(ReplicatorChange change) {
if (change.getStatus().getError() != null)
Log.i(TAG, "Error code :: " + change.getStatus().getError().getCode());
}
});

// Start replication.
synchronized (replicatorPull) {
replicatorPull.start();

}

Log.i(TAG, " PULLING DATA FROM SERVER *************************************End :: ");
}catch (URISyntaxException e){}

} catch (Exception e) {
Log.i(TAG, "Error in Pull Function :: ", e);

}
}

Here is the log trace files while pulling data from couchbase server via sync_gateway.

It looks pull replication is working fine, but I couldn’t see updated data in my mobile database

15:53:06.902 32635-32635/com.sunil.couchbaselite2 I/LiteCore [DB]: Opening database /data/user/0/com.sunil.couchbaselite2/files/couchbasemobile.cblite2/db.sqlite3
08-14 15:53:06.905 32635-32635/com.sunil.couchbaselite2 I/Couchbase Lite Database: PULLING DATA FROM SERVER ***********************************STARTED ::
08-14 15:53:06.922 32635-32635/com.sunil.couchbaselite2 I/Sync: Replicator[< Database@761393{name=‘couchbasemobile’} URLEndpoint{url=ws://192.168.1.109:4984/demobucket}]: Starting
08-14 15:53:06.924 32635-32635/com.sunil.couchbaselite2 I/LiteCoreJNI: [NATIVE] C4Replicator.create()
08-14 15:53:06.924 32635-32635/com.sunil.couchbaselite2 I/LiteCore [DB]: Opening database /data/user/0/com.sunil.couchbaselite2/files/couchbasemobile.cblite2/db.sqlite3
08-14 15:53:06.927 32635-32635/com.sunil.couchbaselite2 I/LiteCore [BLIP]: {N8litecore4blip10ConnectionE#1}==> N8litecore4blip10ConnectionE ->ws:192.168.1.109:4984/demobucket/_blipsync
{N8litecore4blip10ConnectionE#1} Opening connection…
08-14 15:53:06.928 32635-32635/com.sunil.couchbaselite2 I/LiteCore [Actor]: Starting Scheduler<0xad77e1c8> with 4 threads
08-14 15:53:06.929 32635-32635/com.sunil.couchbaselite2 I/LiteCore [Sync]: {Repl#1}==> N8litecore4repl10ReplicatorE /data/user/0/com.sunil.couchbaselite2/files/couchbasemobile.cblite2/ ->ws:192.168.1.109:4984/demobucket/_blipsync
{Repl#1} Pull=continuous, Options={“WS-Protocols”:“BLIP_3+CBMobile_2”,auth:{password:“dost1234”,type:“Basic”,username:“sync_gateway”},headers:{“User-Agent”:“CouchbaseLite/2.0.0-457 (Java; Android 7.1.1; Moto E (4) Plus) Build/0 Commit/8dd6674 LiteCore/ (457)”}}
08-14 15:53:06.929 32635-32635/com.sunil.couchbaselite2 I/LiteCoreJNI: [NATIVE] C4Replicator.create() repl -> 0x0xa693d380
08-14 15:53:06.930 32635-32635/com.sunil.couchbaselite2 I/LiteCoreJNI: [NATIVE] C4Replicator.getStatus() repl -> 0xa693d380
08-14 15:53:06.932 32635-32635/com.sunil.couchbaselite2 I/Sync: Replicator[< Database@761393{name=‘couchbasemobile’} URLEndpoint{url=ws://192.168.1.109:4984/demobucket}] is connecting, progress 0/0, error: null
C4ReplicatorListener.statusChanged() status -> C4ReplicatorStatus{activityLevel=2, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
08-14 15:53:06.934 32635-32635/com.sunil.couchbaselite2 I/Couchbase Lite Database: PULLING DATA FROM SERVER ************************************End ::
08-14 15:53:06.934 32635-32635/com.sunil.couchbaselite2 I/DB: CouchbaseLite/2.0.0-457 (Java; Android 7.1.1; Moto E (4) Plus) Build/0 Commit/8dd6674 LiteCore/ (457)
08-14 15:53:06.934 32635-32655/com.sunil.couchbaselite2 I/LiteCoreJNI: [NATIVE] C4Replicator.getResponseHeaders() repl -> 0xa693d380
08-14 15:53:06.934 32635-32655/com.sunil.couchbaselite2 I/Sync: statusChanged() c4Status -> C4ReplicatorStatus{activityLevel=2, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
08-14 15:53:06.935 32635-32655/com.sunil.couchbaselite2 I/Sync: Replicator[< Database@761393{name=‘couchbasemobile’} URLEndpoint{url=ws://192.168.1.109:4984/demobucket}] is connecting, progress 0/0, error: null
08-14 15:53:06.935 32635-32635/com.sunil.couchbaselite2 I/DB: Opening Database@16455ef{name=‘couchbasemobile’} at path /data/user/0/com.sunil.couchbaselite2/files/couchbasemobile.cblite2
08-14 15:53:06.936 32635-32635/com.sunil.couchbaselite2 I/LiteCore [DB]: Opening database /data/user/0/com.sunil.couchbaselite2/files/couchbasemobile.

Please have a look.

borrrden · August 14, 2018, 11:05am

Is the guest user turned on for sync gateway?

sunilkumarnitdgp91 · August 15, 2018, 3:40pm

Yes, No It is turned off,

Here my sync_gateway.json config file.

{
“log”:[“CRUD+”, “REST+”, “Changes+”, “Attach+”,“HTTP+”],
“interface”:“0.0.0.0:4984”,
“adminInterface”:“0.0.0.0:4985”,
“compressResponses”:false,
“databases”: {
“demobucket”: {
“server”:“http://127.0.0.1:8091”,
“sync”:function (doc) { if (doc) { channel(doc.channel); } },
“username”:“Administrator”,
“password”:“dost1234”,
“users”: {
“GUEST”: {
“disabled”: true,
“admin_channels”: [“public”]
},
“sync_gateway”:{
“password”:“dost1234”,
“admin_channels”: [“sumit”]
}

        },
   "unsupported": {
    	"replicator_2": true
  	    },
    "enable_shared_bucket_access":true,
        "import_docs":"continuous",
    "shadow": {
          "server": "http://127.0.0.1:8091",
          "bucket": "demobucket"
          
        }

    }
}

}

borrrden · August 22, 2018, 10:59am

Looks like the config has changed since last time. Is ‘sumit’ present in the channels property of any docs? If not you won’t have access to pull them.

sunilkumarnitdgp91 · August 23, 2018, 9:08am

Yes, I do have alots of documents in couchbase server including ‘sumit’ as channel
for eg {
“channel”: “sumit”,
“language”: “Java”,
“type”: “SDK”,
“version”: 2,
“_rev”: “2-ca8b6b1f6928fec812e3e9462628b4271c77f0c7”,
“_id”: “433d97cc-cb4a-4815-a585-ec96befee82d”
}

But still such data is not replicating to my android app. But if I save any data from android app it is getting replicated in couchbase server.

Just for your reference I’m sharing some code of android app where pull replication code is written. Do I need to set channel some where in pull replication mechanism?

private static void pullDataFromSrver(Database database) {
try {

        Log.i(TAG, " PULLING DATA FROM SERVER *************************************STARTED ::  ");
        try {
            Endpoint targetEndpoint = new URLEndpoint(new URI("ws://192.168.1.109:4984/demobucket"));
            ReplicatorConfiguration replConfigPull = new ReplicatorConfiguration(database, targetEndpoint);
            replConfigPull.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PULL);

            // Add authentication.
            replConfigPull.setAuthenticator(new BasicAuthenticator("sync_gateway", "dost1234"));
            replConfigPull.setContinuous(true);
            // Create replicator.
            Replicator replicatorPull = new Replicator(replConfigPull);
            replConfigPull.setContinuous(true);
            // Listen to replicator change events.
            replicatorPull.addChangeListener(new ReplicatorChangeListener() {
                @Override
                public void changed(ReplicatorChange change) {
                    if (change.getStatus().getError() != null)
                        Log.i(TAG, "Error code ::  " + change.getStatus().getError().getCode());
                }
            });

            // Start replication.
            synchronized (replicatorPull) {
                replicatorPull.start();

            }

            Log.i(TAG, " PULLING DATA FROM SERVER *************************************End ::  ");
        }catch (URISyntaxException e){}

    } catch (Exception e) {
        Log.i(TAG, "Error in Pull Function ::  ", e);

    }
}

borrrden · August 23, 2018, 9:53am

No, you shouldn’t have to set the channel. The default is to pull all documents that the user has access to. if you directly query the _changes endpoint in the browser (http://192.168.1.109:4984/demobucket/_changes) do you see any entries there?

sunilkumarnitdgp91 · August 23, 2018, 10:02am

This is What shown on in webbrowser after hitting above url.

{“results”:[
{“seq”:2,“id”:"_user/sync_gateway",“changes”:[]}
,{“seq”:65,“id”:“2dd3d423-9cd9-4ac2-8afc-29aecc70aa8a”,“removed”:["!"],“changes”:[{“rev”:“4-a438bce6a589975dd325ba1239c3c390”}]}
],
“last_seq”:“65”}

borrrden · August 23, 2018, 10:03am

Then it looks like there is something going wrong with your sync function (you don’t have access to view any changes). Have you changed it recently? @bbrks Is there a procedure to follow if you change the config?

sunilkumarnitdgp91 · August 23, 2018, 10:21am

I’ve already shared sync_gateway.config file in my first text. From my end sync_gateway.config file is correct. It would be great If you could help me out to get the bottel neck of issue

borrrden · September 1, 2018, 11:58pm

Oh, I had another look at the config. You don’t need to (and should not) use any bucket shadowing, and instead you should be hitting ws://192.168.1.109:4984/sync_gateway which is your main sync gateway bucket.