Replication Crashing after turning off Internet CBL 2.0

Hi,

I want to stop my continuous replication when replication is offline. But even on calling replicator.stop() method i am continuously getting replication changes …

I used following method to create replication:-

public void startReplication(Database database) {

    URI uri;
    try {
        uri = new URI(APPOINTMENT_SYNC_URL);
    } catch (URISyntaxException e) {
        e.printStackTrace();
        return;
    }
    ArrayList<String> channelList = new ArrayList<>();
    channelList.add("xyz");
    ReplicatorConfiguration config = new ReplicatorConfiguration(database, uri);
    config.setReplicatorType(ReplicatorConfiguration.ReplicatorType.PUSH_AND_PULL);
    config.setChannels(channelList);
    config.setAuthenticator(new BasicAuthenticator("username", "password"));
    config.setContinuous(true);
    config.setConflictResolver(getConflictResolver());

    // authentication

    replicator = new Replicator(config);
    replicator.addChangeListener(new ReplicatorChangeListener() {
        @Override
        public void changed(ReplicatorChange change) {
            if (change.getReplicator().getStatus().getActivityLevel().equals(Replicator.ActivityLevel.IDLE)) {
               
                Log.e("Replication Comp Log", "Schedular Completed");

            }
            if (change.getReplicator().getStatus().getActivityLevel().equals(Replicator.ActivityLevel.STOPPED) || change.getReplicator().getStatus().getActivityLevel().equals(Replicator.ActivityLevel.OFFLINE)) {
                stopReplication();
                Log.e("Rep schedular  Log", "ReplicationTag Stopped");
            }
        }
    });

    replicator.start();
}

And following to stop replication:-

public void stopReplication() {
        replicator.stop();
}

But replication is not stopping and its retring again and again. I am getting below mentioned log for the replication:-

 E/Sync: Replicator[<*> blip://xx.xx.xx.xx:4984/case]: Retrying...
12-11 19:32:03.293 6727-6727/com.motionmagic.beta E/Sync: Replicator[<*> 
 blip://xx.xx.xx.xx:4984/case] is connecting, progress 0/0, error: null
12-11 19:32:03.293 6727-6727/com.motionmagic.beta E/Sync: statusChanged() c4Status -> 
C4ReplicatorStatus{activityLevel=2, progressUnitsCompleted=0, progressUnitsTotal=0, 
progressDocumentCount=0, errorDomain=0, errorCode=0, errorInternalInfo=0}
12-11 19:32:03.293 6727-6745/com.motionmagic.beta E/C4Socket: C4Socket.callback.open() 
socket -> 0x7f8ffb3790, scheme -> blip, hostname -> xx.xx.xx.xx, port -> 4984, path -> 
/case/_blipsync
12-11 19:32:03.294 6727-6727/com.motionmagic.beta E/Sync: Replicator[<*> 
blip://xx.xx.xx.xx:4984/case] is connecting, progress 0/0, error: null
12-11 19:32:03.294 6727-6745/com.motionmagic.beta E/C4Socket: optionsFleece: not null
12-11 19:32:03.294 6727-6745/com.motionmagic.beta E/C4Socket: options = {channels=
[doctor_46bab161-03c5-4a67-8ee0-3ec318ecfc7e], auth={username=username, type=Basic, 
password=xxxxxxx}, headers={User-Agent=CouchbaseLite/2.0.0-180 (Java; Android 6.0; 
Lenovo TB3-850F) Build/0 Commit/5ccb746}}
12-11 19:32:03.660 6727-7045/com.motionmagic.beta E/WebSocket: 
WebSocketListener.onFailure() response -> null
                                                           java.net.ConnectException: Failed to connect to /xx.xx.xx.xx:4984
                                                               at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:240)
                                                               at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158)
                                                               at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256)
                                                               at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134)
                                                               at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113)
                                                               at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                                               at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                                               at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                               at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                                               at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
                                                               at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147)
                                                               at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
                                                               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
                                                               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
                                                               at java.lang.Thread.run(Thread.java:818)
                                                            Caused by: java.net.ConnectException: failed to connect to /xx.xx.xx.xx (port 4984) after 10000ms: isConnected failed: ECONNREFUSED (Connection refused)
                                                               at libcore.io.IoBridge.isConnected(IoBridge.java:257)
                                                               at libcore.io.IoBridge.connectErrno(IoBridge.java:193)
                                                               at libcore.io.IoBridge.connect(IoBridge.java:127)
                                                               at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:188)
                                                               at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:457)
                                                               at java.net.Socket.connect(Socket.java:927)
                                                               at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:69)
                                                               at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:238)
                                                               at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158) 
                                                               at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256) 
                                                               at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134) 
                                                               at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113) 
                                                               at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                               at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                               at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                               at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) 
                                                               at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) 
                                                               at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) 
                                                               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) 
                                                               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) 
                                                               at java.lang.Thread.run(Thread.java:818) 
                                                            Caused by: android.system.ErrnoException: isConnected failed: ECONNREFUSED (Connection refused)
                                                               at libcore.io.IoBridge.isConnected(IoBridge.java:246)
                                                               at libcore.io.IoBridge.connectErrno(IoBridge.java:193) 
                                                               at libcore.io.IoBridge.connect(IoBridge.java:127) 
                                                               at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:188) 
                                                               at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:457) 
                                                               at java.net.Socket.connect(Socket.java:927) 
                                                               at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:69) 
                                                               at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:238) 
                                                               at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158) 
                                                               at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256) 
                                                               at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134) 
                                                               at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113) 
                                                               at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                               at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                               at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                               at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                               at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) 
                                                               at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) 
                                                               at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) 
                                                               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) 
                                                               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) 
                                                               at java.lang.Thread.run(Thread.java:818) 

Please help me if any one can??

Can you clarify the use case for this ? Looks like you are trying to emulate a one-shot replication here. So why don’t you use one-shot replication instead ?

Hi @priya.rajagopal,

I want contineous replication but whenever sync_gateway stops or if we stops internet of mobile. Application is crashing. due to fatal error.

 WebSocketListener.onFailure() response -> null
                                                             java.net.ConnectException: Failed to connect to /xx.xx.xx.xx:4984
                                                                 at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:240)
                                                                 at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158)
                                                                 at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256)
                                                                 at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134)
                                                                 at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113)
                                                                 at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                                                 at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                                                 at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                                 at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                                                 at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
                                                                 at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147)
                                                                 at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
                                                                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
                                                                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
                                                                 at java.lang.Thread.run(Thread.java:818)
                                                              Caused by: java.net.ConnectException: failed to connect to /xx.xx.xx.xx (port 4984) after 10000ms: connect failed: ENETUNREACH (Network is unreachable)
                                                                 at libcore.io.IoBridge.connect(IoBridge.java:129)
                                                                 at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:188)
                                                                 at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:457)
                                                                 at java.net.Socket.connect(Socket.java:927)
                                                                 at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:69)
                                                                 at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:238)
                                                                 at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158) 
                                                                 at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256) 
                                                                 at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134) 
                                                                 at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113) 
                                                                 at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                                 at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                                 at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                                 at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) 
                                                                 at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) 
                                                                 at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) 
                                                                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) 
                                                                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) 
                                                                 at java.lang.Thread.run(Thread.java:818) 
                                                              Caused by: android.system.ErrnoException: connect failed: ENETUNREACH (Network is unreachable)
                                                                 at libcore.io.Posix.connect(Native Method)
                                                                 at libcore.io.BlockGuardOs.connect(BlockGuardOs.java:111)
                                                                 at libcore.io.IoBridge.connectErrno(IoBridge.java:170)
                                                                 at libcore.io.IoBridge.connect(IoBridge.java:127)
                                                                 at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:188) 
                                                                 at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:457) 
                                                                 at java.net.Socket.connect(Socket.java:927) 
                                                                 at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:69) 
                                                                 at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:238) 
                                                                 at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158) 
                                                                 at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256) 
                                                                 at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134) 
                                                                 at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113) 
                                                                 at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                                 at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                                 at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                                                 at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                                                 at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) 
                                                                 at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) 
                                                                 at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) 
                                                                 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) 
                                                                 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) 
                                                                 at java.lang.Thread.run(Thread.java:818) 
    
12-11 20:57:11.985 10118-10118/com.motionmagic.beta E/Rep Patient  Log: Replication Stopped
12-11 20:57:13.407 10118-10182/com.motionmagic.beta A/libc: Fatal signal 7 (SIGBUS), code 1, fault addr 0x100000007 in tid 10182 (RenderThread)

Thats why i am not able to understand why this issue is coming.

So are you saying that when in continuous mode, if you stop the Sync Gateway or if you turn on Airplane mode for instance, the app crashes ?

What version of DB are you using ?

( BTW You do not have to manually stop the replicator under the circumstances you indicated above. The Replicator has the smarts to detect changes to network and react accordingly. It also has a back off timer algorithm to attempt to reconnect to the Sync Gateway if the SGW goes down . That’s the intent of continuous mode- its supposed to take the burden of handling these cases away from the app developer. So I hope you are not trying to manually stop the replicator under the circumstances above. )

Hi @priya.rajagopal,

I am using CBL 2.0 DB V 20. Prior I had used db version 19 as well but got same issues on the both versions.

Regards,
Achint Sarkar

OK. Then that seems to be a bug. Will track and keep you posted. It seems to be the case only if you register a listener with the replicator.

@priya.rajagopal I removed change listener from the replicator but still getting the same error.

It is crashing only once when we turn off the internet. If we then start the application its working fine. But it is crashing if we turn off the internet in between of replication. It is giving a fatal error.

12-12 12:11:23.795 14352-15999/? E/WebSocket: WebSocketListener.onFailure() response -> null
                                          java.net.ConnectException: Failed to connect to /xx.xx.xx.xx:4984
                                              at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:240)
                                              at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158)
                                              at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256)
                                              at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134)
                                              at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113)
                                              at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                              at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                              at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                              at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
                                              at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
                                              at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147)
                                              at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
                                              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
                                              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
                                              at java.lang.Thread.run(Thread.java:818)
                                           Caused by: java.net.ConnectException: failed to connect to /xx.xx.xx.xx (port 4984) after 10000ms: connect failed: ENETUNREACH (Network is unreachable)
                                              at libcore.io.IoBridge.connect(IoBridge.java:129)
                                              at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:188)
                                              at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:457)
                                              at java.net.Socket.connect(Socket.java:927)
                                              at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:69)
                                              at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:238)
                                              at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158) 
                                              at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256) 
                                              at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134) 
                                              at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113) 
                                              at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                              at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                              at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                              at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) 
                                              at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) 
                                              at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) 
                                              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) 
                                              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) 
                                              at java.lang.Thread.run(Thread.java:818) 
                                           Caused by: android.system.ErrnoException: connect failed: ENETUNREACH (Network is unreachable)
                                              at libcore.io.Posix.connect(Native Method)
                                              at libcore.io.BlockGuardOs.connect(BlockGuardOs.java:111)
                                              at libcore.io.IoBridge.connectErrno(IoBridge.java:170)
                                              at libcore.io.IoBridge.connect(IoBridge.java:127)
                                              at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:188) 
                                              at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:457) 
                                              at java.net.Socket.connect(Socket.java:927) 
                                              at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:69) 
                                              at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:238) 
                                              at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:158) 
                                              at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256) 
                                              at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134) 
                                              at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113) 
                                              at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                              at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                              at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) 
                                              at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) 
                                              at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) 
                                              at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) 
                                              at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) 
                                              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) 
                                              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) 
                                              at java.lang.Thread.run(Thread.java:818) 

12-12 12:11:23.796 14352-14368/? E/LiteCore [Sync]: {55} Got LiteCore error: Network is unreachable (2/101) 12-12 12:11:23.803 14352-14352/? E/Sync: Replicator[<* blip://xx.xx.xx.xx:4984/generic] is offline, progress 0/0, error: CouchbaseLiteException{domain=2, code=104, msg=null} 12-12 12:11:23.803 14352-14352/? E/Sync: statusChanged() c4Status -> C4ReplicatorStatus{activityLevel=0, progressUnitsCompleted=0, progressUnitsTotal=0, progressDocumentCount=0, errorDomain=2, errorCode=104, errorInternalInfo=1010} 12-12 12:11:23.817 14352-14352/? E/ACRA: ACRA caught a SecurityException for com.motionmagic.beta java.lang.SecurityException at android.os.BinderProxy.transactNative(Native Method) at android.os.BinderProxy.transact(Binder.java:511) at android.app.ActivityManagerProxy.registerReceiver(ActivityManagerNative.java:3143) at android.app.ContextImpl.registerReceiverInternal(ContextImpl.java:1189) at android.app.ContextImpl.registerReceiver(ContextImpl.java:1157) at android.app.ContextImpl.registerReceiver(ContextImpl.java:1151) at android.content.ContextWrapper.registerReceiver(ContextWrapper.java:554) at com.couchbase.lite.AndroidNetworkReachabilityManager.startListening(AndroidNetworkReachabilityManager.java:49) at com.couchbase.lite.NetworkReachabilityManager.addNetworkReachabilityListener(NetworkReachabilityManager.java:38) at com.couchbase.lite.Replicator.startReachabilityObserver(Replicator.java:581) at com.couchbase.lite.Replicator.handleError(Replicator.java:537) at com.couchbase.lite.Replicator.c4StatusChanged(Replicator.java:477) at com.couchbase.lite.Replicator.access$000(Replicator.java:35) at com.couchbase.lite.Replicator$1$1.run(Replicator.java:390) at android.os.Handler.handleCallback(Handler.java:815) at android.os.Handler.dispatchMessage(Handler.java:104) at android.os.Looper.loop(Looper.java:207) at android.app.ActivityThread.main(ActivityThread.java:5728) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:789) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:679) 12-12 12:11:23.955 14352-14369/? A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 14369 (Thread-23815)

Please have look at the log for same…

@priya.rajagopal even its crashing, when I am running replication on multiple sync URLs, is going on. It is giving me fatal error. I having approx 7 sync URLs to sync data. It is crashing randomly. I have approx 150000 documents on the server. When I am trying to sync them all and if we continuously turn on and turn off the Internet. I am getting Fatal every time. Sometimes it is crashing within 2 to 3 attempts and sometimes it is crashing only on the single attempt. and getting the fatal error, when I added a replication listener and without replication listener.

Curious: Can you explain a bit more as to why your client app is syncing with 7 SGW URLs? Are these SGW talking to independent Couchbase Server instances?

Hi @priya.rajagopal

I made two clusters and map them with each other. I made 7 buckets on the server. And As per my calculation, each user will create minimum 150 documents per day. Just for 500 hundred users, there will be more than 75000 in one day and for a month there will be 2.25 million documents in one month.

only two or three buckets will have fewer documents than other buckets that will be around 250000.

Because of the large amount of data I made separate buckets. And all the modules will work offline. To just fulfill my requirement.

No, right now sync_gateway is talking to two cluster servers. Is there any performance restriction with SGW.

Hi @priya.rajagopal,

Any Updates on the crash issue??

I am a bit confused with your terminology.

In your description above, you mentioned 7 buckets. You mentioned “7 sync URLs” earlier.

So do I assume that you have one SGW talking to 7 nodes ? What do you mean by “sync URL”?

Also, think of a bucket roughly corresponding to a database - so not sure why you are trying to scale by adding more buckets. Or did you mean “7 node cluster” when you say “7 buckets” ?
I would recommend looking at this document for scaling guidelines

As for the crash, as I mentioned, we are investigating it . It is being tracked for DB22 at this point . If we are able to fix it sooner, will let you know.

Hi @priya.rajagopal,

Sorry for the confusion. Actually, I have single sync_gateway which is communicating with 7 different buckets. And as per the recommendation, i added two nodes to each other for the replication.

Regards,
Achint Sarkar

@achint and @priya.rajagopal

From the stacktrace above, the issue happens when CBL is registering a receiver to the Android ConnecitvityManager. As the exception is SecurityException, I’m wondering if it is due to permission to use ConnectivityManager. I reformatted the exception here:

java.lang.SecurityException
at android.os.BinderProxy.transactNative(Native Method)
at android.os.BinderProxy.transact(Binder.java:511)
at android.app.ActivityManagerProxy.registerReceiver(ActivityManagerNative.java:3143)
at android.app.ContextImpl.registerReceiverInternal(ContextImpl.java:1189)
at android.app.ContextImpl.registerReceiver(ContextImpl.java:1157)
at android.app.ContextImpl.registerReceiver(ContextImpl.java:1151)
at android.content.ContextWrapper.registerReceiver(ContextWrapper.java:554)
at com.couchbase.lite.AndroidNetworkReachabilityManager.startListening(AndroidNetworkReachabilityManager.java:49)
at com.couchbase.lite.NetworkReachabilityManager.addNetworkReachabilityListener(NetworkReachabilityManager.java:38)
at com.couchbase.lite.Replicator.startReachabilityObserver(Replicator.java:581)
at com.couchbase.lite.Replicator.handleError(Replicator.java:537)
at com.couchbase.lite.Replicator.c4StatusChanged(Replicator.java:477)
at com.couchbase.lite.Replicator.access$000(Replicator.java:35)
at com.couchbase.lite.Replicator$1$1.run(Replicator.java:390)
at android.os.Handler.handleCallback(Handler.java:815)
at android.os.Handler.dispatchMessage(Handler.java:104)
at android.os.Looper.loop(Looper.java:207)
at android.app.ActivityThread.main(ActivityThread.java:5728)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:789)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:679) 12-12 12:11:23.955 14352-14369/? A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 14369 (Thread-23815)

@achint could you please provide some info about the followings so that we could try to reproduce and identify the issue?

  1. What is the Android OS version?
  2. What is the Android device or emulator model?
  3. The ConnectivityManager requires permission to access ACCESS_NETWORK_STATE. Is there anything that may override to disallow the application to access ACCESS_NETWORK_STATE? Maybe something in the application’s manifest.

Hi @pasin,

Thanks for reply,

Answer of your questions are as follows:-

  1. Android version - Marshmello 6.0.1
  2. It was an android device Lenovo yoga tab 2
  3. ACCESS_NETWORK_STATE is already given in the manifest.

Regards,
Achint Sarkar