Performance sync gateway, slow initial replication

Mobile Sync Gateway
1

So we are running couchbase DB (server:community-5.0.1 with xattr enabled), sync gateway (sync-gateway:1.5.1-community) and couchbase lite 1.4.

We’ve been running into replication performance issues, takes about 5 - 10 minutes for users to sync their docs when they first log in. Users don’t have that many docs < 1000 and they aren’t large (no attachments).

We are currently looking into creating some views to return doc ids we are interested, performing a 1 shot replication for those specific docs then start a continuous replication. I’m hoping this would improve performance, but I guess we’ll find out once we implement it.

So I’m wondering if I wanted to improve performance what are some of the things we could do? What about horizontal scaling of couchbase sync gateway? Would I see improvements if we looked into that?

2

I don’t see the need for this for just 1000 documents…

Can you clarify the following :

  • By “sync”, are you referring to
    • (a) push or
    • (b) pull sync ?
      In other words, is the CBL App pushing up the 1000 documents or is the app pulling down the documents from server
  • If it’s a pull replication , is the 5-10 min the
    • (a) time taken by the SGW to process (import and add sync metadata) the documents from the CB server bucket on startup. or
    • (b) or is the time for the documents to show up at the CBL (i.e the documents have all been processed by sync gateway). Factors like network connectivity and document sizes affect this although you mention that your documents are small (??)
3

Hi @priya.rajagopal

So our syncing is a pull sync, the IOS app is slow pulling docs. For the second question, it slow showing in CBL. our Documents are small should all be under < 1MB. As for network connectivity our server is on digital ocean in singapore and we are in Australia. But I don’t think it should take 5 - 10 minutes for the size and number of docs that need syncing, maybe I’m wrong?

I’ve pasted my sync gateway config below as well if that helps.

{
  "interface": ":4984",
  "profileInterface": "80",
  "adminInterface": ":4985",
  "MaxFileDescriptors": 25000,
  "log": [
    "CRUD",
    "CRUD+",
    "HTTP",
    "HTTP+",
    "Access",
    "Cache",
    "Shadow",
    "Shadow+",
    "Changes",
    "Changes+"
  ],
  "databases": {
    "litehq": {
      "username": "${CB_USER}",
      "password": "${CB_PASS}",
      "bucket": "bucket",
      "server": "http://db:8091",
      "revs_limit": 100,
      "changes_feed_limit": 200,
      "enable_shared_bucket_access": true,
      "import_docs": "continuous",
      "sync": `
      function(doc, oldDoc) {
        if(doc.type === "food") {
          channel("public");
        } else {
          channel(doc.members);
        }
      }`,
      "users": {
        "GUEST": { "disabled": false, "admin_channels": ["*"] }
      }
    }
  }
}
4

Are you on a WiFi or cellular network (assuming you are testing on a device)? Because if it’s the latter, it can be quite unpredictable.

The performance could also be impacted by the device storage speed, CPU etc. Can you try with different devices and see if the times are consistent across users across devices?

Regardless of whether its the network that’s causing the delay, the typical design pattern is for you to have your app listen to database changes and to update the UI as and when changes come in while replication continues to update in the background. So that way, your app perceived user experience is still good.

Some suggestions on your config file

  • Leave changes_food_limit to default . This number would impact how many changes CBL would have to handle per cycle . We can tweak it once we identify if it was having an effect.

  • Unrelated to this issue , is there a reason you have set the revs_limit to 200. It’s not an unreasonable number but just want to ensure you understood the implications of changing this value on database sizes and conflicts

  • Unrelated to this issue : Keeping the GUEST user enabled is risky. Its really intended for development and debugging purposes.

5

Are you using Xamarin or iOS native?

The Xamarin SDK seems to have some performance issues (Couchbase Lite performance differences Android Java vs Xamarin).

In my test setup I can pull 100000 documents in around 10min (on mid-range Android devices).

6

Yeah testing was done on an actual devise (wifi). Also tested with different phones (iPhone 6, 6s, 8). All were pretty much the same - slow.

Thats the issue atm. People login, replications occur in the background. But everyone complains that data is missing (comes in eventually).

No worries, will update those config settings. Thanks for pointing out the guest user. I’ll remove that as well.

7

Hey, we are using IOS (react native). Thanks for pointing that out, good to know.