What is the best way to handle a sync config change and resync?

Mobile Sync Gateway
1

We need to update our sync config

We are using 1.3 and SWG is on 3 nodes.

Questions:

  1. Do we need take each SWG offline, and run a resync or just one?
  2. if we have to take each SWG offline and resync, can we do one at a time?
2

They operate independently, so you can take them down one at a time. You do need to restart all of them.

3

But would I need to resync them all after the restart or just one? We have about 25M and it will take over 10 hours

4

Unfortunately you’ll need to restart them all. SG instances don’t share any information.

Have you looked at filtering? This can reduce startup time significantly if you don’t need to sync every document. See https://developer.couchbase.com/documentation/mobile/1.5/guides/sync-gateway/config-properties/index.html#1.5/databases-foo_db-import_filter

5

Here is our procedure.

3 SGW

  1. Update sync-config.json and restart all nodes
  2. Node 1 - Take offline and resync.
  3. Once node 1 is done resync, bring it online.

I would not resync the other nodes.

With this work?

Also for import_filter the documentation default is (Default: function(doc) {return false;}), does that mean nothing will get sync to a mobile device?

6

That has to be true to allow all documents to be imported by SGW
That said, I am a bit confused . You mentioned earlier that you are on 1.3.

We are using 1.3 and SWG is on 3 nodes.

The import _filter is V1.5 only feature with shared buckets. Are you upgrading to v1.5 ?

7

We are 1.3 so that part of the confusion was the previous post said to look at the import filter.

8

Once the sync-config is updated with the new function and restarted, that sync function needs to be applied to all the documents in the bucket. Ideally we would like to take just ONE sync gateway node offline and run resync on that one node while leaving the other nodes ONLINE handling traffic. Taking all three nodes offline simultaneously for the better part of a day would not be practical for us.

Another option that leaves the gateways online would be an application level “touch” of all the impacted documents (write a script that runs through all documents impacted by the sync-function change re-saving the current revision just to trigger a gateway review).

Resync with a single node seems preferable from a performance standpoint, although I’ve seen other posts indicating that the resync somehow impacted document visibility. Hoping there are folks out there that have done a resync while keeping a gateway cluster “live”.

9

Regarding resync in clustered environments -

As described in our documentation

In a clustered environment with multiple Sync Gateway instances sharing the load, all the instances need to share the same configuration, so they all need to be taken offline either by stopping the process or taking them offline using the /{db}/_offline endpoint. After the configuration is updated, one instance should be brought up so it can update the database—​if more than one is running at this time, they’ll conflict with each other. After the first instance finishes opening the database, the others can be started.

That said, bear in mind that a resync is not required every time the sync function is changed.
For instance, you don’t need a resync if the changes are

  • around write security
  • data validation
  • changes are additive (adding new document types, etc)

You would need a resync if the updates

  • are changing channel access rules/read security and if they need to be applied to existing documents
    But you don’t need it if documents are already replicated !