SGW basic comprehension

Hi,

I continue my CBM learning.
I installed Couchbase server 5 and SGW 1.5 and created a cluster named cb-poc.
I then created a bucked named product and add some documents there. My final goal is to interact with these documents on a mobile app through the SGW, but for now I need to test the installation and properly configure the SGW for my future development.
So I started the SGW without any configuration file. then try to query the documents from the command line using curl. I get a response but no documents.
this query for exemple :

curl -X GET "http://localhost:4984/db/_all_docs?access=false&channels=false&include_docs=false&revs=false&update_seq=false&limit=100" -H "accept: application/json"

give me such result :

   {"rows":[
    ],
    "total_rows":0,"update_seq":1}

So the SGW seems to respond but there is clearly something that I dont understand.
May be I query the wrong database? I didnt saw the database in couchbase key concepts but the SGW work with that concept of database. I replaced the db in the curl command with the bucket’s name product , and I receive the folowing message
{"error":"not_found","reason":"no such database \"product\""}
same when I replace with the cluster name cb-poc
why doest SGW recognise db that I never created but neither the bucket name nor the cluster name, that I explicitly created?
should I write a SGW configuration file? how? is there another cause?

Thanks
Patrick

it will use default configuration with memory database that not connect any Couchbase bucket.

maybe you can post your log here.

If you started SG without a configuration file, looking at the logs, you probably saw something like this in them:

Opening db /sync_gateway as bucket "sync_gateway", pool "default", server <walrus:>

By default SG does not try to connect to CB Server. It uses a simple in-memory database known as “walrus”. The only bucket it will recognize is “sync_gateway”. You need to give it a config file. Take a look at the examples in the examples subdirectory wherever you installed SG to see some.

Also, you might to look at this blog series: https://blog.couchbase.com/getting-comfortable-with-couchbase-mobile-sync-gateway-via-the-command-line/ (That one starts with working with SG on the command line. Scroll to the bottom to find links to other posts in the series.)

Thanks for the answer. where do I see the logs?

I saw the logs. please ignore my last post.

I used the basic -couchbase-bucket.json from the examples and still get the same answer from the SGW. Here is the last lines of the log :
2018-02-27T06:16:52.588Z Enabling logging: [HTTP+]
2018-02-27T06:16:52.589Z ==== Couchbase Sync Gateway/1.5.1(4;cb9522c) ====
2018-02-27T06:16:52.589Z requestedSoftFDLimit < currentSoftFdLimit (5000 < 65535) no action needed
2018-02-27T06:16:52.589Z Opening db /db as bucket “db”, pool “default”, server walrus:data
2018-02-27T06:16:52.589Z Opening Walrus database db on walrus:data
2018-02-27T06:16:52.590Z Using default sync function ‘channel(doc.channels)’ for database "db"
2018-02-27T06:16:52.590Z Reset guest user to config
2018-02-27T06:16:52.590Z Starting admin server on 127.0.0.1:4985
2018-02-27T06:16:52.596Z Starting server on 0.0.0.0:4984 …

so the SGW seems to behave as if there where no config file.
I remark there is a default confilg file sync_gateway.json in the folder /home/sync_gateway
is it possible that it still use it, ignoring the config file I give him on the startup?

so the name is “db”

post your /home/sync_gateway/sync_gateway.json here, it should be use this config file as default.

here is the /home/sync_gateway/sync_gateway.json :

"log": ["HTTP+"],
        "adminInterface": "127.0.0.1:4985",
        "interface": "0.0.0.0:4984",
        "databases": {
                "db": {
                        "server": "walrus:data",
                        "users": {
                                "GUEST": {"disabled": false, "admin_channels": ["*"] }
                        }
                }
        }
}

Hi!
I move that sync_gateway.json from the folder /home/sync_gateway/ and start the SGW service using the config from the example and here is how the log looks like :

 2018-02-27T19:32:51.884Z FATAL: Error reading config file /home/sync_gateway/sync_gateway.json: open /home/sync_gateway/sync_gateway.json: no such file or directory -- rest.ParseCommandLine() at config.go:582
2018-02-27T19:32:52.230Z FATAL: Error reading config file /home/sync_gateway/sync_gateway.json: open /home/sync_gateway/sync_gateway.json: no such file or directory -- rest.ParseCommandLine() at config.go:582
2018-02-27T19:32:52.471Z FATAL: Error reading config file /home/sync_gateway/sync_gateway.json: open /home/sync_gateway/sync_gateway.json: no such file or directory -- rest.ParseCommandLine() at config.go:582
2018-02-27T19:32:52.740Z FATAL: Error reading config file /home/sync_gateway/sync_gateway.json: open /home/sync_gateway/sync_gateway.json: no such file or directory -- rest.ParseCommandLine() at config.go:582
2018-02-27T19:32:53.006Z FATAL: Error reading config file /home/sync_gateway/sync_gateway.json: open /home/sync_gateway/sync_gateway.json: no such file or directory -- rest.ParseCommandLine() at config.go:582

So it seems SGW ignore the config file I pass him on startup. Here is the command I use to startup the SGW :
$ sudo service sync_gateway start /vagrant/basic-conf.json

is that startup command right?

hod.greeley

Thank you for your blogs. I think I had better start by reading it.

thanks too atom.

The config file should be in /home/sync_gateway folder . Move the one that you want to use into this folder and rename it sync_gateway.json. The sync gateway service by default looks for config in that folder.

Details here

Thank you Priya, it works.

now starting the SGW with the example/basic-couchbase-bucket.json as config file, I get an authentication error, here is the log that I get in loop :

2018-03-01T23:30:55.201-05:00 Enabling logging: [*]
2018-03-01T23:30:55.202-05:00 ==== Couchbase Sync Gateway/1.5.1(4;cb9522c) ====
2018-03-01T23:30:55.202-05:00 Configured process to allow 5000 open file descriptors
2018-03-01T23:30:55.202-05:00 Opening db /db as bucket "default", pool "default", server <http://localhost:8091>
2018-03-01T23:30:55.202-05:00 GoCBCustomSGTranscoder Opening Couchbase database default on <http://localhost:8091> as user "default"
2018-03-01T23:30:55.211-05:00 WARNING: Error opening bucket: default.  Error: authentication error -- base.GetCouchbaseBucketGoCB() at bucket_gocb.go:118
2018-03-01T23:30:55.211-05:00 Debug: RetryLoop retrying Attempt to connect to bucket : default after 5 ms.
2018-03-01T23:30:55.217-05:00 GoCBCustomSGTranscoder Opening Couchbase database default on <http://localhost:8091> as user "default"
2018-03-01T23:30:55.220-05:00 WARNING: Error opening bucket: default.  Error: authentication error -- base.GetCouchbaseBucketGoCB() at bucket_gocb.go:118
2018-03-01T23:30:55.220-05:00 Debug: RetryLoop retrying Attempt to connect to bucket : default after 10 ms.
2018-03-01T23:30:55.231-05:00 GoCBCustomSGTranscoder Opening Couchbase database default on <http://localhost:8091> as user "default"
2018-03-01T23:30:55.235-05:00 WARNING: Error opening bucket: default.  Error: authentication error -- base.GetCouchbaseBucketGoCB() at bucket_gocb.go:118

and here is the basic-couchbase-bucket.json :

    {
      "log": ["*"],
      "databases": {
        "db": {
          "server": "http://localhost:8091",
          "bucket": "default",
          "users": { "GUEST": { "disabled": false, "admin_channels": ["*"] } }
        }
      }
}

I login to the couchbase server with the defaukt Administration user, do I change some configuration at the CB server or the SGW config file?

  • Did you create a RBAC user with relevant bucket access credentials? You should see the list of users and access permissions under Security tab on Admin console.
    If You have not created the user, then add a user by selecting the “Add User” via the CB server Admin console as discussed in this blog

  • Once you create the user, configure those credentials in the SGW config file as “username”. These are the credentials that the SGW will use to access the relevant bucket.

You should include the following properties within the “databases” object in config file where “admin” and “password” should be replaced with credentials that you have chosen for the RBAC user that you created in previous step.

"username": "admin",
"password": "password",

Hi priya,

I did this, the SGW seems to start well. now I have an authentication issue when runing a simple command to get data from the bucket via the SGW. here is my terminal output when running a curl command to get some data :

$ curl -u "bs_full:100000" http://localhost:4984/db/beer-sample
2018-03-05T16:21:38.292-05:00 HTTP auth failed for username="bs_full"
2018-03-05T16:21:38.292-05:00 HTTP:  #001: GET /db/beer-sample
2018-03-05T16:21:38.292-05:00 HTTP: #001:     --> 401 Invalid login  (1.3 ms)
{"error":"Unauthorized","reason":"Invalid login"}

bs_full is the RBAC user I created. But when I run a such command directly on Couchbase server I get data :
curl -u bs_full:100000 http://localhost:8092/beer-sample/_design/beer/_view/brewery_beers

so the user has the right role. either curl command is not right or my config file is wrong.

here is the SGW config :
{
“log”: ["*"],
“databases”: {
“db”: {
“server”: “http://localhost:8091”,
“bucket”: “beer-sample”,
“username”: “bs_full”,
“password”: “100000”
}
}
}

thanks

Authentication for server and for sync gateway are two separate things. You need to also make a user (or users) in Sync Gateway either via the config file or the admin REST API. A bit more info about that is in the docs

It works thank you borrden i created a user on the SGW config.

Is there a way to retrieve preexisting data from a bucket? I ran the following command to get all beer-sample data and I got a success but an empty JSON :
curl -u "bs_full:123456" -X GET "http://localhost:4984/db/_all_docs"

but after I insert a document using :
curl -u "bs_full:123456" -s -g -X PUT -H 'Content-Type: application/json' -d '{ "test": "entry" }' localhost:4984/db/a_doc

the GET query return me only that document. So the data that pre exist on the database are not returned :
$ curl -u “bs_full:123456” -X GET "http://localhost:4984/db/_all_docs"
2018-03-06T21:25:41.398-05:00 HTTP: #011: GET /db/_all_docs (as bs_full)
2018-03-06T21:25:41.610-05:00 HTTP+: #011: --> 200 (223.5 ms)
{“rows”:[
{“key”:“a_doc”,“id”:“a_doc”,“value”:{“rev”:“1-93996db139ad4024d0c7c982d4f69122”}}
]

That kind of stuff is controlled by the sync function in sync gateway (part of your config), which is at the core of the security of your data.

The easiest, and least secure way to see everything is to add access to the “*” channel for your new user. The same page I sent should show an example of that.

I did it. here is my SGW config :

{
  "log": ["*"],
  "databases": {
    "db": {
      "server": "http://localhost:8091",
      "bucket": "beer-sample",
      "username": "bs_full",
	  "password": "100000",
	  "users": { "bs_full": {"password":"123456", "admin_channels": ["*"] } }
    }
  }
}

For pre-existing docs in bucket, you may want to import the documents so they are processed by the SGW and relevant sync metadata is added,
Take a look at this link for details.

It works thank you preya. thank you to everyone that helped me.