johnfak
January 10, 2020, 7:59pm
21
Yes quicker — but now not returning results. I must have done somehting wrong … I’ll play around with it
{
abhinav
January 10, 2020, 8:05pm
22
Cool, you’d want to make sure that the field name that you’re indexing matches with the field name you’re searching for.
johnfak
January 10, 2020, 8:18pm
23
Trying to index literally type value in JSON
{
I select
JSON Type Filed (type)
Select field : type
select all options - but ensure “store” is deselected.
Create index.
abhinav
January 10, 2020, 8:57pm
24
Just noticed that your index wouldn’t have indexed anything at all.
Do NOT un-select the default type mapping.
Do select “Only index specified fields”.
johnfak
January 10, 2020, 9:13pm
25
Thanks - that works but is slow again.
Do I need to select ‘insert child field’ still or not as Im applying to all ‘type’
abhinav
January 10, 2020, 9:25pm
27
This is incorrect. Follow these steps:
Drop the type mapping “type” and everything within it.
Within the default mapping, first select “only index specified fields” and then add a child field “type”.
Or instead, to make this even simpler, lets do this …
Copy this index mapping into a file, say “temp.json”
{
"name": "faksearch2",
"type": "fulltext-index",
"uuid": "",
"sourceType": "couchbase",
"sourceName": "ice_us",
"sourceUUID": "021160cf87998bf9e4dc96303a90a13d",
"sourceParams": {},
"planParams": {
"maxPartitionsPerPIndex": 171,
"numReplicas": 0,
"indexPartitions": 6
},
"params": {
"mapping": {
"default_mapping": {
"enabled": true,
"dynamic": false,
"properties": {
"type": {
"enabled": true,
"dynamic": false,
"fields": [
{
"name": "type",
"type": "text",
"store": false,
"index": true,
"include_term_vectors": true,
"include_in_all": true,
"docvalues": true
}
]
}
}
},
"default_type": "_default",
"default_analyzer": "standard",
"default_datetime_parser": "dateTimeOptional",
"default_field": "_all",
"store_dynamic": false,
"index_dynamic": true
},
"store": {
"indexType": "scorch"
},
"doc_config": {
"mode": "type_field",
"type_field": "type",
"docid_prefix_delim": "",
"docid_regexp": ""
}
}
}
Next run this command against your node …
curl -XPUT -H "Content-type:application/json" http://<username>:<password>@<ip>:8094/api/index/faksearch2 -d @temp.json
The UI would look like this …
johnfak
January 10, 2020, 9:32pm
28
Works … but back to slow.
Thats 4 hours to index the docstore …
johnfak
January 10, 2020, 9:32pm
29
CPU is idle.
04:28:25 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
abhinav
January 10, 2020, 9:37pm
30
That is pretty slow. Can you confirm that the index definition is exactly identical to what I shared earlier?
sreeks
January 13, 2020, 4:29pm
31
Hi @johnfak , are you sure of the cpu usage of cbft process?
johnfak
January 14, 2020, 2:17pm
32
I believe so … although it formats it differently - sorry was out Monday
{
"type": "fulltext-index",
"name": "faksearch2",
"uuid": "612e4eeb653ebc1b",
"sourceType": "couchbase",
"sourceName": "ice_us",
"sourceUUID": "021160cf87998bf9e4dc96303a90a13d",
"planParams": {
"maxPartitionsPerPIndex": 171,
"indexPartitions": 6
},
"params": {
"doc_config": {
"docid_prefix_delim": "",
"docid_regexp": "",
"mode": "type_field",
"type_field": "type"
},
"mapping": {
"analysis": {},
"default_analyzer": "standard",
"default_datetime_parser": "dateTimeOptional",
"default_field": "_all",
"default_mapping": {
"dynamic": false,
"enabled": true,
"properties": {
"type": {
"dynamic": false,
"enabled": true,
"fields": [
{
"docvalues": true,
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "type",
"type": "text"
}
]
}
}
},
"default_type": "_default",
"docvalues_dynamic": true,
"index_dynamic": true,
"store_dynamic": false,
"type_field": "_type"
},
"store": {
"indexType": "scorch"
}
},
"sourceParams": {}
}
johnfak
January 14, 2020, 2:19pm
33
basically each server (3) has 16GB RAM
10GB to data
sreeks
January 14, 2020, 4:03pm
34
512 MB is very less and try bumping this upto 2-3GB or so. You may adjust the memory quota of other non using services accordingly.
johnfak
January 14, 2020, 4:40pm
35
Definately quicker and ore acceptable/normal for a large index.
Query is down from 40 second (N1QL) to around 3.5 to 4 seconds and pretty on par with redis.@abhinav @sreeks
Final side note.
Is there a guide on when to use one over the other based on performance/features.
Thanks again … very insightful.
1 Like
abhinav
January 15, 2020, 6:44pm
36
There’s documentation on various query types supported by FTS here …https://docs.couchbase.com/server/6.5/fts/fts-query-types.html
And here’s how to leverage FTS from within N1QL in the upcoming release …https://docs.couchbase.com/server/6.5/n1ql/n1ql-language-reference/searchfun.html
As for guidelines on when to use N1QL+GSI vs FTS, I’ll ping @binh.le to check if he can point you to any documentation available or if its a work in progress.
johnfak
January 15, 2020, 6:52pm
37
Thanks @abhinav
appreciated.
