Asciifolding character filter is a new functionality we can see on couchbase 6.5 beta documentation.
In couchbase 6.5 beta2 , FTS UI only manage regexp character filters, no trace of asciifolding
Can anyone provide any FTS index using asciifolding?
Once I select ‘asciifolding’ and click over button +Add and after that ‘Save’ nothing happens and the character filter is not added. In fact the update index botton of the index continues greyed out.
could you try and post the code of the new index with asciifolding created?
{
"type": "fulltext-index",
"name": "Sample_Index",
"uuid": "5fe2bf1d977b4cc7",
"sourceType": "couchbase",
"sourceName": "sample",
"sourceUUID": "f00713afa521156e78465fe2bb10571e",
"planParams": {
"maxPartitionsPerPIndex": 171,
"indexPartitions": 6
},
"params": {
"doc_config": {
"docid_prefix_delim": "",
"docid_regexp": "",
"mode": "type_field",
"type_field": "type"
},
"mapping": {
"analysis": {
"analyzers": {
"custom": {
"char_filters": [
"asciifolding"
],
"token_filters": [
"to_lower"
],
"tokenizer": "unicode",
"type": "custom"
}
}
},
"default_analyzer": "standard",
"default_datetime_parser": "dateTimeOptional",
"default_field": "_all",
"default_mapping": {
"default_analyzer": "custom",
"dynamic": false,
"enabled": false,
"properties": {
"description": {
"dynamic": false,
"enabled": true,
"fields": [
{
"analyzer": "custom",
"docvalues": true,
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "description",
"type": "text"
}
]
}
}
},
"default_type": "_default",
"docvalues_dynamic": true,
"index_dynamic": true,
"store_dynamic": false,
"type_field": "_type",
"types": {
"Airport": {
"dynamic": false,
"enabled": true,
"properties": {
"description": {
"dynamic": false,
"enabled": true,
"fields": [
{
"analyzer": "custom",
"docvalues": true,
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "description",
"type": "text"
}
]
}
}
}
}
},
"store": {
"indexType": "scorch"
}
},
"sourceParams": {}
}
Thanks Sreeks,
I’m using docker version of 6.5beta2.
I will update the index via curl , because when using UI I could not save the modifications after adding asciifolding char filter neither using Chrome nor Edge.
I have tested the index you send me:
I have added this document:
{
“type”: “Airport”,
“description”: “Madrid Airport Adolfo Súarez”
}
When I query without diacritic I have response, but when I query with diacritic I get no response! , asciifolding should work in both cases, Is it right?
Query without diacritic:
root@debian92:~/couchbase# curl -XPOST -H "Content-Type: application/json" -u admin:111111 http://localhost:8094/api/index/Sample_Index/query -d '{"query": {"conjuncts": [{"match":"Suarez"}]}}'
{“status”:{“total”:6,“failed”:0,“successful”:6},“request”:{“query”:{“conjuncts”:[{“match”:“Suarez”,“prefix_length”:0,“fuzziness”:0,“operator”:“or”}]},“size”:10,“from”:0,“highlight”:null,“fields”:null,“facets”:null,“explain”:false,“sort”:["-_score"],“includeLocations”:false,“search_after”:null,“search_before”:null},“hits”:[{“index”:“Sample_Index_6ee8b6d24f7d5c18_f4e0a48a”,“id”:“Airport-1”,“score”:3.280340815507764,“sort”:["_score"]}],“total_hits”:1,“max_score”:3.280340815507764,“took”:363458,“facets”:null}
Query including the diacritic:
root@debian92:~/couchbase# curl -XPOST -H "Content-Type: application/json" -u admin:111111 http://localhost:8094/api/index/Sample_Index/query -d '{"query": {"conjuncts": [{"match":"Súarez"}]}}'
{“status”:{“total”:6,“failed”:0,“successful”:6},“request”:{“query”:{“conjuncts”:[{“match”:“Súarez”,“prefix_length”:0,“fuzziness”:0,“operator”:“or”}]},“size”:10,“from”:0,“highlight”:null,“fields”:null,“facets”:null,“explain”:false,“sort”:["-_score"],“includeLocations”:false,“search_after”:null,“search_before”:null},“hits”:,“total_hits”:0,“max_score”:0,“took”:373230,“facets”:null}
Isn’t the _all coming to play here as in last case ?
oK, Thanks . Same case if needed. _all should have the same analyzer.