FTS mapping strategy for dynamic fields

Dynamic fields are supported, but maybe not for the exact cases you have in mind. Let me know.

For the sake of example, let’s create an index and use the “default” mapping on the travel-sample data set that ships with Couchbase Server 4.5. A default mapping indexes any document without regard for the document’s type field. (You can still do what I’m showing here if you specify a type mapping, that is, if you limit the mapping to documents of type = “hotel”. I’m just trying to keep it simple).

I did one other thing that you probably don’t want to do on production, but it makes the example easier to run. I expanded the advanced options and clicked “store_dynamic” so that when the search service builds the index, it stores all of the fields so that it will return them with your search it will give you result snippets with matches highlighted.

When you query, use the _all field, which you can rename if you want. The _all field is what the server searches if you don’t specify any field scope in a query string query. So, when you go to the “index” tab in the web admin and search for “gorilla”, that is equivalent to “_all:gorilla”.

Down below is an index definition CURL you can run from your command line to create the index I just described. This is identical to creating it in the web console. This index uses the “travel-sample” data set. You can query for things like: “holiday inn victoria emelie dione” and see results being matched in multiple fields. At the end of the article, there’s a curl command that will run that query, in case you’re a command line type person.

OK, a couple more things. In your example, you showed dynamic properties in an embedded object. You can search the sample index we created (called “sir-mix-alot” because we mix up all the fields, naturally). If you search for “inn rooftop”, you will get matches on the embedded “geo” object, which looks like this in a hotel document:

"geo": { "accuracy": "ROOFTOP", "lat": 51.35785, "lon": 0.55818 },

If you want to be more specific with how you index a document, normally you need to “insert a child mapping” (that’s what it’s called in the custom mapping UI). I’ve linked a video below that explains how to create custom maps, and it’s also described in the 4.5 Beta documentation.

Hope that helps!


Index definition
curl -XPUT -H "Content-Type: application/json" \ http://localhost:8094/api/index/sir-mix-alot \ -d '{ "type": "fulltext-index", "name": "sir-mix-alot", "sourceType": "couchbase", "sourceName": "travel-sample", "planParams": { "maxPartitionsPerPIndex": 32, "numReplicas": 0, "hierarchyRules": null, "nodePlanParams": null, "pindexWeights": null, "planFrozen": false }, "params": { "mapping": { "byte_array_converter": "json", "default_analyzer": "standard", "default_datetime_parser": "dateTimeOptional", "default_field": "_all", "default_mapping": { "display_order": "0", "dynamic": true, "enabled": true }, "default_type": "_default", "index_dynamic": true, "store_dynamic": true, "type_field": "type" }, "store": { "kvStoreName": "forestdb" } }, "sourceParams": { "clusterManagerBackoffFactor": 0, "clusterManagerSleepInitMS": 0, "clusterManagerSleepMaxMS": 2000, "dataManagerBackoffFactor": 0, "dataManagerSleepInitMS": 0, "dataManagerSleepMaxMS": 2000, "feedBufferAckThreshold": 0, "feedBufferSizeBytes": 0 } }'


Sample query
curl -XPOST -H “Content-Type: application/json”
http://localhost:8094/api/index/sir-mix-alot/query
-d ‘{
“explain”: true,
“fields”: [
"*"
],
“highlight”: {},
“query”: {
“query”: “holiday inn victoria emelie dion”
}
}’


Videos about indexes that cover the very basics of custom index mapping, first is conceptual, the second is a demo.