Evaluating Full Text Search

Full Text Search
21

https://docs.couchbase.com/server/current/fts/fts-troubleshooting.html#q-can-i-store-specific-document-fields-within-fts-and-retrieve-those-as-a-part-of-the-search-results

22

Thanks @sreeks for the above link. Using store checkbox and fields key in query, i am able to get specific fields. How to store complete document and retrieve it back?

23

There is no way to explicitly store the whole document within FTS.
One could store the whole document only using the store option for each field by field in the document.
For the whole document, fetching from KV is recommended.

Cheers!

24

There is actually a way to store the whole documents within FTS. It’s not one we recommend however - for it could cause the index to have a large footprint (based on the sizes of your documents) …

You’ll need to just use a default mapping and leave it to be dynamic indexed - meaning do not select “Only index specific fields”. Now in the advanced section, you’ll need to select “Store dynamic fields”. This will pretty much index your entire document. However you’ll see that we flatten the JSON structure.

For example, if I were to index this document …

{
  "a": {
    "b": {
      "c": 12,
      "d": 13
    },
    "e": "xyz"
  },
  "f": 123
}

Now if you try to retrieve all properties of a document with a search request using this "fields": ["*"], you will get all properties alongside document IDs but in a flattened format like this …

[
    "a.b.c": 12,
    "a.b.d": 13,
    "a.e": "xyz",
    "f": 123
]

What @sreeks suggested earlier is perhaps the wiser thing to do - you could use N1QL for this - embed the search request to satisfy some criteria to obtain documents of interest, within your N1QL query with a SELECT * (this would inherently involve a KV fetch to retrieve your entire document(s)) …
https://docs.couchbase.com/server/6.6/n1ql/n1ql-language-reference/searchfun.html

25

Thanks guys for helping here.

Which one is better performant?

  1. Using select * in N1QL query for search OR → this involves extra nodes of query service
  2. Get doc ids using raw search query and then get all documents using KV query?

I need help in below 2 queries:

  1. raw query with IN clause just like in SQL. I tried below but did not get results:
    {
    “field”: “field1”,
    “terms”: [“abc”,“def”, “ghi”]
    }
  2. equality operator on numeric value. I tried using range but i need equality one?
    {
    “field”: “field2”,
    “min”: 3, “max”: 3,
    “inclusive_min”: true,
    “inclusive_max”: true
    }

I can also see that pagination is available 6.6 onwards. Is it correct ?

Thanks
Nitesh

26

I’ll recommend the approach with N1QL.

For (1) - your syntax is way OFF. The IN equivalent of a full-text search is a disjuncts query. We can’t seem to stress on this enough but you’ll need to go through documentation on query syntaxes - https://docs.couchbase.com/server/6.6/fts/fts-query-types.html

On (2), for equality check on a numeric operator, you’ll need to use a range with min and max set to the same number and inclusive_min and inclusive_max set to true. In your example you’re checking for "field2": 3.

Pagination has always been supported. You can use size (limit) and from (offset) for it.
https://docs.couchbase.com/server/6.6/fts/fts-queries.html#pagination

Pagination using “search_after/search_before” (see the docs link above) is supported from 6.6 onwards.

27

Thanks @abhinav

  1. For IN equivalent, you mean as below:
    {
    “disjuncts” : [
    {“field” : “field1”, “match” : “abc”},
    {“field” : “field1”, “match” : “def”}
    ]
    }
    What if i need to have 1000s of values to be in IN clause? Will it support and work fast performant by querying on that field?

  2. If in my documents, a field is of array type and and that field is having 10000 values in it and there are many documents in my database. Also, FTS is searchable on that field. Will query on that field be fast or slow performant? If that will be slow, what else do i need to do?

Thanks
Nitesh