Full Text Search using the SDK java

Full Text Search
1

Hi,
my issue is next :
when i use the SearchQuery to find documents containing terms that start with the specified prefix i have no result.

Here the simple code :

    final SearchResult result = this.cluster.searchQuery("data-bucket_index_proc", SearchQuery.prefix("SAO2"), SearchOptions.searchOptions().fields("service"));

    for (final SearchRow row : result.rows()) {
        System.out.println("Score: " + row.score());
        System.out.println("Document Id: " + row.id());
    }

Here my_index :

{
“name”: “data-bucket_index_proc”,
“type”: “fulltext-index”,
“params”: {
“doc_config”: {
“docid_prefix_delim”: “”,
“docid_regexp”: “”,
“mode”: “type_field”,
“type_field”: “type”
},
“mapping”: {
“default_analyzer”: “standard”,
“default_datetime_parser”: “dateTimeOptional”,
“default_field”: “_all”,
“default_mapping”: {
“dynamic”: true,
“enabled”: false
},
“default_type”: “_default”,
“docvalues_dynamic”: false,
“index_dynamic”: true,
“store_dynamic”: false,
“type_field”: “_type”,
“types”: {
“procedure”: {
“dynamic”: true,
“enabled”: true,
“properties”: {
“creationDate”: {
“enabled”: true,
“dynamic”: false,
“fields”: [
{
“include_in_all”: true,
“index”: true,
“name”: “creationDate”,
“store”: true,
“type”: “datetime”
}
]
},
“modifiedDate”: {
“enabled”: true,
“dynamic”: false,
“fields”: [
{
“include_in_all”: true,
“index”: true,
“name”: “modifiedDate”,
“store”: true,
“type”: “datetime”
}
]
},
“service”: {
“enabled”: true,
“dynamic”: false,
“fields”: [
{
“include_in_all”: true,
“index”: true,
“name”: “service”,
“store”: true,
“type”: “text”
}
]
},
“titre”: {
“enabled”: true,
“dynamic”: false,
“fields”: [
{
“include_in_all”: true,
“index”: true,
“name”: “titre”,
“store”: true,
“type”: “text”
}
]
}
}
}
}
},
“store”: {
“indexType”: “scorch”,
“segmentVersion”: 15
}
},
“sourceType”: “gocbcore”,
“sourceName”: “data-bucket”,
“sourceUUID”: “0eda5e9d80c010f98245513c8cc14ad7”,
“sourceParams”: {},
“planParams”: {
“maxPartitionsPerPIndex”: 1024,
“indexPartitions”: 1,
“numReplicas”: 0
},
“uuid”: “759d659f4458be9f”
}

And here a capture of my index in the console

Capture d’écran 2021-06-18 110941
Capture d’écran 2021-06-18 1109411513×659 53.1 KB

I try to find documents with type “procedure” and “service” starts with “SAO2”
Here a example of document which have a “service” starts with “SAO2”:

Capture d’écran 2021-06-18 111146
Capture d’écran 2021-06-18 111146641×706 19.7 KB

I must do something wrong but i don’t see what.

Thanks for your help.
Jerome

2

Hey @jlefray, here’s what I see …

  • You are running a prefix search query over the field “service”
  • The prefix query is what we refer to as a non-analytic query - meaning no analysis is applied over the term you’re looking for. So if you’re searching for a prefix SAO2, the search service will look for tokens starting with SAO2 (case sensitive)
  • By the index definition, you’ve shared - you’re using the standard analyzer for the field “service” (inherited from top level).
  • A standard analyzer generates tokens by splitting text over whitespace, and then converts tokens to lower case and drops english stop words (articles, conjunctions etc.).
  • So taking your example document, SAO23 will be converted to sao23 by the standard analyzer, and therefore your search query is unable to find the token within the document.

For non-analytic queries (queries that search for text as is), we recommend using the keyword analyzer. This indexes text as is. So if you set the analyzer for your field “service” to keyword analyzer, your search query should start seeing some results.

Here’s a service we host where you can check how text is analyzed to tokens by the various analyzers we support and also the behavior of configurable ones … Bleve Text Analysis Wizard

For more information on various analyzers, check out our documentation here or this article here.