Global Search

Content Type: Module
Categories: Utility,Data

Overview

The Global Search module provides advanced search capabilities based on Lucene Search for your Mendix app. More specifically, this module allows you to search through multiple entities and attributes with just one query. Version 4 adds a bunch of new features and simplifications of the module:

- Advanced lucene search: search by using the Lucene Query language.

- Simple and advanced indexing: Simple indexing indexes the mendix entity. Advanced indexing used a json structure to index the entity. This way, you can fully customize what you want to index for each searchable object, including searching over association for example.

- Making entities searchable by creating specializations of GlobalSearch.SearchableObject. No more runtime configuration needed, everything can be set-up during development.

Documentation

Description

The Global Search module provides advanced search capabilities based on Lucene Search for your Mendix app. More specifically, this module allows you to search through multiple entities and attributes with just one query.


The module provides an extended Lucene search query, which allows you to search through text in several ways. Below you find an example of what you can expect when using the search. In this example, the string we want to return as a search result is “This is a search module”, due to the extended query implemented in this module, you can search the following, which will all return the mentioned string:


- Exact word match: e.g. “This”, “search”, “module”.


- Phrase match: e.g. “This is a”, “a search module”.


- Wildcard match: e.g. “sear”, “modul”


- Wildcard Phrase match: e.g. “his is a”, “a search mod”


Furthermore, you can also use the default Lucene query syntax to do a fielded search. For more details, see https://www.lucenetutorial.com/lucene-query-syntax.html.


Typical usage scenario

The Global Search module can be used when searching over multiple objects at once is required.


Features and limitations

Features:

  • Faceted searching
  • Fuzzy searching
  • Fielded searching
  • Simple indexing based on entity definition in the domain model
  • Advanced indexing based on json structure

Dependencies

  • Mendix 8.15.1 or higher
  • Community Commons

Configuration

 

Indexing

To make entities searchable, follow these steps:


0. Add the ASU_GlobalSearch to your after startup microflow.

1. Copy SUB_IndexObject and SUB_DeleteIndex into your own project.

2. Change ACO_SearchableObject and BDE_SearchableObject to use the copies you made in step 1.

3. Let each entity that you want to be searchable inherit from GlobalSearch.SearchableObject. After this, change the two SUB microflows from step 1 to accomodate for these specializations.

For indexing, you can use a simple index. This will index all attributes for an entity with the names you have in the domain model. Here, there is no support for searching over association. You can also index objects based on a JSON string. To do this, create an export mapping for the entity you want to index and add this json to the java action. Only the key/value pairs specified in the json will be indexed. This does support indexing over association, as you can include associated objects/list of objects in the json. For both methods, see the example implementation in SUB_IndexObject. Finally, during indexing, you can also add a list of Facets. These are key value pairs that are stored for that particular object, which can be used for filtering the search results later on.


Searching

The search java action returns a list of SearchableObjects that match the query. In the example snippet, a list view with specialization templates is used in order to display each specialization differently. Of course, your implementation can differ, but this might give a good example. For reference, you can refer to ACT_Searcher_Search.


Index data management

IT IS IMPORTANT TO READ THIS BEFORE DEPLOYING THIS MODULE TO PRODUCTION

Index data to zip

The newest version (5.0.0) of this module contains functionality to pull lucene index data from the TMP folder on the Mendix cloud container in which an app runs (if hosted in the MX cloud). Between deployments data hosted on the container is destroyed due to the inner workings of the Mendix cloud (ephemeral storage). The IndexDataToFileDocument Java action takes all data in the Lucene searchIndex folder and puts it in a Zip file. This file is stored in the IndexZipData entity in the database. This functionality can be accessed from the new Index data to zip tab in the snippet SNIP_Search_View OR it runs in the BSd_StoreIndexData microflow. It is recommended to use this functionality in the Before Shutdown flow of your application to make sure all index data is captured. Set the IndexFileStoreBeforeShutdown constant to True.

Zipped data to the Lucene searchIndex folder

Due to feedback about startup behaviour of apps with this module it is now possible to circumvent the default After startup flow. Set the IndexFileRestoreAfterStartup constant to True. The first time you use this nothing will happen since there is no IndexZipData object available, it is recommended to do a manual sync/rebuild of the index. See below for a how to.

Manual data management

To (re)build the Index data manually use the functionality on the Test Search tab. Click the button ‘Build index’ to trigger a manual rebuild. This can take some time but it will not cause a failed startup flow. It is also possible to use manually clear the current indexed data, but use this functionality with utmost care!

Limitations

The IndexDataToFileDocument stores *all* index data in one IndexZipData filedocument. The limit of a binary attribute on PostgresQL is about 1GB. If your index is more than one gigabyte consider other options for file storage for your app like AWS S3.

Releases

Version: 5.0.0
Framework Version: 8.15.1
Release Notes: - Upgrade to Mx 8.15.1 - Two new Java actions: IndexDataToFileDocument and FileDocumentToIndexData. - Two new constants: IndexFileRestoreAfterStartup and IndexFileStoreBeforeShutdown - Changed behaviour of the After startup flow - Before shutdown microflow to store index data in the database (see documentation). - Index data management, either on BSd and ASu or manually. - ACO on SearchableObject instead of BCO, which allows it to be used in task queues. This is a potential breaking change, check the project for potential side effects. - Manual control over index storage.
Version: 4.0.0
Framework Version: 8.12.7
Release Notes: Reworked the set-up of the module, please be aware that these changes are breaking! - No more runtime configuration needed, everything will be reflected in the mendix model during development. - Fielded search using Lucene's query language. - Searching over association via advanced indexing with json strings, here you can fully influence what you do and what you don't want to index. - Entities that are searchable need to inherit from SearchableObject. This way, all access rights for the specialisations are automatically taken into account when searching and displaying the results, making the search a lot faster. Also, you can now really customise the information you want to show for each entity type. - Highlighting has been removed as it greatly reduces search performance and introduces a lot of overhead.
Version: 3.0.0
Framework Version: 8.12.7
Release Notes: Added features: - Compatibility with Mendix 9 - Upgraded to Lucene 8.9 - Configurable facet searching - Searching over (multiple) association(s) - Highlighting search results - Fuzzy searching - Configuring Title/Details attributes per searchable entity
Version: 2.0.0
Framework Version: 8.6.4
Release Notes: Changes/new features: - The wildcard search option is replaced with a more advanced search query builder, which combines exact match, phrase search and wildcard search. - The new search query builder allows for searching on multiple (partial) words, meaning that the following queries "This is a", "This" and "a new rel" will all return the phrase "This is a new release" as search result. - An after startup microflow to refresh indexes when (re)starting. Bug fixes: - A bug fix for incorrect search results when searching with capitalised characters. - A bug fix for the error that occurred when inserting special characters in the search string. - General improvements of the java code.
Version: 1.0.0
Framework Version: 8.6.4
Release Notes: Updates: - Added support for wild card search. This type of search can be triggered by the boolean in the search snippet. - The refresh in the configuration snippet now refreshes all searchable objects and their attributes. Meaning that with a click of a button, all searchable objects are up-to-date again. - Bug fix for the number of search results. - Bug fix for the error that occurred when trying to index non-persistent objects (those are now not available for indexing anymore). - Quality of life improvement for configuring searchable objects.