SharePoint search on the 2013 and 2016 platform, is build on the Fast search engine, which is a Norwegian company that was acquired by Microsoft in 2018.
Six different components makes the search service, each one can be customized and scaled up based on requirements, the six component are :
Crawl, Content Processing, Index component, Query , Analytics Processing , and the Search Administration component.
Crawl Component – enable to crawl different content sources including local and remote SharePoint sites as well as the file shares.
Content Processing Component – The information gathered by the crawl is send to the content processing component which break the content into artifacts that can be indexed. The Content Processing is also handle some customization such as custom entity extractions.
Indexing Component – the content artifacts from the content processing is passed to the indexing component this component will store the content in indexed tables that can provide fast search result. Large index tables/files can be split into smaller partitions.
Query Component – Enable the user to create and submit queries that are being validated and send the query to be executed by the indexing component. search result returned by the index component is filtered by the query component based on security trimming and than returned to the user.
Search Administrator Component – run the administrative tasks as well as provision on the other components.
The search services in SharePoint 2016 can only be stared on the Search and Custom roles servers. on fault tolerance and high availability we would like to have at least two instances of each the above components. Best practice recommendation is to separate between client side component that servers the users from the backend component that are responsible of building the search behind the scene.
backend components are – Crawl, Content Processing and indexing.
frontend / client side component are – Index, Query
Index file topology: index files can grow rapidly on the server hosting the index component and should be placed on the D: drive rather the C: system drive. You will also need to take into consideration the growth of the D drive. though there is no exact way to calculate the index since different files will require different indexing the thumb rule to estimate your index is 205 of your content DB size. for example 1 TB of content may require 200 GB of storage on your D drive.
use monitoring tool on the index file to make sure you are not running out of storage space.
you also need to check the SharePoint 2016 boundaries to make sure your topology is compliant , for example :
index partition replicas can have 20 million items, and you can have 3 index replicas per partition.