I observed this is Pega 8.4.2 and now in Pega 8.6.3.
The search indices on the filesystem are illegible and if we have just one kind of indices corrupted, then we are not able to isolate them and have to remove all the indices and rebuilt everything.
I recall there was a hotfix for this issue early on in Infinity releases, is there a known fix to this issue?
Hi @pawann I have gone through past support tickets and am unable to find any reference to hotfixes existing to remedy this particular symptom, even in 8.4.
I found one client who had a Cloud environment where the issue occurred because of a faulty configuration base where a cleanup of older nodes did not happen and it picked random util nodes to attach and caused ES corruption. Web node accidently updates replica count which creates impacts Search Stability.
That issue was resolved in 8.6.6 under BUG-747182
Replica count updated only for index nodes
Every time a search node is restarted or reinitialized, a refresh command (refreshClients command) is sent to all nodes. A web node which is not a search node will start a reinitialization designed to reattach the node to ES cluster, and it was expected that the web node would not change information about the replica count since it is not responsible for storing data. However, the non-search node was changing the replica count due to a condition created when locking was not ready on the search node. This has been resolved with an update to insure replica count is updated only for index nodes in PegaSearchProviderImpl::reInitializeESNode.
Therefore it might be worthwhile to update to 8.6.6 to test your use case.
We understand that major upgrades require a lot of planning and time. Patch updates, however, are considered an integral part of keeping Pega up-to-date. Pega Platform patches bundle the bug and security fixes over a given period into a cumulative release that improves the overall quality of the software without touching the environment infrastructure.