Why should you disable THP (Transparent Huge Pages)?
Transparent hugepages can take up too much memory even if the application doesn't need to allocate so much memory. Typically memory pages may be 4KB, but if transparent huge pages are not disabled, they could be up to a few GB (usually about 2MB or more). This means anytime any memory is allocated, it would claim a memory block of this size, even for tiny content.
Linux THP transparently re-organizes memory used by a process inside the kernel, merging some small pages into huge pages and splitting some huge pages into small pages. This background process holds locks on the memory manager and can prevent your process from accessing memory.
Since SingleStore uses a lot of memory, the THP process can potentially block memory use for seconds at a time. This THP lag can be challenging to diagnose, as it can manifest in several subtle ways. One such way it is manifested at the database level is by sporadic query times. For example, if a query usually takes 100ms, but occasionally takes 5 seconds, it may be stuck waiting for a THP defragmentation on one of the leaves. Generally, this will affect all queries currently active on that leaf, so you would often see a group of queries all has a latency spike.
You may also detect this as high system CPU utilization. If you have hundreds of gigabytes of memory in a single node, THP defrags can be detectable as seconds-long spikes of system CPU. See the Disable Transparent Huge Pages section of the documentation on the System Requirements and Recommendations page for more details.