Licensing

Licensing

On Linux increase in file buffer cache results in performance loss Summary: Filling up of file cache results in significant performance loss. The increase in file cache buffer can happen over a period of time because of IO activity. Even after the Fluent session is exited, by default, the OS doesn’t free up the file cache buffer immediately (unless the OS is unable to satisfy a malloc request with available free memory). During memory allocation, this can result in allocation of memory from a different NUMA domain and hence can have significant impact on performance. IMPORTANT NOTE: Both of the options below require that machines are idle and no other job is running on the machine. Otherwise the other running jobs might be adversely impacted.

    • FAQFAQ
      Participant

      Resolution: Option # 1: File Cache Flushing As part of memory allocation if the OS reaches system limit then it will recover required memory by flushing unused cache. As part of flushing the cache, we allocate memory in chunks until we reach the system limit and hence will make the OS to flush the cache. This doesn’t require root privileges It takes some time (few minutes) to flush the cache In some cases, where swap is not available, the OS can kill the process as the allocation reaches system limit. So the flushing operation is done as a separate operation, even in the event of OS killing the process the cache flushing will be achieved. This option can be invoked via -cflush command line flag Option # 2: drop cache The other option is to drop cache via “/sbin/sysctl vm.drop_caches=3” This option requires “sudo” privilege for setting /sbin/sysctl vm.drop_caches=3. This option is instantaneous. This option can be invoked via scheme command as shown below two examples. You can choose any one of them as per your convenience. o (%node-run -1 “sudo /sbin/sysctl vm.drop_caches=3”) (or) If created as a wrapper script as shown in the below example with sudo privileges to execute below script: cat /……./dropcache #!/bin/sh /bin/sync /bin/sleep 10 # /sbin/sysctl vm.drop_caches=3 echo 3 > /proc/sys/vm/drop_caches Then this option can be invoked via scheme command (%node-run -1 “sudo //dropcache”)