Cache Eviction
Cache Eviction Overview
As the storage space used by Alluxio is limited, workers evict old data through several strategies to ensure that there is enough storage space to cache new data.
There are two different ways Alluxio will evict its cache:
Evict on Writing
Background Asynchronous Evicting
Evict on Writing
Evict on writing is to synchronously check and eliminate the cached data when writing pages in Alluxio. The eviction will be triggered when Alluxio is about to write a page that would cause the total cache to exceed the storage capacity.
Cache Evictors
Alluxio provides the following five eviction algorithms to evict cached data:
LRU (default): LRU cache eviction policy
FIFO: FIFO cache eviction policy
LFU: LFU cache eviction policy. Pages are sorted in bucket order based on logarithmic count. Pages inside the bucket are sorted in LRU order.
Nondeterministic LRU: LRU with non-deterministic cache eviction policy. Uniformly evict elements in the LRU tail.
Two Choice Random: Two Choice Random client-side cache eviction policy. It selects two random page IDs and evicts the one least-recently used.
The following configuration in alluxio-site.properties sets LRU eviction policy for worker cache.
Available options are LRU, LFU, FIFO and RANDOM. If Nondeterministic LRU is needed, please set the configuration
as follows:
Background Asynchronous Evicting
Alluxio supports setting different constraints on the cache space that will trigger an eviction:
Setting a limit on the total size that the pages can occupy, i.e. the capacity of the page store;
Setting a limit on the total number of pages in the page store.
On every page put operation, Alluxio checks if any of the constraints is violated. If a constraint is violated, a synchronous eviction takes place to make room for the incoming page. However, synchronous eviction on writing will degrade performance dramatically. Background asynchronous eviction aims at evicting cached data asynchronously beforehand to avoid evicting cached data during a write operation.
Eviction based on Capacity
To enable the background asynchronous evicting feature, add the following configurations to alluxio-site.properties:
By setting the above configuration, Alluxio will create a background thread that checks if the page cache space reaches the high water mark threshold. Once this condition is triggered, it will evict cached pages until the low water mark threshold is reached. The background thread will check periodically defined by the check interval property.
Eviction based on Limit on Number of Pages
Alluxio asynchronously evicts pages that exceeds the limit, in a background thread that periodically scans for excessive pages.
When the total page number exceeds highWatermark * maxPageNumberLimit, it triggers an eviction, until the total page number drops below lowWatermark * maxPageNumberLimit.
To enable this async eviction by page number limit, set alluxio.worker.page.store.max.page.number.limit.enabled to true.
The limit on the maximum number of pages can be specified by alluxio.worker.page.store.max.page.number:
REST API for Updating Configurations Dynamically
Alluxio provides the following REST APIs for users to set and get async eviction configurations dynamically:
Enable async eviction and update the related parameters
Disable async eviction
Get the current async eviction parameters
Last updated