DigitalHippo Introduction

Overall Caching Process

1. Query Without Cache

Cache Penetration

Cache penetration occurs when someone queries nonexistent data. The cache cannot block these requests because the application doesn’t hold the information of the queried data’s existence.

We can typically address this issue by introducing a mechanism to determine a data’s existence. Two algorithms are commonly used for this purpose:

Bloom Filter

Bloom filter is a probabilistic data structure that tests whether an element is in a set, with a chance of false positives but no false negatives, saving memory for large datasets.

Bitmap

Bitmap is a memory-efficient data structure using bits to represent the presence or absence of elements within a fixed range.

I used Bitmap to solve this issue.

I used the bitmap functionality provided by Redis, using the data ID as the identifier.

If the bitmap indicates the data doesn’t exist, MS Product will throw an exception and flag the request as a potential attack.

Cache Breakdown

Cache breakdown occurs when the cache for a hot data expires, before a new cache can be added, a large number of requests will bypass the cache and putting pressure on the database.

To resolve this issue, we need to ensure that, in the same time, for the same data, only one request can reach to the database, while all other requests need to wait and retrieve that data from the cache.

Traditionally, we can use JUC (java.util.concurrent) to control the access to a data in multi-threaded/request environment. However, in a microservice application, this approach won’t work because requests are handled by multiple microservice instances, and JUC only manages locks within a single microservice instance. In this case, we need a distributed lock mechanism that operates outside the individual microservices, ensuring the lock is independent of any specific microservice instance.

I used the Redis Lock (RLock) with Redisson. The logic is as follows:

Pseudocode:

// get distributed lock from redisson
rLock = redissonClient.getLock(lockKey)

// try lock
locked = rLock.tryLock()

if (locked) {
    // you have the lock
    // query database
    data = pjp.proceed()

    // Save record into cache
    cacheService.saveDataToCache(data)

    // Return data
    return data
} else {
    // all other requests need to wait and query the cache
    Thread.sleep(300)
    return cacheService.getDataFromCache()
}

Cache Avalanche

Cache avalanche occurs when a large number of caches expire at the same time, immediately after a lot of requests come in querying those data whose cache has just expired. This will add significant pressure to the database.

This scenario rarely happens, and it’s relatively easy to prevent. We can add a random TTL when saving caches. We can also load hot data into the cache before system startup or before a large-scale cache expiration.

Session-Based Token Auth

Thank you for reading, I’m still working on this document.

Comparison with JWT