feat: make eviction policy configurable and add LFU

[mehcode]

Closes #1530

• Introduce pgdog-cache::Cache<K, V> which is a policy-configurable (LRU or exact-LFU) single-threaded cache implementation. pgdog-cache::Cache<K, V> has O(1) insertion, removal, and lookup.

• Switch the per-connection statement cache to use pgdog_cache::Cache rather than lru::LruCache.

• Add prepared_statements_eviction to [general] taking lru (default) or lfu (also available through PGDOG_PREPARED_STATEMENTS_EVICTION and SET ...).

---

I investigated several crates before committing to pgdog_cache::Cache<K, V>. The per-connection cache in pgdog is interesting because admission to the cache must not affect existing entries and entries can only manually evicted later. Most crates in the ecosystem do not work like that. This is because an eviction of the cache involves sending a Close message to postgres. Additionally several of the crates are either over-tuned for async (with unnecessary overhead for a per-connection cache) or unusable with async (not Send).

Crate	Manual Evict	Disable Auto Evict	Send	Unsafe
pgdog_cache	Yes	N/A	Yes	0 lines
cachekit 0.8.0	Yes	No	Yes	539 lines
evictor 0.7.2	Yes	No	No	0 lines
lru 0.18.14	Yes	Yes	Yes	137 lines
lfu 0.2.5	No	No	No	0 lines
lfu_cache 1.3.0	Yes	Yes	No	87 lines
schnellru 0.2.4	Yes	Yes	Yes	336 lines

---

Full benchmark results of pgdog_cache on my local machine. tldr, pgdog_cache::Cache handily beats every LFU cache crate under review and is near parity with the best LRU crates at representative workloads (see the traffic benchmark).

Details

test insert/pgdog-lru/64 ...       bench:      1,068 ns/iter (+/- 14)
test insert/lru/64 ...             bench:      2,808 ns/iter (+/- 33)
test insert/schnellru/64 ...       bench:      1,654 ns/iter (+/- 48)
test insert/cachekit-lru/64 ...    bench:      2,796 ns/iter (+/- 116)
test insert/evictor-lru/64 ...     bench:      1,649 ns/iter (+/- 34)
test insert/pgdog-lfu/64 ...       bench:      1,572 ns/iter (+/- 29)
test insert/cachekit-lfu/64 ...    bench:      5,204 ns/iter (+/- 36)
test insert/evictor-lfu/64 ...     bench:      1,737 ns/iter (+/- 52)
test insert/pgdog-lru/1024 ...     bench:     25,063 ns/iter (+/- 134)
test insert/lru/1024 ...           bench:     46,045 ns/iter (+/- 304)
test insert/schnellru/1024 ...     bench:     28,090 ns/iter (+/- 167)
test insert/cachekit-lru/1024 ...  bench:     46,458 ns/iter (+/- 101)
test insert/evictor-lru/1024 ...   bench:     27,230 ns/iter (+/- 98)
test insert/pgdog-lfu/1024 ...     bench:     25,413 ns/iter (+/- 165)
test insert/cachekit-lfu/1024 ...  bench:     84,854 ns/iter (+/- 130)
test insert/evictor-lfu/1024 ...   bench:     28,299 ns/iter (+/- 119)

test get/pgdog-lru/64 ...          bench:        523 ns/iter (+/- 40)
test get/lru/64 ...                bench:        499 ns/iter (+/- 20)
test get/schnellru/64 ...          bench:        457 ns/iter (+/- 16)
test get/cachekit-lru/64 ...       bench:        406 ns/iter (+/- 11)
test get/evictor-lru/64 ...        bench:        656 ns/iter (+/- 35)
test get/pgdog-lfu/64 ...          bench:        655 ns/iter (+/- 35)
test get/cachekit-lfu/64 ...       bench:      2,363 ns/iter (+/- 40)
test get/evictor-lfu/64 ...        bench:      1,163 ns/iter (+/- 37)
test get/pgdog-lru/1024 ...        bench:     10,083 ns/iter (+/- 680)
test get/lru/1024 ...              bench:      9,749 ns/iter (+/- 113)
test get/schnellru/1024 ...        bench:     10,227 ns/iter (+/- 622)
test get/cachekit-lru/1024 ...     bench:      8,935 ns/iter (+/- 62)
test get/evictor-lru/1024 ...      bench:     14,832 ns/iter (+/- 603)
test get/pgdog-lfu/1024 ...        bench:     13,520 ns/iter (+/- 570)
test get/cachekit-lfu/1024 ...     bench:     44,016 ns/iter (+/- 110)
test get/evictor-lfu/1024 ...      bench      20,519 ns/iter (+/- 703)

test peek/pgdog-lru/64 ...         bench:        336 ns/iter (+/- 3)
test peek/lru/64 ...               bench:        235 ns/iter (+/- 3)
test peek/schnellru/64 ...         bench:        275 ns/iter (+/- 3)
test peek/cachekit-lru/64 ...      bench:        226 ns/iter (+/- 1)
test peek/evictor-lru/64 ...       bench:        286 ns/iter (+/- 2)
test peek/pgdog-lfu/64 ...         bench:        327 ns/iter (+/- 2)
test peek/cachekit-lfu/64 ...      bench:        522 ns/iter (+/- 1)
test peek/evictor-lfu/64 ...       bench:        287 ns/iter (+/- 2)
test peek/pgdog-lru/1024 ...       bench:      7,944 ns/iter (+/- 59)
test peek/lru/1024 ...             bench:      5,820 ns/iter (+/- 45)
test peek/schnellru/1024 ...       bench:      6,642 ns/iter (+/- 51)
test peek/cachekit-lru/1024 ...    bench:      5,675 ns/iter (+/- 44)
test peek/evictor-lru/1024 ...     bench:      7,246 ns/iter (+/- 60)
test peek/pgdog-lfu/1024 ...       bench:      7,754 ns/iter (+/- 52)
test peek/cachekit-lfu/1024 ...    bench:     11,516 ns/iter (+/- 15)
test peek/evictor-lfu/1024 ...     bench:      7,126 ns/iter (+/- 33)

test remove/pgdog-lru/64 ...       bench:      1,590 ns/iter (+/- 66)
test remove/lru/64 ...             bench:      1,962 ns/iter (+/- 36)
test remove/schnellru/64 ...       bench:      1,251 ns/iter (+/- 27)
test remove/cachekit-lru/64 ...    bench:      2,603 ns/iter (+/- 25)
test remove/evictor-lru/64 ...     bench:      1,287 ns/iter (+/- 24)
test remove/pgdog-lfu/64 ...       bench:      1,542 ns/iter (+/- 36)
test remove/cachekit-lfu/64 ...    bench:      5,356 ns/iter (+/- 47)
test remove/evictor-lfu/64 ...     bench:      1,980 ns/iter (+/- 70)
test remove/pgdog-lru/1024 ...     bench:     30,090 ns/iter (+/- 647)
test remove/lru/1024 ...           bench:     40,262 ns/iter (+/- 272)
test remove/schnellru/1024 ...     bench:     27,378 ns/iter (+/- 860)
test remove/cachekit-lru/1024 ...  bench:     49,076 ns/iter (+/- 218)
test remove/evictor-lru/1024 ...   bench:     29,915 ns/iter (+/- 875)
test remove/pgdog-lfu/1024 ...     bench:     31,062 ns/iter (+/- 287)
test remove/cachekit-lfu/1024 ...  bench:     90,640 ns/iter (+/- 513)
test remove/evictor-lfu/1024 ...   bench:     41,519 ns/iter (+/- 395)

test pop/pgdog-lru/64 ...          bench:      1,529 ns/iter (+/- 38)
test pop/lru/64 ...                bench:      1,864 ns/iter (+/- 36)
test pop/schnellru/64 ...          bench:        834 ns/iter (+/- 35)
test pop/cachekit-lru/64 ...       bench:      3,503 ns/iter (+/- 93)
test pop/evictor-lru/64 ...        bench:      1,208 ns/iter (+/- 30)
test pop/pgdog-lfu/64 ...          bench:      1,534 ns/iter (+/- 38)
test pop/cachekit-lfu/64 ...       bench:      5,591 ns/iter (+/- 81)
test pop/evictor-lfu/64 ...        bench:      2,370 ns/iter (+/- 63)
test pop/pgdog-lru/1024 ...        bench:     23,929 ns/iter (+/- 373)
test pop/lru/1024 ...              bench:     30,338 ns/iter (+/- 253)
test pop/schnellru/1024 ...        bench:     13,691 ns/iter (+/- 121)
test pop/cachekit-lru/1024 ...     bench:     56,242 ns/iter (+/- 337)
test pop/evictor-lru/1024 ...      bench:     19,742 ns/iter (+/- 163)
test pop/pgdog-lfu/1024 ...        bench:     23,943 ns/iter (+/- 399)
test pop/cachekit-lfu/1024 ...     bench:     89,370 ns/iter (+/- 609)
test pop/evictor-lfu/1024 ...      bench:     38,383 ns/iter (+/- 391)

test traffic/pgdog-lru/64 ...      bench:      12,991 ns/iter (+/- 274)
test traffic/lru/64 ...            bench:      14,628 ns/iter (+/- 138)
test traffic/schnellru/64 ...      bench:      12,879 ns/iter (+/- 188)
test traffic/cachekit-lru/64 ...   bench:      23,020 ns/iter (+/- 597)
test traffic/evictor-lru/64 ...    bench:      14,923 ns/iter (+/- 160)
test traffic/pgdog-lfu/64 ...      bench:      13,048 ns/iter (+/- 400)
test traffic/cachekit-lfu/64 ...   bench:      53,384 ns/iter (+/- 1,269)
test traffic/evictor-lfu/64 ...    bench:      23,080 ns/iter (+/- 599)
test traffic/pgdog-lru/1024 ...    bench:     193,915 ns/iter (+/- 2,807)
test traffic/lru/1024 ...          bench:     211,773 ns/iter (+/- 1,934)
test traffic/schnellru/1024 ...    bench:     180,488 ns/iter (+/- 1,959)
test traffic/cachekit-lru/1024 ... bench:     241,637 ns/iter (+/- 2,590)
test traffic/evictor-lru/1024 ...  bench:     219,829 ns/iter (+/- 2,153)
test traffic/pgdog-lfu/1024 ...    bench:     241,257 ns/iter (+/- 4,457)
test traffic/cachekit-lfu/1024 ... bench:     799,642 ns/iter (+/- 6,310)
test traffic/evictor-lfu/1024 ...  bench:     417,210 ns/iter (+/- 3,613)

---

I see the codecov report, I can add more unit test coverage early next week. Leaving this as draft pending additional unit test coverage.

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

❌ Patch coverage is 57.23577% with 263 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pgdog-cache/src/queue.rs	51.95%	160 Missing ⚠️
pgdog-cache/src/cache.rs	45.74%	102 Missing ⚠️
pgdog-config/src/pooling.rs	83.33%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!