Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
Expand All @@ -23,16 +23,18 @@ A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Desktop (please complete the following information):**

- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]

**Smartphone (please complete the following information):**

**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]

**Additional context**
Add any other context about the problem here.
3 changes: 0 additions & 3 deletions .github/ISSUE_TEMPLATE/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,4 @@ about: Describe this issue template's purpose here.
title: ''
labels: ''
assignees: ''

---


2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

**Have you contributed with at least a little donation to support the development of this project?**

- Paypal: https://www.paypal.me/kyberneees

**Is your feature request related to a problem? Please describe.**
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ jobs:
redis:
image: redis
ports:
- 6379:6379
- 6379:6379
options: --entrypoint redis-server

runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
- name: Setup Environment (Bun latest)
uses: oven-sh/setup-bun@v1
uses: oven-sh/setup-bun@v2
with:
bun-version: latest

Expand All @@ -26,4 +26,3 @@ jobs:

- name: Run tests
run: bun run test

22 changes: 17 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,18 @@
# Introduction

[![NPM version](https://badgen.net/npm/v/item-store-redis)](https://www.npmjs.com/package/item-store-redis)
[![NPM Total Downloads](https://badgen.net/npm/dt/item-store-redis)](https://www.npmjs.com/package/item-store-redis)
[![License](https://badgen.net/npm/license/item-store-redis)](https://www.npmjs.com/package/item-store-redis)
[![TypeScript support](https://badgen.net/npm/types/item-store-redis)](https://www.npmjs.com/package/item-store-redis)
[![Github stars](https://badgen.net/github/stars/BackendStack21/item-store-redis?icon=github)](https://github.com/BackendStack21/item-store-redis.git)

<img src="illustration.svg" width="400">
<img src="illustration.svg" width="400">

> Explanatory article: https://medium.com/@kyberneees/introducing-item-store-redis-a-high-performance-redis-based-item-store-for-efficient-data-4510d4e9fcf9

This module provides classes that implement repositories for storing and retrieving data in a Redis database:

- `ItemRepository`: The class provides basic functionality for storing and retrieving individual items, as well as querying all items or a subset of items in a paginated manner. The items are stored in Redis using a key-value data model with expiration support.
- `ItemRepository`: The class provides basic functionality for storing and retrieving individual items, as well as querying all items or a subset of items in a paginated manner. The items are stored in Redis using a key-value data model with expiration support.
- `SortedItemRepository`: The class provides a more advanced functionality for storing and retrieving items in a sorted manner. The items are sorted based on a numeric score, and can be queried in a paginated manner as well. The items are stored in Redis using a sorted set data model, making it more performant and efficient at scale (see `demos/benchmark.ts`). Per-item expiration is not supported, but size and age based retention is available through `maxItems`, `trimToSize` and `deleteByScore`.

```ts
Expand All @@ -21,6 +22,7 @@ import { ItemRepository, SortedItemRepository, IItem } from 'item-store-redis'
## Class ItemRepository

### Constructor

```ts
constructor (name: string, redis?: Redis | Cluster, options?: IRepositoryOptions<T>)

Expand All @@ -30,6 +32,7 @@ interface IRepositoryOptions<T> {
```

### Usage

```ts
import { ItemRepository, IItem } from 'item-store-redis'
import Redis from 'ioredis'
Expand All @@ -38,7 +41,10 @@ const redis = new Redis({ host: 'localhost', port: 6379 })
const repository = new ItemRepository<T>('my-unsorted-repo', redis)

await repository.set({ id: 'a', data }, 60) // expires in 60 seconds
await repository.setMany([{ id: 'b', data }, { id: 'c', data }])
await repository.setMany([
{ id: 'b', data },
{ id: 'c', data }
])
const [a, missing] = await repository.getByIds(['a', 'missing']) // missing === null

for await (const item of repository.iterate({ batchSize: 500 })) {
Expand All @@ -47,6 +53,7 @@ for await (const item of repository.iterate({ batchSize: 500 })) {
```

### Interface

```ts
interface IItemRepository<T> {
name: string
Expand Down Expand Up @@ -74,6 +81,7 @@ Keys are discovered with `SCAN` (never `KEYS`), on every master node when using
## Class SortedItemRepository

### Constructor

```ts
constructor (name: string, redis?: Redis | Cluster, options?: ISortedRepositoryOptions<T>)

Expand All @@ -85,6 +93,7 @@ interface ISortedRepositoryOptions<T> {
```

### Usage

```ts
import { SortedItemRepository, IItem } from 'item-store-redis'
import Redis from 'ioredis'
Expand All @@ -102,6 +111,7 @@ await repository.deleteByScore('-inf', (Date.now() - 86_400_000) * 1000) // rete
```

### Interface

```ts
interface ISortedItemRepository<T> {
name: string
Expand Down Expand Up @@ -142,12 +152,15 @@ interface IScoreRangeOptions {
See `libs/model.ts` for the documented interfaces.

## Validation

Pages start at `1`; `page`, `pageSize`, expirations and `n` arguments must be integers in range, otherwise a `RangeError` is thrown (previously, values like `pageSize = 0` or `n = 0` silently selected the whole repository). Item ids must be non-empty strings.

## Redis Cluster

Both repositories support `Cluster` connections. `SortedItemRepository` stores its index and data under two keys. On standalone Redis, writes and deletes touching both keys run as a `MULTI` transaction. On Redis Cluster those keys live in different slots, so the commands run in a safe order instead (data before index on writes, index before data on deletes) and readers never observe indexed items without data. Set `hashTag: true` to keep both keys in the same slot and get atomic transactions on Cluster as well. Note that this changes the key layout (`items:{name}:`), so existing data stored without the option is not visible with it.

## Custom serialization

```ts
const repository = new ItemRepository<T>('json-repo', redis, {
serializer: {
Expand All @@ -157,7 +170,6 @@ const repository = new ItemRepository<T>('json-repo', redis, {
})
```


# License

```
Expand All @@ -183,4 +195,4 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

```
```
89 changes: 55 additions & 34 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions bunfig.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[fmt]
printWidth = 120
semi = false
singleQuote = true
trailingComma = "none"
Loading
Loading