pantry

Search:
  Source   Edit

This library is a small wrapper around Pantry which is a simple json storage service.

Key information for when following this guide

Common terms

  • Basket: A JSON object that you store data in
  • Pantry: A collection of baskets

Limitations

  • Each bucket can only store 1.44mb max
  • Each pantry can only store 100 buckets max
  • Each pantry can't be over 100mb in size
  • Inactive buckets are deleted after 30 days

Create a pantry and then you can follow along with the rest of the docs.

Creating the client

The client is made with either newPantryClient or newAsyncPantryClient which provide sync and async apis respectively. These clients can be passed a RetryStrategy which handles how they handle a timeout

Example: cmd: -r:off

import pantry
import std/asyncdispatch
let
  # Create normal sync client that will error on 'too many requests'
  client = newPantryClient("pantry-id")
  # Create async client that will sleep and then retry on 'too many requests'
  aClient = newAsyncPantryClient("pantry-id", strat = Retry)

try:
  for i in 1..100:
    echo client.getDetails()
except TooManyPantryRequests:
  echo "Slow down!"

proc spamDetails() {.async.} =
  # This will not error out since it will just sleep if
  # making too many requests
  for i in 1..100:
    echo await aClient.getDetails()

waitFor spamDetails()
close client
close aClient

Adding Data

Buckets can have their data set via the create proc

Warning: This overwrites any data currently present in the bucket

Example: cmd: -r:off

import pantry
let client = newPantryClient("pantry-id")

var data = %* {
  "lenFilms": 145,
  "genres": ["Comedy", "Horror"]
}
# Set the "films" bucket to contain the data
client.create("films", data)

# Reset the bucket so it only contains number of films
client.create("films", %* {"lenFilms": 0})

Getting Data

Once a bucket has data you can then retrive it at a later date using get

Example: cmd: -r:off

import pantry
let client = newPantryClient("pantry-id")

# See example in adding data to see what this is
let films = client.get("films")

assert films["lenFilms"].getInt() == 0
assert "genres" notin films

Updating Data

Buckets can have their contents updated using update which means

  • Values of existing keys are overwritten
  • Values of new keys are added
  • Arrays are merged together

Example: cmd: -r:off

import pantry
let client = newPantryClient("pantry-id")

# Lets add back in the genres and bump up the number of films
discard client.update("films", %* {
  "genres": ["Comedy", "Horror"],
  "lenFilms": 99
})

type
  FilmData = object
    lenFilms: int
    genres: seq[string]

# Now lets add in another genre
let newData = client.update("films", %* {
  "genres": ["Sci-Fi"]
}).to(FilmData)


assert newData.lenFilms == 99
assert newData.genres == ["Comedy", "Horror", "Sci-Fi"]

Removing Data

Very simple operation, deletes the bucket

Danger: Operation cannot be undone, be careful

Example: cmd: -r:off

import pantry
let client = newPantryClient("pantry-id")

client.delete("films")

Using objects instead of JsonNode

It is possible to use objects for sending and receiving objects instead of having to work with JsonNode. This can be done by just passing an object instead of json like so

Example: cmd: -r:off

import pantry
type
  FilmData = object
    lenFilms: int
    genres: seq[string]

let
  client = newPantryClient("pantry-id")
  data = FilmData(lenFilms: 9, genres: @["Comedy", "Action", "Adventure"])

# API is the same has before, except you need to specify type when getting
client.create("films", data)
assert client.get("films", FilmData) == data
This also allows you to avoid exceptions and get Option[T] return types instead

Example: cmd: -r:off

import pantry
import std/options
let client = newPantryClient("pantry-id")

# if 'doesntExist' doesn't exist in the pantry then BasketDoesntExist exception
# would be thrown
try:
  discard client.get("doesntExist")
except BasketDoesntExist:
  discard

# Doesn't need to be JsonNode, any type works
let data = client.get("doesntExist", Option[JsonNode])
# No exception will be raised if it doesn't exist, but it will be `none`
if data.isSome:
  # Do stuff if the basket exists
  discard
else:
  # Do stuff if the basket doesn't exist
  discard

Types

AsyncPantryClient = BasePantryClient[AsyncHttpClient]
  Source   Edit 
BasePantryClient[T] = object
  client: T
  id*: string
  retry*: int
  strat*: RetryStrategy
  Source   Edit 
Basket = object
  name*: string
  ttl*: int
A basket stores json   Source   Edit 
BasketDoesntExist = object of PantryError
Raised if you make a request to a basket that doesn't exist   Source   Edit 
InvalidPantryID = object of PantryError
Raised if you make a request use a pantry ID that is invalid   Source   Edit 
PantryClient = BasePantryClient[HttpClient]
  Source   Edit 
PantryDetails = object
  name*, description*: string
  errors*: seq[string]
  notifications*: bool
  percentFull*: int
  baskets*: Table[string, Basket]
Contains information about a pantry   Source   Edit 
PantryError = object of CatchableError
  Source   Edit 
RetryStrategy = enum
  Exception, Retry, Sleep
Strategy to imploy when pantry gives a too many requests error
  • Exception: Throws an exception (default)
  • Retry: Trys again to make the request
  • Sleep: Waits until you can call pantry again but doesn't retry
  Source   Edit 
TooManyPantryRequests = object of PantryError
Raised when you are calling pantry too many times (limit is 2 times per second)   Source   Edit

Procs

proc close(pc: BasePantryClient)
Closes the pantry client. Closing is automatically done so you shouldn't need to close this unless you want to close earlier than the GC   Source   Edit 
proc create(pc: AsyncPantryClient; basket: string; data: JsonNode): owned(
    Future[void]) {....raises: [Exception],
                    tags: [RootEffect, TimeEffect, ReadIOEffect].}
Creates a basket in a pantry. If the basket already exists then it overwrites it   Source   Edit 
proc create(pc: PantryClient; basket: string; data: JsonNode) {....raises: [
    ValueError, HttpRequestError, LibraryError, Exception, OSError, SslError,
    IOError, TimeoutError, ProtocolError, KeyError, BasketDoesntExist,
    InvalidPantryID, TooManyPantryRequests, PantryError],
    tags: [RootEffect, ReadIOEffect, WriteIOEffect, TimeEffect].}
  Source   Edit 
proc create[T: not JsonNode](pc: AsyncPantryClient; basket: string; data: T): owned(
    Future[void])
like create except it works with normal objects   Source   Edit 
proc create[T: not JsonNode](pc: PantryClient; basket: string; data: T)
  Source   Edit 
proc delete(pc: AsyncPantryClient; basket: string): owned(Future[void]) {.
    ...raises: [Exception], tags: [RootEffect, TimeEffect, ReadIOEffect].}
Delete the entire basket. Warning, this action cannot be undone.   Source   Edit 
proc delete(pc: PantryClient; basket: string) {....raises: [ValueError,
    HttpRequestError, LibraryError, Exception, OSError, SslError, IOError,
    TimeoutError, ProtocolError, KeyError, BasketDoesntExist, InvalidPantryID,
    TooManyPantryRequests, PantryError],
    tags: [RootEffect, ReadIOEffect, WriteIOEffect, TimeEffect].}
  Source   Edit 
proc get(pc: AsyncPantryClient; basket: string): Future[JsonNode] {.
    ...raises: [Exception, ValueError],
    tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
Given a basket name, return the full contents of the basket.   Source   Edit 
proc get(pc: PantryClient; basket: string): JsonNode {....raises: [IOError,
    OSError, JsonParsingError, ValueError, Exception, HttpRequestError,
    LibraryError, SslError, TimeoutError, ProtocolError, KeyError,
    BasketDoesntExist, InvalidPantryID, TooManyPantryRequests, PantryError],
    tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
  Source   Edit 
proc get[T](pc: AsyncPantryClient; basket: string; kind: typedesc[T]): Future[T]
Like create except it parses the JSON and returns an object   Source   Edit 
proc get[T](pc: PantryClient; basket: string; kind: typedesc[T]): T
  Source   Edit 
proc getDetails(pc: AsyncPantryClient): Future[PantryDetails] {.
    ...raises: [Exception, ValueError],
    tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
Returns details for current pantry   Source   Edit 
proc getDetails(pc: PantryClient): PantryDetails {....raises: [KeyError,
    ValueError, JsonKindError, IOError, OSError, JsonParsingError, Exception,
    HttpRequestError, LibraryError, SslError, TimeoutError, ProtocolError,
    BasketDoesntExist, InvalidPantryID, TooManyPantryRequests, PantryError],
    tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
  Source   Edit 
proc newAsyncPantryClient(id: string; strat: RetryStrategy = Exception): AsyncPantryClient {.
    ...raises: [LibraryError, SslError, Exception, IOError],
    tags: [RootEffect, ReadDirEffect, ReadEnvEffect].}
Creates a Pantry client for async use   Source   Edit 
proc newPantryClient(id: string; strat: RetryStrategy = Exception): PantryClient {.
    ...raises: [LibraryError, SslError, Exception, IOError],
    tags: [RootEffect, ReadDirEffect, ReadEnvEffect].}
Creates a Pantry client for sync use   Source   Edit 
proc update(pc: AsyncPantryClient; basket: string; newData: JsonNode): Future[
    JsonNode] {....raises: [Exception, ValueError],
                tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
Given a basket name, this will update the existing contents and return the contents of the newly updated basket. This operation performs a deep merge and will overwrite the values of any existing keys, or append values to nested objects or arrays. Returns the updated data.   Source   Edit 
proc update(pc: PantryClient; basket: string; newData: JsonNode): JsonNode {....raises: [
    IOError, OSError, JsonParsingError, ValueError, Exception, HttpRequestError,
    LibraryError, SslError, TimeoutError, ProtocolError, KeyError,
    BasketDoesntExist, InvalidPantryID, TooManyPantryRequests, PantryError],
    tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
  Source   Edit 
proc update[T: not JsonNode](pc: AsyncPantryClient; basket: string; newData: T): Future[
    T]
Like update except data is an object   Source   Edit 
proc update[T: not JsonNode](pc: PantryClient; basket: string; newData: T): T
  Source   Edit 
proc updateDetails(pc: AsyncPantryClient; name, description: string): Future[
    PantryDetails] {....raises: [Exception, ValueError], tags: [ReadIOEffect,
    WriteIOEffect, RootEffect, TimeEffect].}
Updates a pantry's details   Source   Edit 
proc updateDetails(pc: PantryClient; name, description: string): PantryDetails {....raises: [
    KeyError, ValueError, JsonKindError, IOError, OSError, JsonParsingError,
    Exception, HttpRequestError, LibraryError, SslError, TimeoutError,
    ProtocolError, BasketDoesntExist, InvalidPantryID, TooManyPantryRequests,
    PantryError], tags: [ReadIOEffect, WriteIOEffect, RootEffect, TimeEffect].}
  Source   Edit

Made with Nim. Generated: 2023-06-05 12:01:22 UTC
Other Nim projects by me