Warden

Warden is an outgoing request optimizer for creating fast and scalable applications.

CircleCI npm npm Known Vulnerabilities codecov Codacy

Features

Warden Achitecture

Getting started

Installing

Yarn

yarn add puzzle-warden

Npm

npm i puzzle-warden --save

Quick Guide

1. Register Route

const warden = require('puzzle-warden');
warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

2. Send Request

Using Route Registration
const routeRegistration = warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

routeRegistration({
  url: `https://postman-echo.com/get?foo=value`,
  headers: {
    cookie: `bar=value`
  },
  method: 'get',
  gzip: true,
  json: true
}, (err, response, data) => {
  console.log(data);
});
Using Warden
warden.request('test', {
  url: `https://postman-echo.com/get?foo=value`,
  headers: {
    cookie: `bar=value`
  },
  method: 'get',
  gzip: true,
  json: true
}, (err, response, data) => {
  console.log(data);
});
Using Request Module
request({
  name: 'test',
  url: `https://postman-echo.com/get?foo=value`,
  headers: {
    cookie: `bar=value`
  },
  method: 'get',
  gzip: true,
  json: true
}, (err, response, data) => {
  console.log(data);
});

Identifier

Warden uses identifiers to convert HTTP requests to unique keys. Using these keys it is able to implement cache, holder and other stuff. Let’s assume we want to send a GET request to https://postman-echo.com/get?foo=value&bar=anothervalue. And we want to cache responses based on query string foo. We should use the identifier {query.foo}. There are 5 types of identifier variables.

You can also use javascript to create custom identifiers.

Identifiers can be chained like {query.foo}_{cookie.bar}.

Identifiers get converted to keys for each request. Let’s assume we have an identifier like {query.foo}_{method} We use this identifier for a GET request to /path?foo=bar. Then the unique key of this request will be bar_GET.

Registering Route

You can simply register a route providing an identifier and module configurations. Please see Identifier

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

identifier is an optional field. If an identifier is not provided warden will be use generic identifier which is ${name}_${url}_${JSON.stringify({cookie, headers, query})}_${method}.

Cache

You can simply enable cache with default values using.

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: true,
  holder: true
});

Or you can customize cache configuration by passing an object.

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: {
    plugin: 'memory',
    strategy: 'CacheThenNetwork',
    duration: '2m'
  },
  holder: true
});

Default values and properties

Property Required Default Value Definition
plugin ❌ memory Where cached data will be stored. Please see Cache Plugins for more information. Currently, only memory available.
strategy ❌ CacheThenNetwork Controls when and how things will be cached. Please see Caching Strategy for more information.
duration ❌ 1m Caching Duration. You can use number for ms. Or you can use 1m 1h 1d etc. Please see ms for full list
cacheWithCookie ❌ false Warden never caches responses including set-cookie header. To enable this pass this property as true

Cache Plugins

Cache plugins control where cache will be stored. These are available plugins:

Custom Plugins

Anything that implements interface below can be used as Cache Plugin.

interface CachePlugin {
  get<T>(key: string): Promise<T | null>;

  set(key: string, value: unknown, ms?: number): Promise<void>;
}

You first register the cache plugin

warden.registerCachePlugin('mySuperCache', {
  set(){},
  get(){}
});

Then make route configuration with your plugin name

warden.register('test', {
  identifier: '{query.foo}_{cookie.bar}',
  cache: {
    plugin: 'mySuperCache',
    strategy: 'CacheThenNetwork',
    duration: '2m'
  },
  holder: true
});

Caching Strategy

Caching strategies defines how things will be cached and when cached responses will be used. Currently, the only available caching strategy is CacheThenNetwork

CacheThenNetwork

Simple old school caching. Asks cache plugin if it has a valid cached response. If yes, returns the cached value as the response. If no, passes the request to the next handler. When it receives the response, it caches and returns the value as a response.

Holder

Holder prevents same HTTP requests to be sent at the same time. Let’s assume we have an identifier for a request: {query.foo}. We send a HTTP request /product?foo=bar. While waiting for the response, warden received another HTTP request to the same address which means both HTTP requests are converted to the same key. Then Warden stops the second request. After receiving the response from the first request, Warden returns both requests with the same response by sending only one HTTP request.

Schema

Warden uses custom object -> string transformation to improve performance. Schema will only affect POST requests with json body.

warden.register('test', {
  identifier: '{query.foo}',
  schema: {
    type: 'object',
    properties: {
      name: {
        type: 'string'
      },
      age: {
        type: 'number'
      }
    }
  }
});

warden.request('test', {
  url: 'https://github.com/puzzle-js/puzzle-warden?foo=bar',
  method: 'post',
  json: true,
  body: {
    name: 'Test',
    age: 23
  }
}, (err, response, data) => {
  console.log(data);
})

To enable Schema module, you need to give schema option when registering route. This schema options must be compatible with jsonschema

You should use json: true property.

Retry

When the connection fails with one of ECONNRESET, ENOTFOUND, ESOCKETTIMEDOUT, ETIMEDOUT, ECONNREFUSED, EHOSTUNREACH, EPIPE, EAI_AGAIN or when an HTTP 5xx or 429 error occurrs, the request will automatically be re-attempted as these are often recoverable errors and will go away on retry.

warden.register('routeName', {
  retry: {
    delay: 100,
    count: 1,
    logger: (retryCount) => {
      console.log(retryCount);
    }
  }
}); 

warden.register('routeName', {
  retry: true // default settings
});

Default values and properties

Property Required Default Value Definition
delay ❌ 100 Warden will wait for 100ms before retry
count ❌ 1 It will try for 1 time by default
logger ❌   Logger will be called on each retry with retry count

Debugger

Warden comes with built-in debugger to provide information about request flow.

To enable debug mode:

warden.debug = true;

Flow will be visible on console.

Example log:

4323 | foo_44: HOLDER ---> CACHE

This means the request with the unique id 4323 and identifier value foo_44 is moving from Holder to Cache

Api

warden.register()

Check Registering Route section for better information and usage details

warden.register('routeName', routeConfiguration);

warden.request()

Sends a HTTP request using warden (internally uses request)

warden.request('test', {
  url: `https://postman-echo.com/get?foo=value`,
  method: 'get'
}, (err, response, data) => {
  console.log(data);
});

Any valid property for request module can be used.

warden.requestConfig()

Works exactly like request defaults. It can be used for settings default values for requests.

warden.requestConfig({
  headers: {'x-token': 'my-token'}
});

Sets x-token header with value my-token for all HTTP requests

warden.isRouteRegistered()

Checks whether route is registered.

warden.isRouteRegistered('route'); // true | false

warden.unregisterRoute()

Unregisters route

warden.unregisterRoute('route');

warden.registerCachePlugin()

Registers cache plugin

warden.registerCachePlugin('pluginName', {
  set(){
    
  },
  get(){
    
  }
});

warden.debug

Enables debug mode or disables based on boolean

warden.debug = true;