Gorse v0.3.2

Feature

• Support modify configuration via environment variables (#359).
  There are 8 environment variables available:

Environment Variable	Configuration
GORSE_CACHE_STORE	cache_store	database for caching.
GORSE_DATA_STORE	data_store	database for persist data.
GORSE_MASTER_PORT	port	master port
GORSE_MASTER_HOST	host	master host
GORSE_MASTER_HTTP_PORT	http_port	HTTP API port
GORSE_MASTER_HTTP_HOST	http_host	HTTP API host
GORSE_MASTER_JOBS	n_jobs	number of working jobs
GORSE_SERVER_API_KEY	api_key	secret key for RESTful APIs

• (Experimental) Support IVF-based neighborhood searching (#363).
• (Experimental) Support HNSW based recommended items searching (#368).

Fix

• Split large Redis writes to batches (#352).
• Allow empty JSON string in relational databases (#355).
• Fix offset overflow in recommend API (#365).
• Optimize dashboard overview page layout (#369).

Upgrade Guide

• Configuration: IVF-based neighborhood searching and HNSW based recommended items searching are experimental features.
  • Use enable_xxx_index to enable experimental features.
  • xxx_index_recall is the expected recall of approximate searching.
  • xxx_index_fit_epoch is the number of epochs to adapt indices.

These approximate vector searching indices trade tolerable accuracy (recall) with larger throughput. The index builder tries to reach xxx_index_recall in xxx_index_fit_epoch. The building process will stop when xxx_index_recall or xxx_index_fit_epoch reached. Index-based searching costs more memory than brute force searching and its building process costs additional time, but they are negligible compared to the benefits.

# Enable approximate item neighbor searching using vector index.
enable_item_neighbor_index = false

# Minimal recall for approximate item neighbor searching.
item_neighbor_index_recall = 0.8

# Maximal number of fit epochs for approximate item neighbor searching vector index.
item_neighbor_index_fit_epoch = 3

# Enable approximate user neighbor searching using vector index.
enable_user_neighbor_index = false

# Minimal recall for approximate user neighbor searching.
user_neighbor_index_recall = 0.8

# Maximal number of fit epochs for approximate user neighbor searching vector index.
user_neighbor_index_fit_epoch = 3

# Enable approximate collaborative filtering recommend using vector index.
enable_collaborative_index = false

# Minimal recall for approximate collaborative filtering recommend.
collaborative_index_recall = 0.9

# Maximal number of fit epochs for approximate collaborative filtering recommend vector index.
collaborative_index_fit_epoch = 3

• Redis: Remove incompatible stale cache.

redis-cli KEYS "item_neighbors*" | xargs redis-cli DEL
redis-cli KEYS "user_neighbors*" | xargs redis-cli DEL