Results Caching

Spice supports in-memory caching of query results, which is enabled by default for both the HTTP (/v1/sql) and Arrow Flight APIs.

Results caching can help improve performance for bursts of requests and for non-accelerated results such as refresh data returned on zero results.

Results caching employs a least-recently-used (LRU) cache replacement policy with the ability to specify an item expiry duration, which defaults to 1-second.

version: v1
kind: Spicepod
name: app

runtime:
  results_cache:
    enabled: true
    cache_max_size: 128MiB
    eviction_policy: lru
    item_ttl: 1s

Parameter name	Optional	Description
enabled	Yes	true by default
cache_max_size	Yes	Maximum cache size. Default is 128MiB
eviction_policy	Yes	Cache replacement policy when the cached data reaches the cache_max_size. Default and only currently supported value is lru
item_ttl	Yes	Cache entry expiration duration (Time to Live), 1 second by default.

Cached responses

The response includes an x-cache header indicating the cache status of the query:

Header value	Description
Hit from spiceai	The query result was served from cache
Miss from spiceai	The cache was checked but the result was not found
header not present	The cache was not used for this query (e.g., when cache-control: no-cache is specified or results cache is disabled)

Example cached response:

$ curl -XPOST -i http://localhost:8090/v1/sql -d 'select * from taxi_trips limit 1;'
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
x-cache: Hit from spiceai
vary: origin, access-control-request-method, access-control-request-headers
content-length: 416
date: Thu, 13 Feb 2025 03:05:39 GMT

Example uncached response:

$ curl -XPOST -i http://localhost:8090/v1/sql -d 'select * from taxi_trips limit 1;'
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
x-cache: Miss from spiceai
vary: origin, access-control-request-method, access-control-request-headers
content-length: 416
date: Thu, 13 Feb 2025 03:13:19 GMT

Example uncached response with cache-control: no-cache:

$ curl -H "cache-control: no-cache" -XPOST -i http://localhost:8090/v1/sql -d 'select * from taxi_trips limit 1;'
HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
vary: origin, access-control-request-method, access-control-request-headers
content-length: 416
date: Thu, 13 Feb 2025 03:14:00 GMT

Cache Control

The results cache behavior can be controlled for specific queries through HTTP headers. The Cache-Control header can be used to skip the cache for a specific query, but still cache the results for subsequent queries.

HTTP/Flight API

The SQL query API endpoints (both HTTP and Arrow Flight) understand the standard HTTP Cache-Control header, supporting the no-cache directive. When no-cache is specified, the cache is not used for the current query, but the query results are cached for subsequent queries.

Cache-Control directives other than no-cache are not supported.

HTTP Example

# Default behavior (use cache)
curl -XPOST http://localhost:8090/v1/sql -d 'SELECT 1'

# Skip cache for this query, but cache the results for future queries
curl -H "cache-control: no-cache" -XPOST http://localhost:8090/v1/sql -d 'SELECT 1'

Arrow FlightSQL Example

The following example shows how to skip the cache for a specific query using FlightSQL in Rust:

let sql_command = arrow_flight::sql::CommandStatementQuery {
    query: "SELECT 1".to_string(),
    transaction_id: None,
};
let sql_command_bytes = sql_command.as_any().encode_to_vec();

let mut request = FlightDescriptor::new_cmd(sql_command_bytes).into_request();

request
  .metadata_mut()
  .insert("cache-control", "no-cache");

// Send the request

spice sql CLI

The spice sql command accepts a --cache-control flag that follows the same behavior as the HTTP header:

# Default behavior (use cache if available)
spice sql

# Same as above
spice sql --cache-control cache

# Skip cache for this query, but cache the results for future queries
spice sql --cache-control no-cache