1 - Speed Tables - a High-performance, Memory-resident Database

This chapter gives an overview of speed tables and describes the sort of applications they are useful for.

Speed tables provides an interface for defining tables containing zero or more rows, with each row containing one or more fields. The speed table compiler reads the table definition and generates C code to create and manage corresponding structures, generating a custom C language Tcl extension to create, access and manipulate those tables and building a native code package in a shared library that is loadable on demand via Tcl's "package require" mechanism.

Speed tables are well-suited for applications for which this table/row/field abstraction is useful, with row counts from the dozens to the tens of millions, for which the performance requirements for access or update frequency exceed those of the available SQL database, and the application does not require "no transaction loss" behavior in the event of a crash.

Example Application

Speed Tables is used as a high-speed cache that front-ends a SQL database for a website generating millions of customized page views per day using commodity hardware.

In contrast to ad-hoc tables implemented with some combination of arrays, lists, upvar, namespaces, or even using dicts, Speed tables' memory footprint is far smaller and performance far higher when many rows are present.

Speed tables support tab-separated reading and writing to files and TCP/IP sockets, and has a direct C interface to PostgreSQL. Examples are provided for importing SQL query results into a speed table as well as copying from a speed table to a database table. Speed tables' search function provides a number of powerful capabilities including results sorting, setting offsets and limits, specifying match expressions, and counting.