Improving financial computation speed with full and subproblem memoization

摘要

Analysts prototyping trading strategies often reuse previously computed values: both full problems and subproblems. Avoiding recomputing these would increase productivity. We built a memoization library that caches function computations to files to avoid recomputation. This should minimize the need for users to think about whether caching is appropriate while giving them control over speed, accuracy, and space usage. Guo and Engler built an automatic memoization library by modifying the Python interpreter, while jug and joblib are distributed computing libraries that do memoization. Our library attempts to maintain the ease of use of these libraries while offering a higher degree of control of how caching is carried out. It allows control of space usage for individual functions and all memoization, refreshing memoization for a specific function, and accuracy checking, and uses faster hashing and provides a divide and conquer approach to reuse previously computed subproblems. We show that for Markowitz optimization, Fama–French, and the singular value decomposition, memoization using our library greatly speeds up recomputation, often by over 99% versus no memoization and over 80% versus joblib. We also show how a divide-and-conquer memoization approach can give large speedups for sorting. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.