Workflow Tip: Using Call Caching on Terra.bio
For public health laboratories running pathogen genomics workflows on Terra.bio, small configuration choices can have a big impact on turnaround time and cloud costs. One setting worth understanding is call caching, which allows Terra’s workflow engine, Cromwell, to detect when a task has already been run and reuse previous results instead of recomputing them.
Call caching is especially helpful when you are troubleshooting a workflow, rerunning a partially failed analysis, or repeating the same workflow with the same inputs after a downstream issue has been resolved. According to Terra Support, call caching lets a workflow restart at the task that failed rather than rerunning every successful upstream task from the beginning.
This can be useful for workflows in the Theiagen Public Health Bioinformatics repository, which includes workflows for genomic characterization, genomic epidemiology, and submission preparation for pathogens of public health concern. When teams are validating inputs, adjusting sample metadata, or recovering from a transient failure, avoiding unnecessary recomputation can keep the focus on interpretation rather than rerunning work that already succeeded.
A practical first step is to leave Use call caching enabled for routine reruns unless you have a specific reason not to. Terra notes that call caching is enabled by default, and the workflow submission form includes runtime options such as cost thresholds, call caching, deleting intermediate outputs, memory retry, and resource monitoring.
There is one important tradeoff: call caching and deleting intermediate outputs are not the same cost-saving strategy. Terra’s documentation explains that Use call caching and Delete intermediate outputs cannot be used together in the same way because caching depends on preserving intermediate files. If a workflow run deletes intermediates, it may read from an existing cache, but it will not write its own results back to the cache for future reuse.
Call caching is also not always appropriate. If you are benchmarking a workflow’s true runtime or cost, cached results can make the workflow look faster or cheaper than it would be from a clean run. It may also be less helpful for workflows that intentionally pull changing data from external sources or when inputs, runtime attributes, or output expressions have changed enough to produce a cache miss.
The takeaway: for routine Terra workflow troubleshooting and repeat analyses, call caching can save time and reduce avoidable compute. For formal benchmarking, workflow validation runs where every task should execute fresh, or storage-cleanup scenarios that delete intermediate outputs, review the setting intentionally before launching the run.
