(Reprinted from the 2009 San Diego Software Industry Council annual whitepaper: online at http://www.sdsic.org.)

At Torrey Path, a dominant feature that we see in our industry is the scale of the data involved. Most disciplines in life sciences are highly observational, and there is a lot to observe. Established scanning technologies, such as gene expression microarrays, can easily generate gigabytes of data, and new technologies like next-generation sequencing or functional MRI can produce data on the terabyte scale and beyond. Great new analytics are being developed and applied to making sense of all of the data being produced; some of them locally in San Diego. However, due to the scale and complexity of the data being generated, we continue to see a growing disconnection between the data analyses and the life sciences researchers who are trying to use these data to generate new knowledge. That’s where Torrey Path comes in.

Torrey Path is focused on assembling experimental biological data along with basic, supporting knowledge, and then running biostatistical analyses in bulk across all of the data. Processing just a few experiments this way can take days, so for many analytical techniques it makes economic sense to do it only once at our data center, rather than having our customers do that processing repeatedly and on an ad hoc basis. Other techniques might be more sensitive to end-user input, but even in these cases our customers can benefit from knowing what experiments and what samples might make the most sense to start with. We’ve invested heavily in the processing pipeline, making it so we can add new types of data and new analytical techniques at various points in the process.

The quantity of data we are handling turns out to have not just a numerical effect on the processor times and storage capacities, but an even stronger, qualitative effect on the value chain of analytics and transforming raw data into consumable, sustainable knowledge resources.