The researcher's guide to the data deluge: querying a scientific database in just a few seconds

There is a clear need for interactive exploration of extremely large
databases, especially in the area of scientific data management where
ingestion of multiple Terabytes on a daily basis is foreseen. Unfortunately,
current data management technology is not well-suited for
such overwhelming demands.
In light of these challenges, we should rethink some of the strict
requirements database systems adopted in the past. We envision
that next generation database systems should interpret queries by
their intent, rather than as a contract carved in stone for complete
and correct answers. The result set should aid the user in understanding
the database’s content and provide guidance to continue
the data exploration journey. A scientist can stepwise explore
deeper and deeper into the database, and stop when the result content
and quality reaches his satisfaction point. At the same time,
response times should be close to instant such that they allow a
scientist to interact with the system and explore the data in a contextualized way.
Several research directions are carved out to realize this vision.
They range from engineering a novel database kernel where speed
rather than completeness is the first class citizen, up to refusing to
process a costly query in the first place, but providing advice on
how to reformulate it instead, or even providing alternatives the
system believes might be relevant for the exploration patterns observed.