From computational notebooks to collaborative workflows A decentralized virtual research environment solution

Literate computing environments, such as Jupyter, R Markdown, and Wolfram computational notebooks, have been widely used in scientific studies; they allow users to interactively develop scientific code, test algorithms, and describe the scientific narratives of the experiments in an integrated document. These features empower users not only to conduct scientific experiments but also to document and share their work as cohesive narratives. However, the growing need for distributed data in scientific analyses introduces complex challenges related to distributed data processing, e.g., data access control, privacy preservation, secure sharing, cost, and performance, which often involve the transfer of extensive datasets across diverse infrastructures.
This thesis examines scientific experiments from several ongoing research projects and makes them more manageable by modeling them as workflows, highlighting the pressing issues in data access control, on-demand collaboration, and computing. Based on the detailed analysis of real-world use cases across fields such as digital pathology, experimental biology, and environmental science, and the state-of-the-art literature study, this work proposes a decentralized virtual research environment (VRE) from Jupyter Notebooks to collaborative workflows on decentralized infrastructure. The solution bridges the gap between a Jupyter-enabled private VRE and a decentralized collaborative research ecosystem, leveraging re-configurable workflows on the cloud, workflow scheduling, and decentralized technologies, e.g., blockchains and smart contracts, for managing and operating collaborative scientific workflows.
Future work will focus on extending these approaches to the varied demands of scientific communities and exploring their applicability in decentralized collaborative research and artificial intelligence.