History

Origins

Many elements of Dask that are foundational in the design of Reshapr are explored in this dask_expts notebook (best viewed via nbviewer.org at dask_expts.ipynb). Those elements include:

• Dask’s task graph architecture

• details of its threads, processes, and cluster schedulers

• the role that chunking plays in conjunction with those things

The results of that exploration were applied to one of the research task needs that is a motivator for Reshapr:

• extraction of time series of diatoms biomass from the 2007 to present SalishSeaCast hindcast to be used to nudge the Atlantis ecosystem model (Diatoms Nudging for Atlantis Ecosystem Model)

in this atlantis_nudge_diatoms notebook (best viewed via nbviewer.org at atlantis_nudge_diatoms.ipynb).

Code from that notebook was extracted into a Python module with a minimal command-line interface. That module was the prototype for the reshapr extract sub-command and contributed to the conceptualization of several of the abstractions that are used in Reshapr.