starfish: `write_experiment_json()` doesn't handle non-flat experiment directory structures correctly

Hi all,

So I’ve encountered an issue when running write_experiment_json() in ‘no copy’ mode (see: https://github.com/spacetx/starfish/pull/1053)

The resulting output .json files that starfish produces all assume that image tiles reside in the experiment root directory (see: https://github.com/spacetx/slicedimage/blob/3456f86a10cc9c89996a394cb75a03457a9bf6bc/slicedimage/io.py#L303).

However, our data (and likely many other group’s data) do not live in a flat directory structure. Ours happens to look like:

expt_root_dir
| - Round_1_dir
|    | - FOV_1_dir
|    |   | - channel_1_z_1.tiff
|    |   ...
|    | - FOV_2_dir
|    |   | - channel_1_z_1.tiff
|    |   ...
|    ...
| - Round_2_dir
|   | - FOV_1_dir
|   |    | - channel_1_z_1.tiff
|   |    ...
|   ...
...

Although this issue arises because Starfish can no longer dictate a flat data structure (by forcing copying), copying has to be optional as the overhead of creating a new copy of huge ‘local’ datasets is unreasonable.

From an end-user perspective, having image data organized into subdirectories is also much easier to deal with than having to deal with one folder with thousands of tiffs.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 16 (7 by maintainers)

Commits related to this issue

Most upvoted comments

After checking out starfish to 32ba1f30e2d0485387b5eb57dfa99f0c4f7b12b8 and selectively merging with 131608e901507dabbe1d11597f8b846fddce951b, things look to be working for me now. There was no need for me to change how FOVs are named.

Many thanks @ttung!