kymatio: scattering3d returns a tuple instead of tensor
Each implementation has very different expectations of its inputs and outputs:
- 1D: Expects inputs of size
(B, 1, T), outputs(B, P, T/2**J), wherePis the number of scattering coefficients. Note that if the second dimension of the input is not1, it errors. - 2D: Expects inputs of size
(B, C, M, N), outputs(B, C, P, M/2**J, N/2**J). Note that the number of dimensions changes here between input and output (this may be desirable but is not consistent with 1D). Also, from what I understand, the second dimension here is essentially treated as a batch dimension. - 3D: Expects inputs of size
(B, M, N, O), outputs a tuple with elements of size(B, ?, ?, ?). Not sure what’s going on here.
We should think about harmonizing these to some extent.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 24
Commits related to this issue
- REF Scattering3d now returns tensor, not tuple fixes #158 — committed to kymatio/kymatio by deleted user 6 years ago
- ENH `Scattering3D` now returns tensor, not tuple This harmonizes the output of `Scattering3D` with that of `Scattering1D` and `Scattering2D`. Fixes #158. — committed to kymatio/kymatio by deleted user 6 years ago
- REF `Scattering3D` now returns tensor, not tuple (#210) This harmonizes the output of `Scattering3D` with that of `Scattering1D` and `Scattering2D`. Fixes #158. — committed to kymatio/kymatio by lostanlen 6 years ago
Regarding the relationship between ndims of input and ndims of output, it is true that it would be good to be more consistent. At the beginning I wanted to modify the output of
Scattering1D, but upon second thought I think it’s best to modify the input. So that:(B,T)maps to(B,P,T')just like in other modules. Then in the future we can think about implementing something more generic, as @janden and @eickenberg suggested:(T,)->(P,T')(B,T)->(B,P,T')(B,C,T)->(B,C,P,T')and so forth. I can open a PR for that.FWIW “local” measures scattering transform near a single point, much in the same way as the foveal scattering of @AndreuxMath and @beedotkiran So it makes sense that it does not have spatial coordinates Likewise for integral @janden we might consider implementing integral in 1d in the future, as it runs faster than convolution with phi
@gexarcha I think that there are two issues here. One is how to interface local and integral with standard. The other is the output type: tuple vs tensor. The former can be postponed to
v0.2, but the output type issue needs to be resolved before we releasev0.1.0-alpha.