tensorflow: tf.image.ssim_multiscale does not work in tf-2.0.0

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution: Linux Ubuntu 16.04
TensorFlow installed from: binary (pip)
TensorFlow version: 2.0.0
Python version: 3.5.2
CUDA version: 10.1
GPU model and memory: GTX 1060, 6GB

# example image batches
video1 = tf.random.uniform(shape=[8, 64, 64, 1], minval=0, maxval=1)
video2 = tf.random.uniform(shape=[8, 64, 64, 1], minval=0, maxval=1)

ssim works fine but when I use the multiscale ssim, I am getting the following error message. What am I doing wrong? How do I fix this?

SSIM

ssim_score = tf.image.ssim(img1=video1, img2=video1, max_val=1.0)
print(ssim_score) # tf.Tensor([1. 1. 1. 1. 1. 1. 1. 1.], shape=(8,), dtype=float32)

MS-SSIM

ms_ssim_score = tf.image.ssim_multiscale(img1=video1, img2=video2, max_val=1.0)

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-9-cc68ceec0921> in <module>
----> 1 ms_ssim_score = tf.image.ssim_multiscale(img1=video1, img2=video2, max_val=1.0)

~/.local/lib/python3.5/site-packages/tensorflow_core/python/ops/image_ops_impl.py in ssim_multiscale(img1, img2, max_val, power_factors, filter_size, filter_sigma, k1, k2)
   3405             filter_sigma=filter_sigma,
   3406             k1=k1,
-> 3407             k2=k2)
   3408         mcs.append(nn_ops.relu(cs))
   3409 

~/.local/lib/python3.5/site-packages/tensorflow_core/python/ops/image_ops_impl.py in _ssim_per_channel(img1, img2, max_val, filter_size, filter_sigma, k1, k2)
   3174               math_ops.greater_equal(shape1[-3:-1], filter_size)),
   3175           [shape1, filter_size],
-> 3176           summarize=8),
   3177       control_flow_ops.Assert(
   3178           math_ops.reduce_all(

~/.local/lib/python3.5/site-packages/tensorflow_core/python/util/tf_should_use.py in wrapped(*args, **kwargs)
    196   """
    197   def wrapped(*args, **kwargs):
--> 198     return _add_should_use_warning(fn(*args, **kwargs))
    199   return tf_decorator.make_decorator(
    200       fn, wrapped, 'should_use_result',

~/.local/lib/python3.5/site-packages/tensorflow_core/python/ops/control_flow_ops.py in Assert(condition, data, summarize, name)
    154           op=None,
    155           message="Expected '%s' to be true. Summarized data: %s" %
--> 156           (condition, "\n".join(data_str)))
    157     return
    158 

InvalidArgumentError: Expected 'tf.Tensor(False, shape=(), dtype=bool)' to be true. Summarized data: 8, 8, 8, 1
11

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 2
Comments: 24 (8 by maintainers)

Commits related to this issue

Issue tensorflow#33840 - Fixed filter_size issue in tf.image.ssim_multiscale — committed to aakash30jan/tensorflow by aakash30jan 3 years ago
Better error using MS-SSIM with incompatible power_factors and filter_size. solves #33840 — committed to dmvieira/tensorflow by dmvieira 2 years ago

Most upvoted comments

The issue appears to be with assertion after spatial-dimension reduction. In _ssim_per_channel , the H and W of images is asserted against filter_size . Whereas in ssim_multiscale, downsampling is performed len(power_factors)-1 times.

Here are two workarounds:

Make sure that filter_size is small enough to calculate ssim values for all the four spatial-scales(excluding first scale) after downsampling within ssim_multiscale. Contrarily, ensure both H and W of your image are big enough such that H/(2**4) and W/(2**4) >= filter_size .
Since downsampling is performed len(power_factors)-1 times, you can also use lesser number of _MSSSIM_WEIGHTS or power_factors than default, which means H/(2**(len(power_factors)-1)) and W/(2**(len(power_factors)-1)) >= filter_size .

field1 = tf.random.uniform(shape=[8, 64, 64, 1], minval=0, maxval=1)
field2 = tf.random.uniform(shape=[8, 64, 64, 1], minval=0, maxval=1) 
#Use smaller filter_size
ms_ssim_score = tf.image.ssim_multiscale(img1=field1, img2=field2, max_val=1.0,
                                         filter_size=4)
#Or use lesser number of power_factors
ms_ssim_score = tf.image.ssim_multiscale(img1=field1, img2=field2, max_val=1.0,
                                         power_factors=(0.0448, 0.2856, 0.3001),
                                         filter_size=11)

Hope this helps 😃

+10

aakash30jan on May 25, 2020

It’s a good code example @aakash30jan ! And a great new feature, but my point was about how is the best Interface for the user. I don’t know if build a new function to suggest filter_size is the best way, but perhaps just show a better error with the description explaining that the maximum filter_size should be the value returned by a code similar to suggest_filter_size, instead of a generic error.

Another option is create another param like auto_change_filter that uses suggest_filter_size or another one is filter_size accepts None and uses suggest_filter_size to determine one, but this last one should be very well documented.

My vote to close this issue is in the first one option that just show a better error and how to use. What do you think?

dmvieira on Aug 30, 2022

Hi! @AravindGanesh and Thanks @aakash30jan . Was able to replicate the issue with TF v2.5 with different numbers of power factors starting with default values as mentioned in official tensorflow document .The program ran successfully only when power_factor was 2 or 3 , Please find the gist here …Thanks!

mohantym on Jul 27, 2021

Could replicate the issue with Tf-nightly == 2.2.0.dev20200316. Please find the gist here. Thanks!

gadagashwini-zz on Mar 16, 2020

+1 The following code reproduces the error:

a = np.random.randn(1, 10, 10, 1)
max_val = np.max(np.reshape(a, [-1]))

l = tf.image.ssim_multiscale(a, a, max_val, filter_size=11)

dhruv13J on Nov 15, 2019

Issue is replicating on colab with TF 2.0. Please see the gist. Thanks!

gadagashwini-zz on Oct 31, 2019