tensorflow: BeamSearchDecoder not working

I intalled TensorFlow from 2017-05-10 nightly binaries. The TensorFlow version is v1.1.0-rc2-773-g7fa0cf3 1.1.0-rc2. I have CUDA 8.0 and a Tesla K20c with 4.5G memory. My OS is Linux Ubuntu 14.04.

I tried to run the following code to test the latest BeamSearchDecoder:

lstm = rnn.OutputProjectionWrapper(
rnn.LayerNormBasicLSTMCell(n_hidden, dropout_keep_prob=keep_prob), n_classes)
infer_decoder = BeamSearchDecoder(lstm, 
embedding=lambda tokens:tf.nn.embedding_lookup(embedding_matrix, tokens),
start_tokens=start_tokens, end_token=EOS, initial_state=encoder_state, beam_width=5)
decoder_outputs_infer, decoder_state_infer, decoder_seq_infer = dynamic_decode(infer_decoder)

But I got :

Traceback (most recent call last):
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/tensor_util.py", line 458, in make_tensor_proto
    str_values = [compat.as_bytes(x) for x in proto_values]
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/tensor_util.py", line 458, in <listcomp>
    str_values = [compat.as_bytes(x) for x in proto_values]
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/util/compat.py", line 65, in as_bytes
    (bytes_or_text,))
TypeError: Expected binary or unicode string, got None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/anxf/test_beam.py", line 127, in <module>
    decoder_outputs_infer, decoder_state_infer, decoder_seq_infer = dynamic_decode(infer_decoder)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 286, in dynamic_decode
    swap_memory=swap_memory)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2705, in while_loop
    result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2534, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2484, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/contrib/seq2seq/python/ops/decoder.py", line 234, in body
    decoder_finished) = decoder.step(time, inputs, state)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py", line 437, in step
    length_penalty_weight=length_penalty_weight)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py", line 516, in _beam_search_step
    final_shape=[static_batch_size, beam_width])
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/contrib/seq2seq/python/ops/beam_search_decoder.py", line 638, in _tensor_gather_helper
    output = array_ops.reshape(output, final_shape)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2548, in reshape
    name=name)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/op_def_library.py", line 493, in apply_op
    raise err
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/op_def_library.py", line 490, in apply_op
    preferred_dtype=default_dtype)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 714, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/home/anxf/.local/lib/python3.4/site-packages/tensorflow/python/framework/tensor_util.py", line 462, in make_tensor_proto
    "supported type." % (type(values), values))
TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [None, 5]. Consider casting elements to a supported type.

Process finished with exit code 1

I wonder how I can fix this problem? Or does this mean BeamSearchDecoder is still testing and does not work right now?

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 2
Comments: 25 (7 by maintainers)

Commits related to this issue

[tf.contrib.seq2seq] Better docstrings for AttentionWrapper and BeamSearchDecoder. Fixes #9832, #12569. PiperOrigin-RevId: 170553460 — committed to sb2nov/tensorflow by ebrevdo 7 years ago

Most upvoted comments

Great, thanks! A couple of quick follow-up questions:

When using BeamSearchDecoder with dynamic_decode, it seems that we can’t pass in impute_finished=True without obtaining an error – is this expected?
Also, BeamSearchDecoder appears to expect initial_state to have a shape whose first dimension is batch_size*beam_size. However, it is not entirely clear to me how you would form this initial state, say, from the last hidden state of an encoder, which would have a first dimension of batch_size. Would you simply tile the encoded hidden state beam_size number of times? Or perhaps perform some type of interleaving?

Some clarification on this would be much appreciated!

classicCoder16 on May 29, 2017

classicCoder: the BeamSearchDecoder uses tf.nn.top_k to update the beams at each iteration (top_k by default returns sorted values). The final computation uses GatherTree to identify the true indices at each time step. This means that the 0th beam is the one with the highest score; and you should be able to use predicted_ids[:, :, 0] to access it.

On Sun, Jun 4, 2017 at 2:49 PM, classicCoder16 notifications@github.com wrote:

@ebrevdo https://github.com/ebrevdo Thanks for your help thus far! I’ve been looking over the implementation of the tests in beam_search_decoder_test.py and they are quite helpful. However, I have a bit of a fundamental question: when using dynamic_decode with the BeamSearchDecoder, it seems that the final_outputs that is returned has an attribute predicted_ids whose shape is (batch_size, T, beam_width), which are the final sequences found by beam search.

How then do we go from final_outputs.predicted_ids to the actual most probable sequence for each example in the batch, to therefore obtain something of size (batch_size, T)?

Thanks in advance!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/9832#issuecomment-306069454, or mute the thread https://github.com/notifications/unsubscribe-auth/ABtim8viN2KW4FrFKMd1WHS9PJPfU9Gcks5sAyZagaJpZM4NXsUR .

ebrevdo on Jun 5, 2017

I encountered the same issue, and I found out this error is gone when the batch size of the input of BeamSearchDecoder is specified. This error is due to the line 638 of beam_search_decoder.py, output = array_ops.reshape(output, final_shape). Since this final_shape parameter is final_shape=[static_batch_size, beam_width] and static_batch_size = tensor_util.constant_value(batch_size), static_batch_size is remained as None if the shape of the input batch size is not specified. Therefore, reshaping tensor as [None, beam_width] fails.

shuuki4 on May 12, 2017

Added documentation and an example to BeamSearchDecoder, AttentionWrapper.__init__, and AttentionWrapper.zero_state. Should show up in a day or two.

On Fri, Sep 29, 2017 at 1:35 PM, Sumeet Singh notifications@github.com wrote:

This is pretty poor documentation on part of BeamSearchDecoder. Is it really that hard to add one line of documentation (this is as of Tensorflow v1.3)? It is very important to use tf.contrib.seq2seq.tile_batch and not tf.tile

but one wouldn’t know that unless one stumbled upon this page. I used tf.tile about 2 months back in response to a cryptic error about not having created a zero state of size = batch_size*beam_width. And I had been struggling to get my model accuracy to become reasonable. Turns out it was because I was using tf.tile to tile my init-state-model. In my case the init-state is itself a neural-network that is conditioned by a sample-specific context. Therefore the initial-state is different for each input-sequence - not a straight zero-state. Hence it is very important in my case to line-up the init-state with the samples.

BeamSearchDecoder folks - please document this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/9832#issuecomment-333232331, or mute the thread https://github.com/notifications/unsubscribe-auth/ABtim6AlG-lmxyb9FlL-6BFIHaLg0QE8ks5snVSCgaJpZM4NXsUR .

ebrevdo on Sep 29, 2017

I have the same problem with @classicCoder16:

When using BeamSearchDecoder with dynamic_decode, it seems that we can't pass in impute_finished=True without obtaining an error -- is this expected?

Of course, with impute_finished=False, the error is gone and the result seems to be reasonable. I’m using tf 1.2.1 on Mac OS X.

ruanchong on Aug 12, 2017