重复卷积张量流的零梯度?

最后发布: 2017-09-01 09:55:57


问题

当我尝试对相同的输入进行重复卷积时,对于1个以上的重复,渐变将变为零。 这里可能出什么问题?

W = tf.Variable(tf.zeros([3, 3, 1, 1]))
output = input_image # a 4D tensor [batch_size, 16, 16, 1]
for _ in range(4):
    output = tf.nn.conv2d(
        output, 
        W, 
        [1, 2, 2, 1], 
        padding="SAME"
    )

preds = tf.reshape(output, shape=[batch_size])

loss = tf.reduce_mean(preds, labels)

# this gradient zero when num of repetitive layers > 1??
tf_gradient = tf.concat(0, tf.gradients(loss, W))
gradient = session.run(tf_gradient)

print(gradient.reshape(3**2))
#prints [ 0.  0.  0.  0.  0.  0.  0.  0.  0.]
python tensorflow
回答

W使用随机初始化。 W = tf.get_variable(name="W", initializer=tf.glorot_uniform_initializer, shape=[3, 3, 1, 1], dtype=tf.float32)