Tensorboard plugin - Beholder: Request features

[SrMouraSilva]

Request features

• Show only specific variables, not all trainables: filter by code and by UI
• Add option to do not resize the views (or add option to resize proportionally)
• For better visualization, add add post training operations with a variable (reshape, reverse and other transformations). To get a similar result, I have to create a new variable, do a series of operations and update the new variable in each iteration of the learning process (when I update the original variable, I update the new variable)

Example of use

In RBM (and similars), is possible show the Weight values:

http://www.dmi.usherb.ca/~larocheh/publications/icml-2008-discriminative-rbm.pdf

Current result in Beholder:

Image: Above: Weight (visible = 28*28. hidden = 10). Below: self.W_inspect.assign(tf.reverse(tf.transpose(tf.reshape(self.W, [28*10, 28])), [0]))

(Obs: The vector in my implementation of the RBM is a column vector. This is why the matrix is [10, 28 ** 2] and not the opposite)

[nfelt]

Other than the UI-controlled selection, I think this is possible today by using the arrays parameter to Beholder.update() to pass in an explicit list of numpy arrays to visualize. That should make it possible to show specific variables and to do post-filtering transformations.

[SrMouraSilva]

Hello, @nfelt,

I'm currently checking how to do it (somehow more automatic or more elegant). If I succeed, I'll send a pull request.

I hope to post something here soon to discuss :)

[SrMouraSilva]

I hoped that the following method existed:

# W is a tensor with shape [image width * image height * number of images]
# Example:
#  10 images from mnist
#  W.shape == [28 * 28 * 10]

visualizer = Beholder()
visualizer.register(variable=W, image_shape=[28, 28], columns=5, lines=2, name='W_filters')

# In learning
# [...]
visualizer.update(self.session, frame=updated_values[0])

The expected output in Tensorboard Beholder:

visualizer.register will execute post-filtering transformations and add the result in tensorboard

Problems

For a visualization with 40 images (5 columns and 8 lines)

tf.trainable_variables() and b.update(arrays=[NP_ARRAYS])

In these modes, there are really cool options, but the image is displayed "stretched".

Note: If the image width is greater then that the limit, the image shows property

Above: Image not resized 😄 . Below: Image resized (not proportionally) 😞

b.update(frame=NP_ARRAY)

Problem 1. The image may look very small

I miss the button

Problem 2. Isn't possible show more then one image

[nfelt]

I think maybe the most generic solution here would be to replace frame with a frames option that 1) takes a list of NP_ARRAY and displays all of them and 2) only scales them proportionally. That way we kind of have the arrays option when you want them to be treated like raw data (proportion doesn't matter as much) and frames when you want them to be treated like images (proportion does matter)?

We could also have a checkbox or toggle for "show actual size" (vs size-to-fit).

I'm not sure I quite follow how you'd like the post-filtering transformations to work. I'm inclined to think it's probably still simplest if users do those in their own TF code and pass the result to Beholder, rather than trying to extend Beholder to accommodate all the types of transformations that might be wanted.

[SrMouraSilva]

I'm not sure I quite follow how you'd like the post-filtering transformations to work.

I have a W variable with shape [1 x n]. It contains m images with the same size [width, height]:
Then, n = m * width * height.

I would like to visualize in a a x b array of images.

Example:

• m = 4 images
• width = 4
• height = 4
• Grid a x b = 4 x 4
• W = [w_{1,1} w_{1,2} w_{1,3} w_{1,4} w_{2,1} w_{2,2} w_{2,3} w_{2,4} w_{3,1} w_{3,2} w_{3,3} w_{3,4} w_{4,1} w_{4,2} w_{4,3} w_{4,4}].
• w_{i, j}: i-th image. j-th of the pixel of the image

The W transformation result:

W = [
    [w_{1,1} w_{1,2} w_{2,1} w_{2,2}],
    [w_{1,3} w_{1,4} w_{2,3} w_{2,4}],
    [w_{3,1} w_{3,2} w_{4,1} w_{4,2}],
    [w_{3,3} w_{3,4} w_{4,3} w_{4,4}],
]

My implementation: https://github.com/SrMouraSilva/RBM/blob/tensorflow/rbm/train/task/beholder_task.py#L26-L55
method: register(W, image_shape=[width, height], columns=b):. Observe that the number of lines (a) isn't informed, because it is calculated based on the amount of lines needed to display all images given the amount of columns.

---

I think maybe the most generic solution here would be to replace frame with a frames option that 1) takes a list of NP_ARRAY and displays all of them and 2) only scales them proportionally

This would solve the problem. However, interesting beholder features (such as colormap, variance over train steps) would not be available.

---

I'm inclined to think it's probably still simplest if users do those in their own TF code and pass the result to Beholder, rather than trying to extend Beholder to accommodate all the types of transformations that might be wanted.

(Still) I do not have knowledge as visualization in other learning algorithms, however, in RBM (and I believe in some Deep Belief Networks algorithms) is useful.

---

We could also have a checkbox or toggle for "show actual size" (vs size-to-fit).

Checkbox to hide and show specific variables also sounds great.
Also, when change the tab ('beholder' to 'scalar') the last configuration realized in beholder tab could not be lost.

[nfelt]

Closing this issue since we are intending to deprecate the Beholder plugin as described in #3843. Please see that issue for recommendations if you want to keep using the plugin or adopt it yourself.