Reading from SFTP (windows)

[adizhol]

Hi,

When running tensorboard (1.12 or 1.14) on logs from an SFTP drive using sftp 2 driver, tensorboard doens't refresh the scalar display, and I see errors like
"[Thread-2 core_plugin.py:165] Unable to get first event timestamp for run ..."

Any idea how can I make it work?

Thanks

[stephanwlee]

Hi @adizhol, would you be able to give us more complete picture into what you are doing? How are you invoking TensorBoard? How are you using sftp too?

[adizhol]

Hi @stephanwlee,

I map the SFTP to a drive letter, say X:, and I start tensorboard with logdir=X:\logs.
It launches, but after a few minutes it stops showing new data.
I assume it's not receiving filesystem events.
I ended up using win-sshfs to map a network drive.

[nfelt]

I'm not very familiar with windows SFTP mapping but it seems possible that the issue is similar to that with rsync, where by default the files are replaced rather than actually appended, so TensorBoard doesn't see the new data: #349

If there's a similar way to make SFTP reuse the same files, that might work.

Alternatively, if win-sshfs works for you instead, it sounds like perhaps the problem is resolved well enough?

[rmothukuru]

@adizhol,
Can you please let us know if @nfelt's comments helped you so that we can work towards closure of this issue. Thanks!