You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Whether your code var_update_op = state_ops.assign(var, var_update) change the w1,i of COCOB-Backprop Algorithm?
In your paper Training Deep Networks without Learning Rates
Through Coin Betting
You said that:
Note that the update in line 10 is carefully defined: The algorithm does not use the previous wt,i in
the update. Indeed, this algorithm belongs to the family of the Dual Averaging algorithms, where the
iterate is a function of the average of the past gradients [Nesterov, 2009].
You do so in Algorithm 2 COCOB-Backprop line 10
The text was updated successfully, but these errors were encountered:
Whether your code
var_update_op = state_ops.assign(var, var_update)
change the w1,i of COCOB-Backprop Algorithm?In your paper Training Deep Networks without Learning Rates
Through Coin Betting
You said that:
Note that the update in line 10 is carefully defined: The algorithm does not use the previous wt,i in
the update. Indeed, this algorithm belongs to the family of the Dual Averaging algorithms, where the
iterate is a function of the average of the past gradients [Nesterov, 2009].
You do so in Algorithm 2 COCOB-Backprop line 10
The text was updated successfully, but these errors were encountered: