Incorrect renegotiation in DTLS

[wariua]

mbedtls_ssl_check_ctr_renegotiate() doesn't take care of the special record counter format in DTLS. It simply compares the full eight bytes. So depending on the current epoch value and conf->renego_period value, it may return an incorrect result.

For example,

in_ctr	conf->renego_period	expected	actual return value
00 00 FF FF FF FF FF 01	FF FF FF FF FF FF FF 00	true	false
00 01 00 00 00 00 00 00	00 00 FF FF FF FF FF 00	false	true

It is expected to ignore the first two bytes in conf->renego_period and check only the last six bytes of in_ctr/out_ctr in DTLS.

[simonbutcher]

Thanks for the bug report. We'll look into it.

[ciarmcom]

ARM Internal Ref: IOTSSL-1099

[sg-]

#729 was merged. Should this issue be closed?

[simonbutcher]

Good point. This has been resolved can now be closed.