Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent connections to replica on replica restart #57

Merged

Conversation

indiebrain
Copy link

There is a brief period of time during a Redis replica restart when HAProxy can send writes to it; causing the following error:

READONLY You can't write against a read only replica.

This is because HAProxy (< 3.1.0) always treats newly detected backends as immediately UP - ready to serve traffic - UNTIL they failed their checks. HAProxy v3.1.0 adds the ability to configure a backend's initial state - init-state - and how/when it HAProxy determines that the backend is ready to receive traffic.

This changes the HAProxy configuration so that a new Redis node is disqualified from client receiving traffic until HAProxy determines that the node is indeed a "master" node.

This also updates the default operator's HAProxy image accordingly to receive the init-state HAProxy configuration option.

References:

There is a brief period of time during a Redis replica restart when
HAProxy can send writes to it; causing the following error:

```
READONLY You can't write against a read only replica.
```

This is because HAProxy (< 3.1.0) always treats newly detected
backends as immediately UP - ready to serve traffic - UNTIL they
failed their checks. HAProxy v3.1.0 adds the ability to configure a
backend's initial state - init-state - and how/when it HAProxy
determines that the backend is ready to receive traffic.

This changes the HAProxy configuration so that a new Redis node is
disqualified from client receiving traffic until HAProxy determines
that the node is indeed a "master" node.

This also updates the default operator's HAProxy image accordingly to
receive the `init-state` HAProxy configuration option.

References:

- haproxy/haproxy#51
- haproxy/haproxy@50322df
@indiebrain indiebrain self-assigned this Dec 4, 2024
@indiebrain indiebrain requested a review from a team as a code owner December 4, 2024 13:11
@indiebrain indiebrain changed the title Prevent writes to replica on replica restart Prevent connections to replica on replica restart Dec 4, 2024
@indiebrain indiebrain merged commit 83e73dc into master Dec 4, 2024
12 checks passed
@indiebrain indiebrain deleted the 20241202--prevent-sending-writes-to-replicas-on-failover branch December 4, 2024 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants