Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Segment Replication] Remove replica commits when remote store is enabled. #8027

Closed
Tracked by #4448
mch2 opened this issue Jun 12, 2023 · 0 comments · Fixed by #8050
Closed
Tracked by #4448

[Segment Replication] Remove replica commits when remote store is enabled. #8027

mch2 opened this issue Jun 12, 2023 · 0 comments · Fixed by #8050
Assignees
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request v2.10.0

Comments

@mch2
Copy link
Member

mch2 commented Jun 12, 2023

Today with node-node replication we perform independent commits on replicas in two instances.

  1. When a replica receives a new commit point from primary so that it can fsync and purge from its local xlog.
  2. Before engine close so that if promoted as primary it can continue and replay from translog from the latest checkpoint before the engine flip.

With remote storage, all segments are durably persisted in the store. So we can skip performing these commits on replica shards. Further, the commits will bump segment generation and segments_n file, which could potentially cause segment conflicts during failover cases.

@anasalkouz anasalkouz added distributed framework enhancement Enhancement or improvement to existing feature or request v2.9.0 'Issues and PRs related to version v2.9.0' labels Jun 12, 2023
@anasalkouz anasalkouz moved this from Todo to In Progress in Segment Replication Jun 29, 2023
@dreamer-89 dreamer-89 added v2.10.0 and removed v2.9.0 'Issues and PRs related to version v2.9.0' labels Jul 13, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in Segment Replication Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request v2.10.0
Projects
Status: Done
4 participants