feat(pb, ds): move to edge #1942

MasterPtato · 2025-01-25T02:27:48Z

Changes

cloudflare-workers-and-pages · 2025-01-25T02:27:51Z

Deploying rivet with Cloudflare Pages

Latest commit:	`2c4df8b`
Status:	✅ Deploy successful!
Preview URL:	https://970b6e7c.rivet.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet.pages.dev

View logs

MasterPtato · 2025-01-25T02:28:02Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

greptile-apps

PR Summary

This PR moves pegboard and ds services to the edge infrastructure, introducing new workflow handling capabilities and SQLite database integration.

Added new edge-monolith-workflow-worker in /packages/services/edge/monolith/standalone/workflow-worker/ with concerning error handling that unconditionally bails on worker exit
Empty test table migrations in /packages/services/edge/pegboard/src/workflows/actor/migrations.rs and /packages/services/edge/pegboard/src/workflows/client/migrations.rs need proper schema definitions
Added SQLite database access in ActivityCtx and Connection for per-workflow state storage with workflow-specific database names
Implemented comprehensive client workflow in /packages/services/edge/pegboard/src/workflows/client/mod.rs for handling registration, events, and actor management
New feature flag system in /packages/services/edge/pegboard/Cargo.toml organizes dependencies into tiers (default, workflows, chirp)

_{12 file(s) reviewed, 16 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

greptile-apps · 2025-01-27T22:01:05Z

packages/services/edge/monolith/standalone/workflow-worker/Cargo.toml

+ds.workspace = true
+pegboard.workspace = true


logic: ds and pegboard dependencies should specify features needed for workflow integration, e.g. pegboard = { workspace = true, features = ["workflows"] }

greptile-apps · 2025-01-27T22:01:05Z

packages/services/edge/monolith/standalone/workflow-worker/Cargo.toml

+[dependencies]
+chirp-workflow.workspace = true
+rivet-config.workspace = true
+rivet-health-checks.workspace = true
+rivet-metrics.workspace = true
+rivet-runtime.workspace = true


style: consider adding tokio with appropriate runtime features since this is a worker service that will likely need async functionality

greptile-apps · 2025-01-27T22:01:42Z

packages/services/edge/monolith/standalone/workflow-worker/src/lib.rs

+
+	// Start worker
+	worker.wake_start(config, pools).await?;
+	bail!("worker exited unexpectedly");


logic: unconditional bail! means worker will always exit with error, even for graceful shutdowns - should check exit condition first

greptile-apps · 2025-01-27T22:01:43Z

packages/services/edge/monolith/standalone/workflow-worker/src/lib.rs

+	let reg = ds::registry()?
+		.merge(pegboard::registry()?)?;
+
+	let db = db::DatabaseFdbSqliteNats::from_pools(pools.clone())?;


style: pools is cloned here but original pools object is passed to wake_start - consider reusing the cloned pools

greptile-apps · 2025-01-27T22:01:43Z

packages/services/edge/pegboard/Cargo.toml

+workflows = ["chirp"]
+chirp = ["chirp-workflow", "sqlx", "nix", "server-spec"]


logic: workflows feature depends on chirp but chirp is not listed as a dependency in the feature definition - should be workflows = ["chirp-workflow"] or include both

greptile-apps · 2025-01-27T22:02:15Z

packages/services/edge/pegboard/src/workflows/client/migrations.rs

+	sql_execute!(
+		[ctx, pool]
+		"
+		CREATE TABLE test (


style: 'test' appears to be a placeholder name. Use a descriptive table name that reflects its purpose in production.

greptile-apps · 2025-01-27T22:02:21Z

packages/services/edge/pegboard/src/workflows/mod.rs

+pub mod client;
+pub mod actor;


style: module order should be alphabetical - 'actor' should come before 'client'

greptile-apps · 2025-01-27T22:02:56Z

packages/services/edge/pegboard/src/workflows/client/mod.rs

+	// TODO(RVT-4450): `last_event_idx < $2` and `ON CONFLICT DO NOTHING` is a workaround
+	let inserted_rows = sql_fetch_all!(


logic: The workaround for RVT-4450 using last_event_idx < $2 and ON CONFLICT DO NOTHING could potentially miss events if they arrive out of order. Consider adding additional validation or logging.

greptile-apps · 2025-01-27T22:02:56Z

packages/services/edge/pegboard/src/workflows/client/mod.rs

+	// TODO: Send as a single message
+	for (i, raw_command) in raw_commands.into_iter().enumerate() {
+		let wrapped_command = protocol::CommandWrapper {
+			index: index + i as i64,
+			inner: raw_command,
+		};
+
+		// Forward signal to ws as message
+		ctx.msg(ToWs {
+			client_id,
+			inner: protocol::ToClient::Commands(vec![wrapped_command]),
+		})
+		.send()
+		.await?;
+	}


style: Sending commands individually could cause performance issues with high command volumes. Consider implementing the TODO to batch commands into a single message.

greptile-apps · 2025-01-27T22:02:57Z

packages/services/edge/pegboard/src/workflows/client/mod.rs

+			if matches!(signal.try_into()?, Signal::SIGTERM | Signal::SIGKILL) {
+				let res = ctx


logic: The signal type check should handle conversion errors from try_into() more explicitly to avoid potential panics

cloudflare-workers-and-pages · 2025-01-28T01:43:32Z

Deploying rivet-hub with Cloudflare Pages

Latest commit:	`2c4df8b`
Status:	✅ Deploy successful!
Preview URL:	https://64c45ed0.rivet-hub-7jb.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet-hub-7jb.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-20T22:00:25Z

Deploying rivet-hub with Cloudflare Pages

Latest commit:	`2c4df8b`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-20T22:05:37Z

Deploying rivet-hub with Cloudflare Pages

Latest commit:	`2c4df8b`
Status:	✅ Deploy successful!
Preview URL:	https://f2b5364b.rivet-hub-7jb.pages.dev
Branch Preview URL:	https://graphite-base-2008.rivet-hub-7jb.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-21T05:22:48Z

Deploying rivet-hub with Cloudflare Pages

Latest commit:	`592c28d`
Status:	✅ Deploy successful!
Preview URL:	https://1670d0fe.rivet-hub-7jb.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet-hub-7jb.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-21T05:23:53Z

Deploying rivet with Cloudflare Pages

Latest commit:	`592c28d`
Status:	✅ Deploy successful!
Preview URL:	https://46990fd6.rivet.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-21T20:10:27Z

Deploying rivet with Cloudflare Pages

Latest commit:	`2c4df8b`
Status:	✅ Deploy successful!
Preview URL:	https://ca29bf81.rivet.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet.pages.dev

View logs

This was referenced Jan 25, 2025

chore: restructure server binaries #1941

Open

feat: fdb sqlite workflows driver #1850

Open

fix(workflows): allow op ctx to do all the things #1938

Open

feat(workflows): add loop state #1939

Closed

MasterPtato mentioned this pull request Jan 25, 2025

fix(cluster): create new cluster server wf with stateful loop #1940

Closed

MasterPtato marked this pull request as ready for review January 27, 2025 22:00

MasterPtato requested review from NathanFlurry and removed request for NathanFlurry January 27, 2025 22:00

MasterPtato marked this pull request as draft January 27, 2025 22:00

MasterPtato mentioned this pull request Jan 27, 2025

chore(workflows): move wf gc and metrics publish into worker #1943

Open

greptile-apps bot reviewed Jan 27, 2025

View reviewed changes

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 1023d01 to 9f75ea9 Compare January 27, 2025 22:18

MasterPtato changed the base branch from 01-25-chore_restructure_server_binaries to 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker January 27, 2025 22:18

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from df8b94d to f6f8a7b Compare January 28, 2025 01:43

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 9f75ea9 to 176d520 Compare January 28, 2025 01:43

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch 9 times, most recently from 9c05edc to 0ce8234 Compare January 31, 2025 21:37

NathanFlurry force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 2c4df8b to 592c28d Compare February 20, 2025 02:55

This was referenced Feb 20, 2025

fix(docker): fix corepack not enabled error #2058

Open

feat: add support for JSON5/JSONC config files #2059

Open

chore: add basic load test #2060

Open

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from ebe7baf to 4e622d1 Compare February 20, 2025 22:00

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 592c28d to 2c4df8b Compare February 20, 2025 22:00

MasterPtato mentioned this pull request Feb 20, 2025

fix(workflows): fix activity error backoff, in memory polling #2062

Open

MasterPtato mentioned this pull request Feb 21, 2025

fix(fdb): conflict ranges #2063

Open

NathanFlurry force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from 4e622d1 to ebe7baf Compare February 21, 2025 05:17

NathanFlurry force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 2c4df8b to 592c28d Compare February 21, 2025 05:17

NathanFlurry mentioned this pull request Feb 21, 2025

fix(job-runner): fix build pack in job-runner Dockerfile #2064

Open

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from ebe7baf to 4e622d1 Compare February 21, 2025 20:04

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 592c28d to 2c4df8b Compare February 21, 2025 20:04

MasterPtato mentioned this pull request Feb 21, 2025

fix(ds): ds list #2066

Open

NathanFlurry force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from 4e622d1 to ebe7baf Compare February 22, 2025 00:19

NathanFlurry force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 2c4df8b to 592c28d Compare February 22, 2025 00:19

This was referenced Feb 22, 2025

fix(js-utils): correctly handle esbuild with import-statement not matching node stdlib #2067

Closed

chore: add s3 example #2068

Closed

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from ebe7baf to 4e622d1 Compare February 22, 2025 02:52

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 592c28d to 2c4df8b Compare February 22, 2025 02:52

MasterPtato mentioned this pull request Feb 22, 2025

fix: various bug fixes #2069

Open

NathanFlurry force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from 4e622d1 to ebe7baf Compare February 22, 2025 12:25

NathanFlurry force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 2c4df8b to 592c28d Compare February 22, 2025 12:25

This was referenced Feb 22, 2025

chore(js-utils): upgrade to unenv 2 #2070

Closed

chore(js-utils): switch from unenv to node-stdlib-browser #2071

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pb, ds): move to edge #1942

feat(pb, ds): move to edge #1942

MasterPtato commented Jan 25, 2025

cloudflare-workers-and-pages bot commented Jan 25, 2025 •

edited

Loading

MasterPtato commented Jan 25, 2025 •

edited

Loading

greptile-apps bot left a comment

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

cloudflare-workers-and-pages bot commented Jan 28, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 20, 2025

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

		workflows = ["chirp"]
		chirp = ["chirp-workflow", "sqlx", "nix", "server-spec"]

		// TODO(RVT-4450): `last_event_idx < $2` and `ON CONFLICT DO NOTHING` is a workaround
		let inserted_rows = sql_fetch_all!(

		if matches!(signal.try_into()?, Signal::SIGTERM \| Signal::SIGKILL) {
		let res = ctx

		ds.workspace = true
		pegboard.workspace = true

feat(pb, ds): move to edge #1942

Are you sure you want to change the base?

feat(pb, ds): move to edge #1942

Conversation

MasterPtato commented Jan 25, 2025

Changes

cloudflare-workers-and-pages bot commented Jan 25, 2025 • edited Loading

Deploying rivet with Cloudflare Pages

MasterPtato commented Jan 25, 2025 • edited Loading

How to use the Graphite Merge Queue

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Jan 28, 2025 • edited Loading

Deploying rivet-hub with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 20, 2025

Deploying rivet-hub with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 20, 2025 • edited Loading

Deploying rivet-hub with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying rivet-hub with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying rivet with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying rivet with Cloudflare Pages

cloudflare-workers-and-pages bot commented Jan 25, 2025 •

edited

Loading

MasterPtato commented Jan 25, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Jan 28, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading