Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GraphQL route for dumping entity changes in subgraph and block #3275

Merged
merged 13 commits into from
Mar 2, 2022

Conversation

neysofu
Copy link
Member

@neysofu neysofu commented Feb 24, 2022

Closes #3274.

The GraphQL API implemented here looks like the following (a bit different from #3274's first draft; after collecting feedback):

type Query {
  entityChangesInBlock(subgraphId: String!, blockNumber: Int!): EntityChanges!
}

type EntityChanges {
  updates: [EntityData!]!
  deletes: [String!]!
}

type EntityData {
  name: String!
  fields: [EntityField!]!
}

type EntityField {
  field: String!
  valueAsJSON: String!
}

@neysofu neysofu self-assigned this Feb 24, 2022
Copy link
Collaborator

@lutter lutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Looks good so far!

@neysofu neysofu changed the title (WIP) GraphQL route for dumping entity changes in subgraph and block GraphQL route for dumping entity changes in subgraph and block Feb 28, 2022
This commit changes the GraphQL interface for the `entityChangesInBlock`
route to clearly demarcate entity updates and entity deletions.
Insertions and updates are still *not* discernible, which is less of a
concern. Entity fields and values are returned as JSON strings rather
than scalars for simplicity.
@neysofu neysofu force-pushed the filippo/entities-changed-in-block branch from 4729f6e to eafd61b Compare February 28, 2022 16:22
@neysofu neysofu marked this pull request as ready for review February 28, 2022 16:23
@neysofu neysofu requested review from tilacog and Jannis February 28, 2022 16:23
lutter
lutter previously requested changes Mar 1, 2022
@neysofu neysofu force-pushed the filippo/entities-changed-in-block branch from 2566832 to 4e8c705 Compare March 1, 2022 13:56
Copy link
Contributor

@Jannis Jannis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last request: can we ensure that all returned values are sorted deterministically so we don't have to do it in the client? I see we're doing it for entity fields, but not for entities, entity types, deletions etc.

@neysofu
Copy link
Member Author

neysofu commented Mar 2, 2022

One last request: can we ensure that all returned values are sorted deterministically so we don't have to do it in the client? I see we're doing it for entity fields, but not for entities, entity types, deletions etc.

Yeah, that would be a good idea. Any reasonable order (e.g. ID, alphabetic) as long as it's deterministic?

@Jannis
Copy link
Contributor

Jannis commented Mar 2, 2022

One last request: can we ensure that all returned values are sorted deterministically so we don't have to do it in the client? I see we're doing it for entity fields, but not for entities, entity types, deletions etc.

Yeah, that would be a good idea. Any reasonable order (e.g. ID, alphabetic) as long as it's deterministic?

I'd order type names alphabetically, and then the entities by ID (alphabetically again).

@Jannis
Copy link
Contributor

Jannis commented Mar 2, 2022

@neysofu Another small request: There are a few tables in deployment schemas that are internal, e.g. poi$, poi2$. It would be good to exclude those in the SQL query so we don't leak internals and raw POIs to others.

I think after that we're done here, I've tested the PR and it works great!

@neysofu
Copy link
Member Author

neysofu commented Mar 2, 2022

@Jannis I'm also removing __typename since it's redundant data.

Copy link
Contributor

@Jannis Jannis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!

@neysofu neysofu dismissed lutter’s stale review March 2, 2022 12:34

Addressed all questions and refactored x-checking logic between updates and deletions.

@neysofu neysofu merged commit 1ea75cf into master Mar 2, 2022
@neysofu neysofu deleted the filippo/entities-changed-in-block branch March 2, 2022 12:35
neysofu added a commit that referenced this pull request Mar 8, 2022
* store,server: Add GraphQL route for `entitiesChangedInBlock`

* graph,store,index-node: Separate deletes & updates

This commit changes the GraphQL interface for the `entityChangesInBlock`
route to clearly demarcate entity updates and entity deletions.
Insertions and updates are still *not* discernible, which is less of a
concern. Entity fields and values are returned as JSON strings rather
than scalars for simplicity.

* store: Optimize query filtering on `block_range`

* index-node: Renaming of `EntityData` subfields

* graph,index-node,store: Switch to EntityOperation

* graph,index-node: Add support for fields as JSONObject

* index-node: Aggregate entities with the same type

* store: Improve & document deletions x-checking

* index-node: Remove bad `subgraph` GraphQL route

* index-node: Use object! macro for constructing maps

* index-node: Leave it to object! to do type conversions

* index-node: Deterministic results in entityChangesInBlock

* store: Remove metadata from entity changes results
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GraphQL API for entity changes in a subgraph and block
3 participants