Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Endpoints GetObjects & GetObjectInfo #579

Merged
merged 9 commits into from
Mar 1, 2022
Merged
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
226 changes: 159 additions & 67 deletions sui/src/rest_server.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright (c) 2022, Mysten Labs, Inc.
// SPDX-License-Identifier: Apache-2.0

use dropshot::{endpoint, PaginationParams, Query, ResultsPage, TypedBody};
use dropshot::{endpoint, Query, TypedBody};
use dropshot::{
ApiDescription, ConfigDropshot, ConfigLogging, ConfigLoggingLevel, HttpError, HttpResponseOk,
HttpResponseUpdatedNoContent, HttpServerStarter, RequestContext,
Expand All @@ -20,8 +20,9 @@ use futures::stream::{futures_unordered::FuturesUnordered, StreamExt as _};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use std::fs;
use std::net::{Ipv6Addr, SocketAddr};
use std::net::{Ipv4Addr, SocketAddr};
use std::path::PathBuf;
use sui_types::object::ObjectRead;
use tokio::task::{self, JoinHandle};
use tracing::{error, info};

Expand All @@ -30,7 +31,7 @@ use std::sync::{Arc, Mutex};
#[tokio::main]
async fn main() -> Result<(), String> {
let config_dropshot: ConfigDropshot = ConfigDropshot {
bind_address: SocketAddr::from((Ipv6Addr::LOCALHOST, 5000)),
bind_address: SocketAddr::from((Ipv4Addr::new(127, 0, 0, 1), 5000)),
..Default::default()
};

Expand Down Expand Up @@ -185,8 +186,9 @@ async fn genesis(
format!("Wallet config was unable to be created: {error}"),
)
})?;
// Need to use a random id because rocksdb locks on current process which means even if the directory is deleted
// the lock will remain causing an IO Error when a restart is attempted.
// Need to use a random id because rocksdb locks on current process which
// means even if the directory is deleted the lock will remain causing an
// IO Error when a restart is attempted.
Comment on lines +189 to +191
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @laura-makdah for visibility of cleanup conditions (relevant to an unrelated effort)

let client_db_path = format!("client_db_{:?}", ObjectID::random());
wallet_config.db_folder_path = working_dir.join(&client_db_path);
*server_context.client_db_path.lock().unwrap() = client_db_path;
Expand Down Expand Up @@ -403,8 +405,7 @@ async fn get_addresses(
// TODO: Find a better way to utilize wallet context here that does not require 'take()'
let wallet_context = server_context.wallet_context.lock().unwrap().take();
let mut wallet_context = wallet_context.ok_or_else(|| {
HttpError::for_client_error(
None,
custom_http_error(
StatusCode::FAILED_DEPENDENCY,
"Wallet Context does not exist.".to_string(),
)
Expand Down Expand Up @@ -451,39 +452,33 @@ async fn get_addresses(
}

/**
Scan parameters used to retrieve objects owned by an address.
Describes the set of querystring parameters that your endpoint
accepts for the first request of the scan.
* 'GetObjectsRequest' represents the request to get objects for an address.
*/
#[derive(Deserialize, Serialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
struct GetObjectsScanParams {
struct GetObjectsRequest {
/** Required; Hex code as string representing the address */
address: String,
}

/**
Page selector used to retrieve the next set of objects owned by an address.
Describes the information your endpoint needs for requests after the first one.
Typically this would include an id of some sort for the last item on the
previous page. The entire PageSelector will be serialized to an opaque string
and included in the ResultsPage. The client is expected to provide this string
as the "page_token" querystring parameter in the subsequent request.
*/
#[derive(Deserialize, Serialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
struct GetObjectsPageSelector {
/** Required; Hex code as string representing the address */
address: String,
struct Object {
/** Hex code as string representing the object id */
object_id: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the restrictions on the types that can appear in a Response--e.g., could this be object_id: ObjectID instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just needs to derive Serialize/Deserialize and JsonSchema. i.e. ObjectID would need to derive from JsonSchema and so would AccountAddress and so on.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha + makes sense. I would be in favor of adding these #derive(JsonSchema)'s everywhere instead of converting our Rust types into strings.

I did not know about this schemars library that provides JsonSchema--it is pretty nifty. Am wondering if we can express our SuiJSON via a schema (not a priority, but could be useful for unifying JSON input/output formats in the future). CC @oxade for thoughts

https://graham.cool/schemars/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be great! I attempted to do this earlier but I started getting into the weeds with PublicKeyBytes/dalek and I thought it was better to punt the idea. But now that SuiAddress & PublicKeyBytes are decoupled this may be easier? I will open an issue for this to either to derive from JsonSchema or a custom SuiJsonSchema

/** Object version */
sequence_number: String,
/** History of signed effects used for local validation of object */
object_digest: String,
}

/**
* 'GetObjectsResponse' is a collection of objects owned by an address.
*/
#[derive(Deserialize, Serialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
struct Object {
/** Hex code as string representing the object id */
object_id: String,
/** Contains the object id, sequence number and object digest */
object_ref: serde_json::Value,
struct GetObjectsResponse {
objects: Vec<Object>,
}

/**
Expand All @@ -497,50 +492,62 @@ Returns list of objects owned by an address.
}]
async fn get_objects(
rqctx: Arc<RequestContext<ServerContext>>,
query: Query<PaginationParams<GetObjectsScanParams, GetObjectsPageSelector>>,
) -> Result<HttpResponseOk<ResultsPage<Object>>, HttpError> {
let pag_params = query.into_inner();
let limit = rqctx.page_limit(&pag_params)?.get();
let tmp;
let (objects, scan_params) = match &pag_params.page {
dropshot::WhichPage::First(scan_params) => {
let object = Object {
object_id: String::new(),
object_ref: json!(""),
};
(vec![object], scan_params)
}
dropshot::WhichPage::Next(page_selector) => {
let object = Object {
object_id: String::new(),
object_ref: json!(""),
};
tmp = GetObjectsScanParams {
address: page_selector.address.clone(),
};
(vec![object], &tmp)
query: Query<GetObjectsRequest>,
) -> Result<HttpResponseOk<GetObjectsResponse>, HttpError> {
let server_context = rqctx.context();

let get_objects_params = query.into_inner();
let address = get_objects_params.address;

let wallet_context = &mut *server_context.wallet_context.lock().unwrap();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@huitseeker I am not sure exactly why I am able to borrow a mutable reference here without the macro complaining. The only difference is that I don't call an async function on client state (which is created from wallet context). Not sure if you see something I don't here.

let wallet_context = wallet_context.as_mut().ok_or_else(|| {
custom_http_error(
StatusCode::FAILED_DEPENDENCY,
"Wallet Context does not exist.".to_string(),
)
})?;

let address = &decode_bytes_hex(address.as_str()).map_err(|error| {
custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("Could not decode address from hex {error}"),
)
})?;

let client_state = match wallet_context.get_or_create_client_state(address) {
Ok(client_state) => client_state,
Err(error) => {
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("Could not get or create client state: {error}"),
));
}
};

Ok(HttpResponseOk(ResultsPage::new(
objects,
scan_params,
|last, scan_params| GetObjectsPageSelector {
address: scan_params.address.clone(),
},
)?))
let object_refs = client_state.object_refs();

Ok(HttpResponseOk(GetObjectsResponse {
objects: object_refs
.map(|(_, (object_id, sequence_number, object_digest))| Object {
object_id: object_id.to_string(),
sequence_number: format!("{:?}", sequence_number),
object_digest: format!("{:?}", object_digest),
})
.collect::<Vec<Object>>(),
}))
}

/**
Request containing the object for which info is to be retrieved.

If owner is specified we look for this obejct in that address's account store,
otherwise we look for it in the shared object store.
*/
#[derive(Deserialize, Serialize, JsonSchema)]
#[serde(rename_all = "camelCase")]
struct GetObjectInfoRequest {
/** Optional; Hex code as string representing the owner's address */
owner: Option<String>,
// TODO: Refactor client state code so that owner can be an optional field.
/** Required; Hex code as string representing the owner's address */
owner: String,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In https://docs.google.com/document/d/1LYTRODgj5GuufocEqKTqzozJJ7MusFOK_s0f8JKoEaw/ (and I believe in the current CLI) the owner is optional. We should probably do the same here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current implementation of the CLI (which the rest server uses) requires the owner field to get a client state. I will add a TODO here to change that. The documentation in PR#544 also reflects the correct behavior.

/** Required; Hex code as string representing the object id */
object_id: String,
}
Expand Down Expand Up @@ -579,16 +586,101 @@ async fn object_info(
rqctx: Arc<RequestContext<ServerContext>>,
query: Query<GetObjectInfoRequest>,
) -> Result<HttpResponseOk<ObjectInfoResponse>, HttpError> {
let object_info_response = ObjectInfoResponse {
owner: String::new(),
version: String::new(),
id: String::new(),
readonly: String::new(),
obj_type: String::new(),
data: json!(""),
let server_context = rqctx.context();
let object_info_params = query.into_inner();

// TODO: Find a better way to utilize wallet context here that does not require 'take()'
let wallet_context = server_context.wallet_context.lock().unwrap().take();
let mut wallet_context = wallet_context.ok_or_else(|| {
custom_http_error(
StatusCode::FAILED_DEPENDENCY,
"Wallet Context does not exist.".to_string(),
)
})?;

let object_id = match ObjectID::try_from(object_info_params.object_id) {
Ok(object_id) => object_id,
Err(error) => {
*server_context.wallet_context.lock().unwrap() = Some(wallet_context);
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("{error}"),
));
}
};

let owner = match decode_bytes_hex(object_info_params.owner.as_str()) {
Ok(owner) => owner,
Err(error) => {
*server_context.wallet_context.lock().unwrap() = Some(wallet_context);
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("Could not decode address from hex {error}"),
));
}
};

// Fetch the object ref
let client_state = match wallet_context.get_or_create_client_state(&owner) {
Ok(client_state) => client_state,
Err(error) => {
*server_context.wallet_context.lock().unwrap() = Some(wallet_context);
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!(
"Could not get client state for account {:?}: {error}",
owner
),
));
}
};

Ok(HttpResponseOk(object_info_response))
let (object, layout) = match client_state.get_object_info(object_id).await {
Ok(ObjectRead::Exists(_, object, layout)) => (object, layout),
Ok(ObjectRead::Deleted(_)) => {
*server_context.wallet_context.lock().unwrap() = Some(wallet_context);
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("Object ({object_id}) was deleted."),
));
}
Ok(ObjectRead::NotExists(_)) => {
*server_context.wallet_context.lock().unwrap() = Some(wallet_context);
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("Object ({object_id}) does not exist."),
));
}
Err(error) => {
*server_context.wallet_context.lock().unwrap() = Some(wallet_context);
return Err(custom_http_error(
StatusCode::FAILED_DEPENDENCY,
format!("Error while getting object info: {:?}", error),
));
}
};

let object_data = object.to_json(&layout).unwrap_or_else(|_| json!(""));

*server_context.wallet_context.lock().unwrap() = Some(wallet_context);

Ok(HttpResponseOk(ObjectInfoResponse {
owner: format!("{:?}", object.owner),
version: format!("{:?}", object.version().value()),
id: format!("{:?}", object.id()),
readonly: format!("{:?}", object.is_read_only()),
obj_type: format!(
"{:?}",
object
.data
.type_()
.map_or("Type Unwrap Failed".to_owned(), |type_| type_
.module
.as_ident_str()
.to_string())
),
data: object_data,
}))
}

/**
Expand Down