bump version

CHANGELOG.md

@@ -0,0 +1,18 @@
+# Changelog
+
+All notable changes starting with v0.1.33 to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+# Unreleased
+
+# v0.1.35 (2021-12-04)
+
+- **changed:** more nits on the docs
+- **changed:** alphabetized `arloader::commands`
+
+# v0.1.34 (2021-12-04)
+
+- **added:** `command_write_metaplex_items` to write links to json file formatted for use by metaplex candy machine to create NFTs
+- **changed:** moved cli command functions from `main` to separate `commands` module

Cargo.toml

@@ -2,7 +2,7 @@
 name = "arloader"
 authors = ["calebeverett <[email protected]>"]
 description = "Command line application and library for uploading files to Arweave."
-version = "0.1.34"
+version = "0.1.35"
 edition = "2021"
 license = "Apache-2.0"
 repository = "https://github.com/CalebEverett/arloader"

README.md

@@ -7,7 +7,7 @@
 
 Command line application and library for effortlessly uploading files to [Arweave](https://www.arweave.org/). Arweave enables you to store documents and applications forever.
 
-Upload gigabytes of files with one command specifying a glob pattern to match files against. Files are read and posted to [arweave.net](https://arweave.net) asynchronously and computationally intensive bundle preparation is performed in parallel across multiple threads.
+Upload gigabytes of files with one command specifying a glob pattern to match files against. Files are read and posted to [arweave.net](https://arweave.net) asynchronously with computationally intensive bundle preparation performed in parallel across multiple threads.
 
 ## Contents
 * [Installation](#installation)
@@ -21,9 +21,10 @@ Upload gigabytes of files with one command specifying a glob pattern to match fi
 
 1. The easiest way to use arloader is to download the binary for your system (Linux, Mac or Windows) from the [releases on github](https://github.com/CalebEverett/arloader/releases).
 
-You can also install from [crates.io](https://crates.io) once you have [rust installed](https://www.rust-lang.org/tools/install).
+You can also install from [crates.io](https://crates.io) once you have [rust installed](https://www.rust-lang.org/tools/install) with the nightly toolchain.
 
 ```
+rustup default nightly
 cargo install arloader
 ```
 
@@ -37,7 +38,7 @@ cargo install arloader
 
 NFTs consist of an on-chain token, an asset (image, animation, video, or other digital media) and metadata describing the asset. Since on-chain storage is expensive, the token itself typically only includes a link to a metadata file stored off chain that includes a link to the asset stored off chain as well. Arweave is an excellent choice for storing assets and metadata since you only pay once and your files are stored forever. Neither you nor anyone else who might end up with your NFTs ever has to worry about funding storage in the future. Once uploaded to Arweave, your assets and metadata are stored forever!
 
-So, in order to create your NFTs, you need your assets uploaded to Arweave, your metadata files to include links to the assets and finally, the updated metadata files to be uploaded to Arweave. Once these steps are completed and your upload transactions have been confirmed, you can use the links returned from uploading your metadata files to create your NFTs.
+In order to create your NFTs, you need your assets uploaded to Arweave, your metadata files to include links to the assets and finally, the updated metadata files to be uploaded to Arweave. Once these steps are completed and your upload transactions have been confirmed, you can use the links returned from uploading your metadata files to create your NFTs.
 
 1. Upload your assets
 2. Update your metadata files to include the links to your assets
@@ -70,7 +71,7 @@ See [Token Metadata Standard](https://docs.metaplex.com/nft-standard) for detail
 arloader upload "*.png" --log-dir "status/asset/"
 ```
 
-At this point, you can also go ahead and create and upload a manifest file. A manifest is a special file that Arweave will use to access your files by their names relative to the id of the manifest transaction: `https://arweave.net/<MANIFEST_ID>/<FILE_PATH>`. You'll still be able to access your files at `https://arweave.net/<BUNDLE_ITEM_ID>` if you want, but creating and uploading a manifest gives you the option of using either link. You'll also be able to use this file later to automatically update your metadata files to include links to your uploaded asset files. 
+At this point, you can also go ahead and create and upload a manifest file. A manifest is a special file that Arweave will use to access your files by their names relative to the id of the manifest transaction: `https://arweave.net/<MANIFEST_ID>/<FILE_PATH>`. You'll still be able to access your files at `https://arweave.net/<BUNDLE_ITEM_ID>`, but creating and uploading a manifest gives you the option of using either link. You'll also be able to use this file to automatically update your metadata files to include links to your uploaded asset files. 
 
 ```
 arloader upload-manifest --log-dir "status/assets/" --reward-multiplier 2
@@ -141,7 +142,7 @@ Now that your metadata files include links to your uploaded assets, you're ready
 arloader upload "*.json" --log-dir "status/metadata/"
 ```
 
-Go ahead and create and upload a separate manifest for your metadata files. You can then use the links in the `manifest_<TXID>.json` in the `status/metadata/` to create your NFTs, using either of the id or file based links to your metadata.
+Go ahead and create and upload a separate manifest for your metadata files as well.
 
 ```
 arloader upload-manifest --log-dir "status/metadata/"
@@ -164,7 +165,7 @@ arloader get-status <MANIFEST_ID>
 Once each of your transactions has been confirmed at least 25 times, you are good to go - grab the `manifest_<TXID>.json` file in `status/metadata/` and use the included links to create your NFTs!
 
 If you happen to be creating your NFTs with the [Metaplex Candy Machine](https://docs.metaplex.com/create-candy/introduction), you can create a json file of links you can copy
-and paste into your candy machine config by running the command below where `<GLOB>` is a pattern that will match your metdata files (something `*.json`).
+and paste into your candy machine config by running the command below where `<GLOB>` is a pattern that will match your metadata files (something `*.json`).
 
 ```
 arloader write-metaplex-items <GLOB> --manifest-path <MANIFEST_PATH> --log-dir <MANIFEST_PATH>
@@ -187,8 +188,6 @@ you can use the file based link (`https://arweave.net/<MANIFEST_ID>/<FILE_PATH>`
         },
 ```
 
-
-
 ## General Usage
 
 If you're uploading more than one file, you should pretty much always be using bundles. Bundles take multiple files and packages them together in a single transaction. This is better than uploading multiple individual files because you only have to wait for one transaction to be confirmed. Once the bundle transaction is confirmed, all of your files will be available. Larger transactions with larger rewards are more attractive to miners, which means a larger bundled transaction is more likely to get written quickly than a bunch of smaller individual ones.
@@ -303,7 +302,6 @@ where `<LOG_DIR>` is the directory containing your bundle status json files. Thi
         ],
         "id": "Os-tEyRqdjwwyNo1mpLaPGu8_r3KbV-iNRH-aPtJFOw"
     },
-    ...
 ```
 
 You can run the following command to get an update on the status of your manifest transaction.

src/lib.rs

@@ -3,62 +3,63 @@
 //! ## CLI
 //! See [README.md](https://crates.io/crates/arloader) for usage instructions.
 //!
-//! The main cli application is all in `main` a follows a pattern of specifying arguments,
-//! matching them, and then in turn passing them to commands, which are all broken out
-//! separately in the [`commands`] module to facilitate re-use in other command
-//! line applications or as library functions.
+//! The main cli application is all in `main` and follows a pattern of specifying arguments,
+//! matching them, and then in turn passing them to commands, all of which are included
+//! in the [`commands`] module in order to facilitate their use as library functions and
+//! re-use in other command line applications.
 //!
 //! ## Library
 //!
 //! #### Overview
-//! The library is mostly focused on uploading files as efficiently as possible. Arweave has
+//! The library is focused on uploading files as efficiently as possible. Arweave has
 //! two different transaction formats and two different upload formats. Transactions can either
 //! be normal, single data item transactions (see [transaction format](https://docs.arweave.org/developers/server/http-api#transaction-format)
 //!  for details), or bundle transactions (see [bundle format](https://github.com/joshbenaron/arweave-standards/blob/ans104/ans/ANS-104.md)
 //! for details). The bundle format, introduced mid-2021, bundles together individual data items
 //! into larger transactions, making uploading much more efficient and reducing network congestion.
 //! The library supports both formats, with the recommended approach being to use the bundle format.
 //!
-//! There are also two upload formats, whole transactions, which if they are less than 12 MB get uploaded
-//! to the `tx/` endpoint, and chunks, which get uploaded in 256 KB chunks to the  `chunk/`endpoint.
-//! Arloader includes functions for both.
+//! There are also two upload formats, whole transactions, which if they are less than 12 MB can be
+//! uploaded to the `tx/` endpoint, and in chunked transactions, which get uploaded in 256 KB chunks
+//! to the  `chunk/`endpoint. Arloader includes functionality for both formats.
 //!
 //! #### Transactions and DataItems
 //! Both transaction formats start with chunking file data and creating merkle trees from the chunks.
 //! The merkle tree logic can be found in the [`merkle`] module. All of the hashing functions and other crypto
 //! operations are in the [`crypto`] module. Once the data is chunked, hashed, and a merkle root
 //! calculated for it, it gets incorporated into either a [`Transaction`], which can be found in the
-//! [`transaction`] module, or a [`DataItem`] (if it is going to be included in a bundle format transaction),
+//! [`transaction`] module, or if it is going to be included in a bundle format transaction, a [`DataItem`],
 //! which can be found in the [`bundle`] module.
 //!
 //! #### Tags
 //! [`Tag`]s are structs with `name` and `value` properties that can be included with either [`Transaction`]s or
 //! [`DataItem`]s. One subtlety is that for [`Transaction`]s, Arweave expects the content at each key to be a base64 url
 //! encoded string, whereas for [`DataItem`]s, Arweave expects utf8-encoded strings. [`Tag`]s have been implemented for
-//! Arloader includes implementations for both as, [`Tag<Base64>`] and [`Tag<String>`].
+//! for both types as [`Tag<Base64>`] and [`Tag<String>`]. Another subtlety is that [`Tag`]s for [`DataItem`]s are serialized
+//! and deserialized using [`avro_rs`], the schema of which is implemented in [`bundle::get_tags_schema`].
 //!
 //! A [`Tag`] with a name property of `Content-Type` is used by the Arweave gateways to communicate the mime type of
-//! the related content to browsers. Arloader creates the appropriate content type type from a a mime-type database
-//! based on file extension if one is provided, otherwise from magic numbers based on the content bytes.
+//! the related content to browsers. Arloader creates a content type tag based on file extension if one is provided
+//! or from the bytes of the data using [magic numbers](https://en.wikipedia.org/wiki/File_format#Magic_number) if not.
 //!
 //! #### Bytes and Base64Url Data
-//! The library stores all data, signatures and addresses as a [`Base64`] struct with implementations for serialization
-//! and deserialization that automatically convert the underlying bytes to and from the Base64Url format required for
-//! submission to Arweave.
+//! The library stores all data, signatures and addresses as a [`Base64`] struct with methods implemented for
+//! serialize and deserialize the underlying bytes to and from the base64 url format required for uploading
+//! to Arweave.
 //!
 //! #### Signing
 //! A key part of constructing transactions is signing them. Arweave has a specific algorithm for generating the
-//! digest that gets signed and then hashed to serve as a transaction id, called deep hash. It takes various elements
-//! of either the [`Transaction`] or [`DataItem`], including nested arrays of [`Tag`]s, and successively hashes and
-//! concatenates and them together. The required elements are assembled using the [`ToItems`] trait, implemented
-//! separately for [`Transaction::to_deep_hash_item`] and [`DataItem::to_deep_hash_item`]. Arloader's implementation
-//! of the deep hash algorithm can be found in [`crypto::Provider::deep_hash`].
+//! digest that gets signed and then hashed to serve as a transaction id, called [deepHash](https://github.com/ArweaveTeam/arweave-js/blob/92d690a52ed0b647f36d58a6eba28542ef609b18/src/common/lib/deepHash.ts#L8).
+//! It takes various [`Transaction`] or [`DataItem`] elements, including nested arrays of [`Tag`]s, and successively
+//! hashes and concatenates them together. Arloader assembles the required elements via the [`ToItems`] trait, which
+//! is implemented separately as [`Transaction::to_deep_hash_item`] and [`DataItem::to_deep_hash_item`] for each transaction
+//! format. [`crypto::Provider::deep_hash`] is Arloader's implementation of the deep hash algorithm.
 //!
 //! #### Higher Level Functions
-//! The functions for creating [`Transaction`]s, [`DataItem`]s and bundles are all consolidated on the [`Arweave`] struct.
-//! In general, there are lower level functions for creating single items from data, that are then used in successively
-//! higher level functions to create multiple items from collections of file paths, ultimately uploading streams of items
-//! to Arweave.
+//! The functions for creating [`Transaction`]s and bundles of [`DataItem`]s are all consolidated on the [`Arweave`] struct.
+//! In general, there are lower level functions for creating single items from data that are then composed in successively
+//! higher level functions to allow multiple items to be created from collections of file paths and ultimately upload streams
+//! of transactions to Arweave.
 //!
 //! #### Status Tracking
 //! The library includes additional functionality to track and report on transaction statuses. There are two status structs,

src/solana.rs

@@ -1,4 +1,4 @@
-//! Includes functionality for paying for transaction in SOL.
+//! Functionality for funding transactions in SOL.
 
 use crate::error::Error;
 use crate::transaction::{Base64, DeepHashItem};

tests/integration.rs

@@ -38,7 +38,6 @@ async fn airdrop(arweave: &Arweave) -> Result<(), Error> {
         arweave.crypto.wallet_address().unwrap().to_string()
     ))?;
     let resp = reqwest::get(url).await?.text().await?;
-    // Give the node server a chance
     println!("mine resp: {}", resp);
     Ok(())
 }