Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: remote pinning services #369

Merged
merged 29 commits into from
Feb 19, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
63f88a2
Sketch out pinning service page; amend existing pining page
jessicaschilling Jul 14, 2020
c160ecf
First draft of pinning services page
jessicaschilling Jul 14, 2020
9d2a268
Update docs/how-to/pin-files.md
jessicaschilling Jul 23, 2020
a731044
Update copy, placeholder screenshots
jessicaschilling Dec 18, 2020
087c55f
Update docs/how-to/pin-files.md
jessicaschilling Dec 18, 2020
b275852
Update docs/how-to/pin-files.md
jessicaschilling Dec 18, 2020
bcd6df1
Update docs/how-to/pin-files.md
jessicaschilling Dec 18, 2020
5bafed8
Update docs/how-to/pin-files.md
jessicaschilling Dec 18, 2020
472315e
Update docs/how-to/pin-files.md
jessicaschilling Dec 18, 2020
faf1a21
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
acdbde9
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
a5b41fd
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
1fed18c
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
8a34606
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
95a06b5
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
a11d0ba
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
620dd58
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
a8a4253
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
6f18bff
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
21c9db4
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
758e5ef
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
a196852
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
512340e
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
457296c
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
702715e
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
9feee4d
Update docs/how-to/work-with-pinning-services.md
jessicaschilling Dec 18, 2020
42fa8fc
Merge branch 'main' into feat/pinning-services
lidel Feb 17, 2021
cb629c7
refactor: hide GUI, prioritize CLI
lidel Feb 17, 2021
5896c56
style: remove porcelain, tweak style
lidel Feb 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,7 @@ module.exports = {
],
'/how-to/work-with-blocks',
'/how-to/pin-files',
'/how-to/work-with-pinning-services',
[
'https://github.com/ipfs/go-ipfs/blob/master/docs/file-transfer.md',
'Troubleshoot file transfers'
Expand Down
24 changes: 19 additions & 5 deletions docs/how-to/pin-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@ description: Learn how to pin files in IPFS in order to keep your files and othe

# Pin files using IPFS

Pinning is a very important concept in IPFS. IPFS semantics try to make it feel like every single object is local — there is no "retrieve this file for me from a remote server", just `ipfs cat` or `ipfs get`, which act the same way no matter where the actual object is located. While this is nice, sometimes you want to be able to control what you keep around. Pinning is the mechanism that allows you to tell IPFS to always keep a given object local. IPFS has a fairly aggressive caching mechanism that will keep an object local for a short time after you perform any IPFS operation on it, but these objects may get garbage-collected fairly regularly. To prevent that garbage collection, simply pin the hash you care about. Objects added through `ipfs add` are pinned recursively by default.
Pinning is a very important concept in IPFS. IPFS semantics try to make it feel like every single object is local — there is no "retrieve this file for me from a remote server", just `ipfs cat` or `ipfs get`, which act the same way no matter where the actual object is located.

While this is nice, sometimes you want to be able to control what you keep around. **Pinning** is the mechanism that allows you to tell IPFS to always keep a given object somewhere — the default being your local node, though this can be different if you use a third-party remote pinning service (see below for details). IPFS has a fairly aggressive caching mechanism that will keep an object local for a short time after you perform any IPFS operation on it, but these objects may get garbage-collected fairly regularly. To prevent that garbage collection, simply pin the hash you care about. Note that objects added through `ipfs add` are pinned recursively by default.

Let's look at this example to explore pinning to your local IPFS node in a bit more depth:

```
echo "ipfs rocks" > foo
Expand All @@ -19,11 +23,11 @@ ipfs pin ls --type=all

## Three kinds of pins

As you may have noticed, the first `ipfs pin rm` command didn't work — it should have warned you that the given hash was "pinned recursively". There are three types of pins in the IPFS world:
As you may have noticed in the example above, the first `ipfs pin rm` command didn't work — it should have warned you that the given hash was "pinned recursively". What does this mean? There are three types of pins in the IPFS world:

- Direct pins, which pin just a single block, and no others in relation to it
- Recursive pins, which pin a given block and all of its children
- Indirect pins, which are the result of a given block's parent being pinned recursively
- **Direct pins**, which pin just a single block and no others in relation to it
- **Recursive pins**, which pin a given block and all of its children
- **Indirect pins**, which are the result of a given block's parent being pinned recursively

A pinned object cannot be garbage-collected — try this for proof:

Expand All @@ -40,3 +44,13 @@ ipfs pin rm -r <foo hash>
ipfs repo gc
ipfs cat <foo hash>
```

## Local versus remote pinning

All the information above assumes that you're pinning items locally — that is, to your local IPFS node. That's the default behavior for IPFS, but it's also possible to pin your files to a _remote pinning service_. These third-party services offer the opportunity for you to pin files not to your own local node, but to nodes that they operate, meaning that you don't need to worry about your own node's available disk space or uptime when it comes to availability.

While you can use a remote pinning service's own GUI, CLI, or other dev tools to manage IPFS files pinned to their service, you can also work directly with pinning services in IPFS itself — meaning that you don't need to learn a pinning service's unique API or other tooling.

- The [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec/) offers a specification that enables developers to integrate any pinning service that supports the spec, using simple, standardized schemas and fields.
- If you use go-ipfs from the command line, you can add your favorite pinning service(s), pin CIDs under human-readable names, get pin statuses, and more straight from the CLI. [Learn how →](/how-to/work-with-pinning-services/)
- [IPFS Desktop](https://github.com/ipfs-shipyard/ipfs-desktop) and its equivalent in-browser IPFS web interface, the [IPFS Web UI](https://github.com/ipfs-shipyard/ipfs-webui), both support remote pinning services, so you can pin to your favorite pinning service(s) straight from the UI. [Learn how →](/how-to/work-with-pinning-services/)
114 changes: 114 additions & 0 deletions docs/how-to/work-with-pinning-services.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
title: Work with pinning services
description: Learn how to use or create remote pinning services with IPFS, the InterPlanetary File System.
---

# Work with remote pinning services

Depending on how you use IPFS, you might find it helpful to use a **remote pinning service** instead of, or in addition to, pinning files on your local IPFS node. Whether it happens remotely or locally, **pinning** an item in IPFS identifies it as something you always wish to keep available, exempting it from the routine "garbage collection" that IPFS does on infrequently-used items in order to efficiently manage storage space. (For more details on pinning as a concept, [see this guide](/how-to/pin-files).)

If you've got just one local IPFS node, and it's always running, local pinning may be all you need to insure your important items are persisted and never garbage-collected. However, using a remote pinning service — or creating your own! — might be useful to you if:

- Your local node isn't always online, but you need items to be consistently available
- You'd like to keep a persistent backup of your local node's files somewhere else
- You don't have all the disk space you need on your local node
- You run more than one IPFS node, and would like to use one of them as a "personal pinning service" as your preferred location for permanent storage

There are a number of commercial pinning services that make it easy for you to purchase pinning capacity for your important files (examples of these include Pinata, Temporal, Infura, and others). Each of these third-party services has its own unique interface for pinning files and managing those pins; this could include a GUI, an API, CLI commands, or other tooling.

However, you don't need to learn new commands or tools if your pinning service of choice supports the [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec/) specification — because those pinning services are supported within IPFS itself via the command line, [IPFS Desktop](https://github.com/ipfs-shipyard/ipfs-desktop), and the [IPFS Web UI](https://github.com/ipfs-shipyard/ipfs-webui). And, if you're interested in creating your own pinning service for your own personal or shared use, you can directly integrate it with IPFS Desktop/Web UI and the IPFS CLI by using the IPFS Pinning Service API.

As of January 2021, [Pinata](https://pinata.cloud/) supports the IPFS Pinning Service API, with more pinning services on the way!

## Use an existing pinning service

To add and use a remote pinning service directly in IPFS, you'll first need to have an account with that service (you can do this by signing up directly with the service itself). Once you're got an account, follow these steps to add and use it:

### IPFS Desktop or IPFS Web UI

You can add your favorite pinning service(s) to IPFS Desktop/Web UI directly, enabling you to pin and unpin items from the Files screen in the same way as as you do local pins.

#### Adding a new pinning service

To add a new pinning service, open up IPFS Desktop or Web UI, navigate to the “Pinning Services” section of the Settings screen, and click the “Add Service” button:

![The Desktop/Web UI Settings screen, ready for adding a new pinning service](https://user-images.githubusercontent.com/1507828/102558464-b0c07700-408a-11eb-8ae4-cd30e3ce81fd.png)

Then, select your chosen pinning service from the modal that pops up. If the pinning service you'd like to add isn't listed in that modal, don't worry — you can add any remote pinning service that supports the IPFS Pinning Service API by clicking the "Add a custom one" link.

![Desktop/Web UI modal for selecting a pinning service to add](https://user-images.githubusercontent.com/1507828/102558471-b918b200-408a-11eb-9a28-b06d03f99121.png)

In the next screen, you’ll be asked for a few other details:

- A nickname for this service (this can be helpful if, for example, you want to add two accounts from the same service)
- The URL for your service's API endpoint _(note: this field only appears if you've selected a custom pinning service!)_
- Your secret API key (this is the unique token provided to you by the pinning service — check its documentation for more info)
- Whether you want to automatically add all files in your local node’s Files directory to the pinning service, or choose which ones you upload

![Details modals for adding a new pinning service or a new custom pinning service in Desktop/Web UI](https://user-images.githubusercontent.com/1507828/102558910-ca15f300-408b-11eb-9fe3-742186c077c7.png)

After you hit “Save”, you’ll see your new pinning service added to the “Pinning Services” section of your Settings screen.

![Desktop/Web UI Settings screen with a new pinning service added](https://user-images.githubusercontent.com/1507828/102558530-db123480-408a-11eb-9e3b-58bbd59b2880.png)

#### Using a pinning service

Now that you’re set up, you can pin or unpin files to your new pinning service directly from the Files screen. Just right-click any file (or click the “three dots” action icon in the files list) and select “Set pinning”:

![Desktop/Web UI Files screen with the action menu open and "Set pinning" visible](https://user-images.githubusercontent.com/1507828/102558546-e6656000-408a-11eb-97b8-5fdb060602d2.png)

### IPFS CLI

In addition to integrating your favorite pinning service into IPFS Desktop or Web UI, you can also use it directly from the command line using the `ipfs` command.

#### Adding a new pinning service

To add a new pinning service, use the following command:

```bash
ipfs pin remote service add nickname https://api.mypinningservice.com/endpoint myAccessToken
```

- `nickname` is a unique name for this particular instantiation of a pinning service (this can be helpful if, for example, you want to add two accounts from the same service)
- `https://api.mypinningservice.com/endpoint` is the endpoint supplied to you by the pinning service (check its documentation for more info)
- `myAccessToken` is the unique token provided to you by the pinning service (check its documentation for more info)

#### Using a pinning service

Here are a few CLI commands to get you started. In all examples, replace `nickname` with the unique name you gave the pinning service when you added it.

To pin a CID under under a human-readable name:

```bash
ipfs pin remote add --service=nickname --name=war-and-peace.txt bafybeib32tuqzs2wrc52rdt56cz73sqe3qu2deqdudssspnu4gbezmhig4
```

To list successful pins:

```bash
ipfs pin remote ls --service=nickname
```

To list pending pins:

```bash
ipfs pin remote ls --service=nickname --status=queued,pinning,failed
```

For more commands and general help:

```bash
ipfs pin remote --help
```

## Create your own pinning service

As noted above, you aren't limited to adding the remote pinning services listed in the Settings screen of Desktop/Web UI. Any remote pinning service that uses the [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec) can be added as a custom pinning service — which also means that you can create your own! This might be useful in circumstances like:

- Designating one of your own IPFS nodes to be a "personal pinning service" as a preferred location for permanent storage
- Running a private pinning service for your friends or company
- Starting your own commercial pinning service

As noted above, your service must use the [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec) in order to be interoperable with IPFS Desktop/Web UI and the IPFS CLI. You may also wish to read continuing details on how the API is evolving in the [Pinning Service API Spec GitHub repo](https://github.com/ipfs/pinning-services-api-spec), and be part of the discussion on its further development!

If you'd like to make your custom pinning service available to every IPFS user, we welcome your submissions. Once you're ready to open the doors to the public, make a PR against the [IPFS Web UI GitHub repo](https://github.com/ipfs-shipyard/ipfs-webui) in order to add it to the default list of pinning services that are displayed in the Desktop/Web UI Settings screen, and one of the core maintainers will be in touch.