Encoding not supported

[Varunram]

Output From ipfs version --all

go-ipfs version: 0.4.10-
Repo version: 5
System version: amd64/linux
Golang version: go1.8.3

The problem seems to be version independent (as it occurs on other versions too

Type:

Critical - System crash, application panic.

Description:

We host files on a server via ipfs and keep getting this error once every 2/3 days, requiring a restart.. Any help would be greatly appreciated. Thanks!

[Stebalien]

What exactly are you doing when you get this error and what is the exact error you're seeing. Where do you get this error. From the ipfs command? In the logs? Do you see any errors in the logs?

The problem seems to be version independent (as it occurs on other versions too

There's no such thing as a "version independent" bug (just bugs that span multiple versions). Are you getting it on 0.4.13?

[Varunram]

Hey, thought I replied to this earlier, sorry. I tested this with versions 0.4.4 through 0.4.13 and it occurs, so I thought it may be persistent, apologies for not being clear enough. The error in the logs is:

Daemon is ready
17:14:14.899 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
21:19:42.200 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
21:26:40.616 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
21:26:53.752 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
14:50:39.283 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
14:50:41.316 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
19:18:37.833 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
19:18:44.689 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
12:39:18.998 ERROR core/serve: ipfs resolve -r /ipfs/QmNyta8Ht9jHPzDjefAL5hP9e13zbGosk68TXiqV6gii14: merkledag: not found gateway_handler.go:584
20:50:13.453 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584
20:50:13.453 ERROR core/serve: invalid ipfs path: selected encoding not supported gateway_handler.go:584

Hope this helps. Thanks!
P.S. I'm at 0.4.10 again because that's the version we've been in for a while, no specific reason.

[ConorTighe]

Was this error ever resolved? getting a similar error for the following:

cat wordalg.json | ipfs dag put > word-task.hash
Error: selected encoding not supported

[schomatis]

Hey @ConorTighe1995, if you're trying to add data to IPFS (and are expecting a hash) you should use the ipfs add command, e.g., cat wordalg.json | ipfs add -Q > word-task.hash. The ipfs dag command is expecting an already formatted file DAG (which I'm assuming your wordalg.json doesn't contain), hence the "encoding not supported" error.

[ConorTighe]

Ah ok I was following the tutorial on computes.io:
https://docs.computes.com/docs/simple-task

Thanks for the help!

[schomatis]

Oh, thanks for calling our attention to that.

/cc @Mr0grog

[Mr0grog]

Hmmm, well, Computes isn’t our project, but I’m not sure those directions are wrong, either. Pretty much any JSON doc should be a valid IPLD doc and therefore valid input to ipfs dag put (exception: docs with properties named / that do not have values that are CIDs, like {"/": "why hello there, I'm just normal text"}).

I think we’d need to know what is in wordalg.json to get a sense of what’s happening here.

[schomatis]

Thanks for the clarification! My bad @ConorTighe1995, it turns out you can use the dag command with the JSON input, could you provide us the content of wordalg.json please?

@Mr0grog One more question, does the JSON need to have the same attributes as an IPLD node? I mean, simple-task from the example has,

{
  "input": {
    "dataset": 2
  },
  "taskDefinition": {
    "runner": {
      "type": "docker-json-runner",
      "manifest": {
        "*": {
          "image": "computes/fibonacci-sum-split:latest"
        }
      }
    },
    "result": {
      "action": "set",
      "destination": {
        "dataset": { 
          "init": "simple-task-results"
        },
        "path": "split/results"
      }
    }
  },
  "status": {
    "init": "split-task-status"
  }
}

I would expect field names like data, /, Size, etc.

[schomatis]

Sorry, one last question, what do you mean with IPLD doc? Any JSON file with the right format?

[ConorTighe]

This is whats giving me the error, I created a new VM and updated to the newest version of IPFS and computes, Im still getting the Error: selected encoding not supported error

{
 "input": {
   "dataset": "i love skynet"
 },
 "taskDefinition": {
   "runner": {
     "type": "docker-json-runner",
     "metadata": {
       "image": "ctighe/computes-test:latest"
     }
   },
   "result": {
     "action": "set",
     "destination": {
       "dataset": { "/": "$results" },
       "path": "split/results"
     }
   }
 },
 "status": {
   "/": "$status"
 }
}

[schomatis]

Yes, as explained by @Mr0grog, the / field means it's a link, and a valid CID should follow, obviously $results is not a valid CID (I suspect that string and $status should have been replaced somewhere before in the JSON production chain with a CID), when go-ipfs tries to parse that it complains that the bits in that string do not represent a valid encoding: selected encoding not supported.

Replace those strings with any CID, e.g., Qmc5m94Gu7z62RC8waSKkZUrCCBJPyHbkpmGzEePxy2oXJ and the dag command should work.

That being said, these "encoding not supported" cryptic errors definetely need to be worked on, the main error (and what the user should see) is that this is an invalid CID, and only as a sub-error you can say why (selected encoding not supported).

[Stebalien]

@schomatis, @ConorTighe1995: I've filed an issue here ipfs/go-ipld-cbor#36.

Let's revert to the original issue and move all discussion there.

[Mr0grog]

what do you mean with IPLD doc? Any JSON file with the right format?

Oops, I should have said “IPLD node” and “JSON doc” (where a JSON doc is the JSON serialization of an IPLD Node). Bad wording on my part.

does the JSON need to have the same attributes as an IPLD node? …I would expect field names like data, /, Size, etc.

Ah! So when we are talking about “IPLD” (as opposed to, say, UnixFS, which is a very particular subset of IPLD), absolutely any structure at all is acceptable (IPLD is really abstract). As long as we’re talking about JSON input to ipfs dag put (i.e. JSON that is going to be converted to CBOR), that means: absolutely any valid JSON document (except the above mentioned case of properties named / that aren’t parseable as CIDs) is valid input.

---

An aside on IPLD

Note that the above stuff is different from a UnixFS node, which has a much more strict format ({"Links": [...], "Data": <bytes>}). That uses “MerkleDAG,” which you can think of as a stricter subset of what you could represent with CBOR, which is a stricter subset of what qualifies as IPLD.

The thing about IPLD that isn’t very clearly stated anywhere is that, format-wise, it’s really more of an abstract concept than a format. Absolutely any data is considered IPLD as long as it links to other bits of data using a cryptographic hash instead of something else, like a URL. (And if it has no links, then we can assume it just has 0 hash-based links, and it’s valid 😛) That’s why the IPLD Node interface in the Go implementation only has methods and no concrete values — it’s up to the actual struct for a given Node to do things like extracting a set of links or traversing a path to return a value, because a Node could be pretty much any stream of bytes.

Also note that those links do not have to be CIDs, so long as we know enough to up-convert one to a CID. For example, if you know the IPLD node you’re working with is a Git object, you can treat all SHA1s in it as links and up-convert them to CIDs using 0x78 as the codec.

Finally, it’s also good to keep in mind that the whole thing with {"/": "<cid>"} isn’t really anything special in IPLD or about how you’d handle JSON in IPLD. It’s just the way IPFS converts JSON to CBOR (and vice-versa) and the recommended way to represent links in JSON. But you could have any particular flavor of JSON you wanted, with its own IPLD resolver, that formats links differently.

I hope I’m not describing things everybody already knows here (and sorry if I am). It took me quite a lot of time and seemingly naïve questions to figure all the above out. (See also ipld/ipld#39 (comment))

[schomatis]

Agreed, sorry for hijacking this one.

[Stebalien]

@Varunram If you're still there, I'm not sure if that error message is related. That's an error message you'll see if a user asks for a non /ipfs/QmId... path. We should definitely reduce the verbosity.

[schomatis]

I hope I’m not describing things everybody already knows here (and sorry if I am)

You're most definitely not. Help me write a quick and dirty IPLD intro for the docs directory in this repo. Yes, we'll be duplicating work, yes, there's already a specific repo for that, yes, there's even an IPLD specification which has more relevance than a simple document, but it would really really help me (and others I think) to have an anchor to reference instead of repeating the same on information on every other issue/PR. We could structure it like a FAQ.

(Sorry @Stebalien, last comment in this issue, I swear :))

[ConorTighe]

Replace those strings with any CID, e.g., Qmc5m94Gu7z62RC8waSKkZUrCCBJPyHbkpmGzEePxy2oXJ and the dag command should work.

I assume this is where the results will be saved to so should I not do an ipfs add of a results.txt file?

thanks for the help

[Varunram]

Thanks @Stebalien, that makes sense, I shall close this issue now.