Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ipfs object get discrepency in curl vs cli api #3359

Closed
Kubuxu opened this issue Nov 5, 2016 · 3 comments · Fixed by #10375
Closed

ipfs object get discrepency in curl vs cli api #3359

Kubuxu opened this issue Nov 5, 2016 · 3 comments · Fixed by #10375
Labels
kind/bug A bug in existing code (including security flaws) status/wontfix This will not be addressed

Comments

@Kubuxu
Copy link
Member

Kubuxu commented Nov 5, 2016 

~$ curl -s 'http://127.0.0.1:8080/api/v0/version'
{"Version":"0.4.5-dev","Commit":"4919645","Repo":"4","System":"amd64/linux","Golang":"go1.7.1"}

~$ curl -s 'http://127.0.0.1:8080/api/v0/object/get/QmPVasCTAsJfqjxdFdvMhVhNwoH1eqxmt61kEBeReXLHPk?enc=protobuf' | hexdump -C
00000000  12 2b 0a 22 12 20 b0 ca  26 ac 9a ca 23 6a 24 6b  |.+.". ..&...#j$k|
00000010  5e 11 61 91 57 58 a7 a0  e3 3d 88 60 df 1a e8 79  |^.a.WX...=.`...y|
00000020  fb 8a 23 98 e8 66 12 00  18 ae d4 e0 15 12 2b 0a  |..#..f........+.|
00000030  22 12 20 87 d8 16 4d 08  29 33 ae 77 bb 70 d9 5c  |". ...M.)3.w.p.\|
00000040  11 b9 a7 f4 29 2a 76 7e  9d 02 b3 d3 15 b5 73 4e  |....)*v~......sN|
00000050  9c c8 f7 12 00 18 ae d4  e0 15 12 2b 0a 22 12 20  |...........+.". |
00000060  ae 0d 9e a1 2d 95 a1 05  02 b0 8e e3 ae fa 54 1c  |....-.........T.|
00000070  78 26 1d f8 0f 0f 8f 67  87 f5 d4 84 95 5d 67 30  |x&.....g.....]g0|
00000080  12 00 18 ae d4 e0 15 12  2b 0a 22 12 20 c6 48 28  |........+.". .H(|
00000090  1d 9e 09 fc 8c a2 bc 5f  47 71 4e 15 05 06 92 7e  |......._GqN....~|
000000a0  4d ef 57 bf 4f 14 31 be  a3 3b db 6c 15 12 00 18  |M.W.O.1..;.l....|
000000b0  ae d4 e0 15 12 2b 0a 22  12 20 c2 52 b4 6d f1 ac  |.....+.". .R.m..|
000000c0  33 43 5f e9 97 e6 4c 26  79 53 a2 2f 6d b7 b4 f4  |3C_...L&yS./m...|
000000d0  f3 12 49 32 7f 0e 8b 26  04 58 12 00 18 ae d4 e0  |..I2...&.X......|
000000e0  15 12 2b 0a 22 12 20 6d  9e fe 09 6a 08 22 65 c8  |..+.". m...j."e.|
000000f0  4a 17 52 be fb 13 62 33  65 5a 88 2d b5 43 9f 66  |J.R...b3eZ.-.C.f|
00000100  0d 5a b4 92 77 26 fc 12  00 18 ae d4 e0 15 12 2b  |.Z..w&.........+|
00000110  0a 22 12 20 e5 ec 3e 16  16 56 41 bc a8 88 af b1  |.". ..>..VA.....|
00000120  89 23 b4 6b ce 66 3f 0b  77 c1 5e 9f 1c 45 0f 7e  |.#.k.f?.w.^..E.~|
00000130  79 dc a0 ab 12 00 18 ae  d4 e0 15 12 2b 0a 22 12  |y...........+.".|
00000140  20 16 fb 65 25 dd 26 0c  37 21 dc 82 8f dd 36 8b  | ..e%.&.7!....6.|
00000150  c3 65 d0 44 bd bc aa a4  8d 5f af 3f 43 2a dd 16  |.e.D....._.?C*..|
00000160  66 12 00 18 ae d4 e0 15  12 2b 0a 22 12 20 cc 2e  |f........+.". ..|
00000170  9a 7e ce b5 2f 42 eb 00  49 0a 94 e6 ba 86 05 17  |.~../B..I.......|
00000180  e8 ad 84 83 33 53 f6 fb  44 76 47 bd 2b b7 12 00  |....3S..DvG.+...|
00000190  18 ae d4 e0 15 12 2b 0a  22 12 20 72 d5 4b b4 86  |......+.". r.K..|
000001a0  c4 18 26 f1 1d 49 6a e9  99 23 b4 34 90 81 e7 31  |..&..Ij..#.4...1|
000001b0  4b 8d a5 56 e6 0d f4 c5  92 82 67 12 00 18 ae d4  |K..V......g.....|
000001c0  e0 15 12 2b 0a 22 12 20  6c 18 c3 84 bb 52 6d 91  |...+.". l....Rm.|
000001d0  41 58 e8 2f 44 cd 5e b4  20 a8 94 fd 01 4a 4a 2f  |AX./D.^. ....JJ/|
000001e0  dc da 53 44 f4 2f db bd  12 00 18 ae d4 e0 15 12  |..SD./..........|
000001f0  2b 0a 22 12 20 94 88 72  21 c0 41 f1 0d 3e a1 2a  |+.". ..r!.A..>.*|
00000200  9c 1e 29 e8 80 ea da f6  01 c5 c3 b9 d0 2f ef 09  |..)........../..|
00000210  79 c9 6c e2 82 12 00 18  ae d4 e0 15 12 2b 0a 22  |y.l..........+."|
00000220  12 20 63 96 ef 2e f7 52  41 40 1a d7 95 f4 ed 5e  |. c....RA@.....^|
00000230  c1 96 9d b1 e0 51 6e cd  7b 9b 29 1c 80 2d 23 65  |.....Qn.{.)..-#e|
00000240  1d ac 12 00 18 ae d4 e0  15 12 2b 0a 22 12 20 cb  |..........+.". .|
00000250  d4 58 c3 fa 6c 5c b1 7a  01 cd 2b a2 5c 53 31 fa  |.X..l\.z..+.\S1.|
00000260  50 b4 83 9c f4 2c 2f cf  ac 7b f5 2c 91 e6 90 12  |P....,/..{.,....|
00000270  00 18 ae d4 e0 15 12 2b  0a 22 12 20 16 c0 3a 02  |.......+.". ..:.|
00000280  47 42 1f f4 11 94 5e 80  9b 44 71 8e b7 da 78 19  |GB....^..Dq...x.|
00000290  ea 1e 19 53 8a f2 30 c4  aa 6c 80 4a 12 00 18 ae  |...S..0..l.J....|
000002a0  d4 e0 15 12 2b 0a 22 12  20 ea 3b 2d 58 83 30 7e  |....+.". .;-X.0~|
000002b0  16 d6 62 f0 7b 36 b8 5f  2d 1a 83 14 ab b8 1b 3b  |..b.{6._-......;|
000002c0  30 4b fd e8 e0 86 bb ee  64 12 00 18 b3 b5 a9 01  |0K......d.......|
000002d0  0a 58 08 02 18 80 b0 c9  c7 02 20 80 80 e0 15 20  |.X........ .... |
000002e0  80 80 e0 15 20 80 80 e0  15 20 80 80 e0 15 20 80  |.... .... .... .|
000002f0  80 e0 15 20 80 80 e0 15  20 80 80 e0 15 20 80 80  |... .... .... ..|
00000300  e0 15 20 80 80 e0 15 20  80 80 e0 15 20 80 80 e0  |.. .... .... ...|
00000310  15 20 80 80 e0 15 20 80  80 e0 15 20 80 80 e0 15  |. .... .... ....|
00000320  20 80 80 e0 15 20 80 b0  a9 01                    | .... ....|
0000032a



Yet note the garbage at the end

~$ gostuff/bin/ipfs object get --enc protobuf QmPVasCTAsJfqjxdFdvMhVhNwoH1eqxmt61kEBeReXLHPk | hexdump -C
00000000  12 2b 0a 22 12 20 b0 ca  26 ac 9a ca 23 6a 24 6b  |.+.". ..&...#j$k|
00000010  5e 11 61 91 57 58 a7 a0  e3 3d 88 60 df 1a e8 79  |^.a.WX...=.`...y|
00000020  fb 8a 23 98 e8 66 12 00  18 ae d4 e0 15 12 2b 0a  |..#..f........+.|
00000030  22 12 20 87 d8 16 4d 08  29 33 ae 77 bb 70 d9 5c  |". ...M.)3.w.p.\|
00000040  11 b9 a7 f4 29 2a 76 7e  9d 02 b3 d3 15 b5 73 4e  |....)*v~......sN|
00000050  9c c8 f7 12 00 18 ae d4  e0 15 12 2b 0a 22 12 20  |...........+.". |
00000060  ae 0d 9e a1 2d 95 a1 05  02 b0 8e e3 ae fa 54 1c  |....-.........T.|
00000070  78 26 1d f8 0f 0f 8f 67  87 f5 d4 84 95 5d 67 30  |x&.....g.....]g0|
00000080  12 00 18 ae d4 e0 15 12  2b 0a 22 12 20 c6 48 28  |........+.". .H(|
00000090  1d 9e 09 fc 8c a2 bc 5f  47 71 4e 15 05 06 92 7e  |......._GqN....~|
000000a0  4d ef 57 bf 4f 14 31 be  a3 3b db 6c 15 12 00 18  |M.W.O.1..;.l....|
000000b0  ae d4 e0 15 12 2b 0a 22  12 20 c2 52 b4 6d f1 ac  |.....+.". .R.m..|
000000c0  33 43 5f e9 97 e6 4c 26  79 53 a2 2f 6d b7 b4 f4  |3C_...L&yS./m...|
000000d0  f3 12 49 32 7f 0e 8b 26  04 58 12 00 18 ae d4 e0  |..I2...&.X......|
000000e0  15 12 2b 0a 22 12 20 6d  9e fe 09 6a 08 22 65 c8  |..+.". m...j."e.|
000000f0  4a 17 52 be fb 13 62 33  65 5a 88 2d b5 43 9f 66  |J.R...b3eZ.-.C.f|
00000100  0d 5a b4 92 77 26 fc 12  00 18 ae d4 e0 15 12 2b  |.Z..w&.........+|
00000110  0a 22 12 20 e5 ec 3e 16  16 56 41 bc a8 88 af b1  |.". ..>..VA.....|
00000120  89 23 b4 6b ce 66 3f 0b  77 c1 5e 9f 1c 45 0f 7e  |.#.k.f?.w.^..E.~|
00000130  79 dc a0 ab 12 00 18 ae  d4 e0 15 12 2b 0a 22 12  |y...........+.".|
00000140  20 16 fb 65 25 dd 26 0c  37 21 dc 82 8f dd 36 8b  | ..e%.&.7!....6.|
00000150  c3 65 d0 44 bd bc aa a4  8d 5f af 3f 43 2a dd 16  |.e.D....._.?C*..|
00000160  66 12 00 18 ae d4 e0 15  12 2b 0a 22 12 20 cc 2e  |f........+.". ..|
00000170  9a 7e ce b5 2f 42 eb 00  49 0a 94 e6 ba 86 05 17  |.~../B..I.......|
00000180  e8 ad 84 83 33 53 f6 fb  44 76 47 bd 2b b7 12 00  |....3S..DvG.+...|
00000190  18 ae d4 e0 15 12 2b 0a  22 12 20 72 d5 4b b4 86  |......+.". r.K..|
000001a0  c4 18 26 f1 1d 49 6a e9  99 23 b4 34 90 81 e7 31  |..&..Ij..#.4...1|
000001b0  4b 8d a5 56 e6 0d f4 c5  92 82 67 12 00 18 ae d4  |K..V......g.....|
000001c0  e0 15 12 2b 0a 22 12 20  6c 18 c3 84 bb 52 6d 91  |...+.". l....Rm.|
000001d0  41 58 e8 2f 44 cd 5e b4  20 a8 94 fd 01 4a 4a 2f  |AX./D.^. ....JJ/|
000001e0  dc da 53 44 f4 2f db bd  12 00 18 ae d4 e0 15 12  |..SD./..........|
000001f0  2b 0a 22 12 20 94 88 72  21 c0 41 f1 0d 3e a1 2a  |+.". ..r!.A..>.*|
00000200  9c 1e 29 e8 80 ea da f6  01 c5 c3 b9 d0 2f ef 09  |..)........../..|
00000210  79 c9 6c e2 82 12 00 18  ae d4 e0 15 12 2b 0a 22  |y.l..........+."|
00000220  12 20 63 96 ef 2e f7 52  41 40 1a d7 95 f4 ed 5e  |. c....RA@.....^|
00000230  c1 96 9d b1 e0 51 6e cd  7b 9b 29 1c 80 2d 23 65  |.....Qn.{.)..-#e|
00000240  1d ac 12 00 18 ae d4 e0  15 12 2b 0a 22 12 20 cb  |..........+.". .|
00000250  d4 58 c3 fa 6c 5c b1 7a  01 cd 2b a2 5c 53 31 fa  |.X..l\.z..+.\S1.|
00000260  50 b4 83 9c f4 2c 2f cf  ac 7b f5 2c 91 e6 90 12  |P....,/..{.,....|
00000270  00 18 ae d4 e0 15 12 2b  0a 22 12 20 16 c0 3a 02  |.......+.". ..:.|
00000280  47 42 1f f4 11 94 5e 80  9b 44 71 8e b7 da 78 19  |GB....^..Dq...x.|
00000290  ea 1e 19 53 8a f2 30 c4  aa 6c 80 4a 12 00 18 ae  |...S..0..l.J....|
000002a0  d4 e0 15 12 2b 0a 22 12  20 ea 3b 2d 58 83 30 7e  |....+.". .;-X.0~|
000002b0  16 d6 62 f0 7b 36 b8 5f  2d 1a 83 14 ab b8 1b 3b  |..b.{6._-......;|
000002c0  30 4b fd e8 e0 86 bb ee  64 12 00 18 b3 b5 a9 01  |0K......d.......|
000002d0  0a c0 01 08 02 18 ef bf  bd ef bf bd ef bf bd ef  |................|
000002e0  bf bd 02 20 ef bf bd ef  bf bd ef bf bd 15 20 ef  |... .......... .|
000002f0  bf bd ef bf bd ef bf bd  15 20 ef bf bd ef bf bd  |......... ......|
00000300  ef bf bd 15 20 ef bf bd  ef bf bd ef bf bd 15 20  |.... .......... |
00000310  ef bf bd ef bf bd ef bf  bd 15 20 ef bf bd ef bf  |.......... .....|
00000320  bd ef bf bd 15 20 ef bf  bd ef bf bd ef bf bd 15  |..... ..........|
00000330  20 ef bf bd ef bf bd ef  bf bd 15 20 ef bf bd ef  | .......... ....|
00000340  bf bd ef bf bd 15 20 ef  bf bd ef bf bd ef bf bd  |...... .........|
00000350  15 20 ef bf bd ef bf bd  ef bf bd 15 20 ef bf bd  |. .......... ...|
00000360  ef bf bd ef bf bd 15 20  ef bf bd ef bf bd ef bf  |....... ........|
00000370  bd 15 20 ef bf bd ef bf  bd ef bf bd 15 20 ef bf  |.. .......... ..|
00000380  bd ef bf bd ef bf bd 15  20 ef bf bd ef bf bd ef  |........ .......|
00000390  bf bd 01                                          |...|
00000393

Reported by mib_27dh283 on IRC
@mib-kd743naq
Copy link
Contributor

This one is still present as of 0c413de

@Kubuxu Kubuxu added status/deferred Conscious decision to pause or backlog kind/bug A bug in existing code (including security flaws) labels Dec 5, 2016
@whyrusleeping whyrusleeping added the help wanted Seeking public contribution on this issue label Sep 2, 2017
@Coderlane
Copy link

That ef bf bd is the unicode replacement character. I'd bet something in the CLI is attempting to convert the raw protobuf data to a unicode string.

@Stebalien Stebalien removed status/deferred Conscious decision to pause or backlog help wanted Seeking public contribution on this issue labels May 15, 2019
@Stebalien
Copy link
Member

Stebalien commented May 15, 2019 •
edited
Loading

This is what you get when you encode a binary string into a JSON object. Unfortunately, the commands lib always transfers objects as JSON and then re-encodes on the client.

I have no intention of fixing this. Consider ipfs object get broken and stick with ipfs block get until we switch to a sane API transport.

edit 1: I take that back. I'm just going to break this as it's already broken.
edit 2: Nah, I'm really not going to bother with this

Stebalien added a commit that referenced this issue May 15, 2019
@Stebalien
fix ipfs object get
16dec9a 
Always base64 encode when encoding to JSON. Fixes #3359.

This is a breaking change.

License: MIT
Signed-off-by: Steven Allen <[email protected]>
@Stebalien Stebalien mentioned this issue May 15, 2019
Closed
@Stebalien Stebalien added the status/wontfix This will not be addressed label May 15, 2019
@lidel lidel mentioned this issue Mar 21, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug A bug in existing code (including security flaws) status/wontfix This will not be addressed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

core/commands!: remove deprecated object APIs ipfs/kubo
fix ipfs object get ipfs/kubo
5 participants
@Coderlane @Stebalien @whyrusleeping @Kubuxu @mib-kd743naq