IPNS very slow (Kubo <0.20) #3860
Comments
|
@nezzard This is generally a known issue, but providing more information is helpful. Are you resolving from your local node? Or are you resolving through the gateway? |
|
@whyrusleeping throw local, but sometimes gateway faster, sometimes local faster |
|
@nezzard when using locally, how many peers do you have connected? ( DHT based ipns isnt as fast as something more centralized, but you can generally cache the results for longer than ipfs caches them. We should take a look at making these caches more configurable, and look into other ipns slowdowns. When you say its 'very slow', what time range exactly are you experiencing? 1-5 seconds? 5-10, 10+ ? |
|
@whyrusleeping Sometimes it's really fast, sometimes i have Ipfs swarm |
Kubuxu
added
topic/ipns
|
Which ipfs version are you running? |
|
@nezzard What tool are you using in your screenshot? I've seen it many times in the forums but I can't find it anywhere. |
|
@kikoncuo it's a tool from cloud service like dropbox |
|
@Kubuxu The last at the time |
|
@nezzard I meant the tool which you took the screenshot from, my bad |
|
@Kubuxu this tool inside the program yandex disk |
|
So let me tell you some tweaks I've made which has helped quite a bit.
|
|
We can probably add flags to the ipfs name resolve api that allow selection (per resolve) of the query size parameter, and also to say "just give me whatever value you have cached". Both of those would be simple enough to implement without actually having to change too much |
|
Another thing we could do it have a command that returns ipns results as they come in, and then when enough comes in to make a decision, says "This is the best one". This way you could start working with the first one you receive, then when the right one comes in, switch to using that |
|
I have some trouble as well with IPNS. I have a linux box and a windows box on the same LAN running ipfs 0.4.9 and I can't resolve IPNS addresses published from the other side, even after several minutes. I have 400 peers connected on one side, 250 on the other. @cpacia your changes are in a branch somewhere ? That looks like a very handy addition for my project. |
|
Answering to myself, the fork is here: https://github.com/OpenBazaar/go-ipfs @whyrusleeping any idea how I can debug this issue ? |
|
@MichaelMure you cant resolve at all? Or its just very slow? |
|
Sometimes it just take times before being able to resolve and once it has been resolved once it works properly. But in this case it didn't resolve at all even after 30 minutes. It might be another issue but without a way to find out what's going on in ipfs, well ... |
|
I think ipns is very bad for use It's load for 15 - 20 seconds |
|
I'm also experiencing massive resolution times with IPNS. Same behavior over here - the first resolution can take multiple minutes, then once a it's loaded, I can refresh the content in under a second. If I leave it for a few minutes, then do another refresh, and that request cycle repeats the same behavior. The "cache" for the resolution only appears to stay warm for a short period of time. |
|
I'm using a CNAME with _dnslink, for what it's worth. Content is at www.ember-cli-deploy-ipfs.com |
|
Ipfs is unusable. I have the daemon running on both of my computers inside a LAN and one "serving" one file (a video) that the other doesn't have. When I try to access that video from the pc that doesn't have the file, using localhost:8080/ipfs/... on my browser, the video is stopping and taking huge amount of times to load. HUGE amount of time in such a way that I can't watch the video. |
You should ask for a refund 👍 |
|
@kesar I mean this out of love. @jbenet (Juan Benet) says that it is going to release us from the backbone but currently ipfs network performance is very weak. |
|
It took me more than a minute to resolve the domain published by my own computer... And it's not the DNS resolution it hangs at resolving the actual IPNS entry. |
|
We're working on fixing some low hanging fruit in the DHT that should alleviate this: libp2p/go-libp2p-kad-dht#88. You can expect this to appear in a release month or so (4.12 or 4.13). We're also working on bypassing the DHT for recently accessed IPNS addresses by using pubsub ( #4047). However, that will likely remain under an experimental flag for a while as our current pubsub implementation is very naive. |
|
Ah! now we are getting somewhere! I will try a multi-part reply, as this is getting long. First: sync vs. async. I claim that sync "never makes sense". Either you are doing a publish to What other scenarios are there? |
|
Part 2: current and future use cases for IPNS. My use-case is this: I want to share keys with dozens or hundreds of others (who are doing compute of some kind, and are publishing results on ipfs). With the current IPNS design, I would need to share millions, up to 100 million keys with them. Why such an obscenely large number? Cause in my current infrastructure, backed by postgres, I have an extremely sparse dataset, of which only a few million or tens-of-millions of entries are non-zero (they follow a Zipfian distribution in size, and connectivity). Each one of my entries is immutable, thus has a unique content hash, but I need to hang time-varying data off of it (probabilities, counts, sensor-readings, etc.) and so I need dozens or hundreds or more processes cooperating to publish the latest readings hanging off of the immutable sign-posts that have been set up. This currently works fine on PostgreSQL and I can hit several K publishes/second (which is slower than I'd like, but whatever). I can point you at the unit tests. With IPNS/IPFS.. ugh. Clearly, having to share millions of keys with hundreds of peers seems ... the opposite of "decentralized". In some ideal world, enhanced IPNS would do this: instead of what it currently does: That way, I could share only one key with peers, and whenever those peers needed to look up the CID associated with some "well-known hash" they can just do that. |
There are plenty of possible scenarios since what you are doing is giving the user an extra piece of information (that the publish completed) that you wouldn't have otherwise. Simple example: If I want to write code in JS but benefit from features or performance of go-ipfs I might have my application spin up a go-ipfs daemon and talk to it over the HTTP API. In this case my application would want to know when it is "safe to close". @linas Your IPNS use case of needing millions of keys seems highly suspicious to me and is not relevant to this issue. If you could post on https://discuss.ipfs.io/ that would be great, the conversation could be continued there. |
!? Give one real example. The history of computer science has been the elimination of sync writes, from the 1970's onward. The invention of interrupts to avoid sync writes of machine status. The invention of caches to avoid sync writes to DRAM. The invention of DMA to avoid sync writes by I/O. The invention of register write-back queues. And that's just hardware. The invention of mutex locks. The invention of software message queues. The invention of publish/subscribe. All of these were driven by the need to eliminate the stalls associated with sync writes. All of these have become very popular precisely because because they avoid sync execution. I'm kind of frustrated: one didn't have to make sync the default. It could have been async by default. In my code, I have to launch and dettach a thread for each IPNS publish, because I can't afford to wait for a 60/90 second timer to pop in the IPFS code. This bug itself has been open 2.5 years, and has accumulated cruft of me-too's. All discussion on all discussion forums have all basically concluded that "IPNS is broken and lets hunker down and wait until it's fixed". And you are trying to tell me that, no actually, there is a simple fix, nearly trivial -- to change the default (an undocumented API flag) , and everything will work for everybody? Holy cow! Change the default! I mean, I know what I wrote here sounds hostile, but I don't know how else to put it. You're trying to tell me that "its a feature not a bug", but you really got to consider that pretty much the rest of the world thinks its a bug ... |
We were missing the documentation for that option, just created a PR to add it ipfs/js-ipfs#2569. Thanks for the heads up!
Regarding the CLI in JS, it should also work with |
Yes it does, I think if you actually read through this issue you will see that the problem you're complaining about is not the problem this issue is about. Additionally, you have already been told how to get around this problem. As you can see someone has already put up a PR for documentation improvement (thanks @vasco-santos). If you have further complaints unrelated to this issue I'd recommend opening a new issue, this one is already pretty long. |
This change doesn't seem unreasonable (i.e. having go-ipfs spin up a background thread instead of asking the client to do so, which is also a viable option). However, it's not necessarily the right move as it does end up being a UX change that ends up with people issuing complaints like:
We end up with the confusion above anyway with IPFS (since if someone is trying to find data published in the DHT for the first time they have to wait for the publish to complete) so perhaps it's worth doing, but it definitely has tradeoffs. |
|
I guess I deleted my comment about the same time you were writing a response, because I was thinking the --offline option might be the intended solution: at least it works on the command line. But errm no:
I want to really publish, just don't want to wait for it. Then I wrote ferristseng/rust-ipfs-api#62 ... and then I wondered if the answer to that is to just post the http request to the daemon and hang up, don't wait for a response if you don't want to? But if ipfs daemon will give up on publishing as soon as the client hangs up, it won't work. And if ipfs daemon is running as a system service (which IMO it always should be, on as many systems as possible), it really had better be able to handle many simultaneous requests. So ipns publishing should be pooled: one thread should probably be able to publish many names simultaneously; and if not, then spin up a few more. (I was thinking of using ipns a lot: perhaps every uniquely-addressable largish category of data that I'm storing should have its own name, because there's no other mechanism to hold a handle to mutable data, is there? CIDs change with every data mutation, no matter how deep you bury the mutation in your data structure. E.g. I'm trying to write a database, so I think each database instance needs its own IPNS name, and each insert or update of the database will result in the need to update the IPNS record. Is that not the right way to use it?) If we make a comparison to DNS publishing: you probably use a web UI at your hosting provider, right? If you want to change the IP address/dnslink/mx record or whatever that your domain points to, everybody knows it can take hours for DNS changes to propagate worldwide, but that does not mean you expect the web UI to keep showing a spinning animation that whole time and refuse to do anything else. |
|
The time that name_publish() takes is also unpredictable. With ipfs 0.7.0 (with option --enable-namesys-pubsub) I'm seeing times from around 30 sec to a couple of minutes, when it succeeds. I'm trying to do it once per minute, in a cron job; so if it takes longer than that, I end up with overlapping processes. When it hangs, the process that is trying to do that gets hung indefinitely. (I thought it was perhaps racy too, but that turned out to be my fault.) The ipfs daemon process needs to deal with all possible problems, and return immediately from this API call. But because it's broken, I need to find a way to put a timeout into my application. This really shouldn't be the application's problem. Perhaps this is one reason orbitdb uses pubsub directly, but not ipns? I don't quite understand how they got around the naming and discovery problem yet. (Yes I read the field manual: https://github.com/orbitdb/field-manual/blob/master/02_Thinking_Peer_to_Peer/01_P2P_vs_Client-Server.md has a clue, that they believe in using one pubsub channel for queries and another for responses, so that queries can be distributed.) But it seems to me that ipns ought to be the ideal way to name a database instance, at least in the one-to-many publishing case like I'm building. It looks like OrbitDB is designed for collaboration, a true distributed system where each peer contributes something; but the one I'm trying to build is simpler, so I don't need the CRDT overhead (I especially don't want to sign each entry separately), but I do want to publish a structured feed, not just individual byte arrays. When there is one authoritative source for a dataset, IPNS makes sense. When there are multiple collaborating peers providing different portions of the data, it could still make sense for each peer to name his own subset of the data with IPNS. I agree with linas on his points. (Well maybe it's unreasonable to expect ipns to scale to millions of records at this time, but eventually, why not?) aschmahmann as to your point about needing a sync api in case you turn your phone off or shutdown the ipfs daemon on your end: I don't really agree that it's a good use case for ipfs. A long-running daemon has to be running somewhere. Either you have an arrangement for pinning elsewhere and a way to find out when the pinning is done, or else you keep your own daemon running and available on the network all the time. I continue to think ipfs should be a system daemon, like a web server, not bundled many times into every application, because what if you multi-task? You don't really want multiple application-specific daemons running, do you? And I know that when I publish a video to d.tube for example, I'm not paying them to store it: I'm providing the storage myself. Therefore I keep my ipfs daemon running 24/7 on my home computer(s). With that architecture in mind, I'm writing my database using the http API, as a short-lived process that will talk to my local daemon, ask it to do things on its behalf, and then exit. The ipfs daemon has the responsibility to maintain the integrity and availability of the data. It should never shut down, and it should keep juggling all the tasks at once, that's it's job. |
|
This is still a problem after 6years of this thread. Publish mostly takes upwards of 5mins. I was hoping there would be a solution from the comments of @ec1oud . I too want to publish the data as a background job instead of having the client to wait. I've tried finding if a solution exists for this and cant seem to find it. There are a lot of different posts raising this issue though. https://discuss.ipfs.tech/t/why-ipns-is-so-slow/2161 I've also updated the LowHigh/HighWater to low values and set peers from Enabling this Routing.AcceleratedDHTClient does help quite a bit |
|
Are you seeing the same lag with the |
|
Hey @DougAnderson444 , |
|
|
Hey, @DougAnderson444 I've seen this but it doesn't really tell you how to enable it. Did this https://github.com/ipfs/kubo/blob/master/docs/config.md#ipnsusepubsub and then I'm assuming I don't have to do https://github.com/ipfs/kubo/blob/master/docs/config.md#pubsub since its depricated. I disabled the timing my publish doesn't seem to change. Same thing from the RPC api.
|
|
Maybe it would be better to use PutValue, GetValue? I checked. It works fast. |
|
Could you share the doc link for this? cant find it here https://docs.ipfs.tech/reference/kubo/rpc/ |
|
I'm talking about dht putvalue and getvalue. Does Kubo have a direct api to dht? |
|
Seems to be depricated https://docs.ipfs.tech/reference/kubo/rpc/#api-v0-dht-put |
|
And what's better than that? |
|
It remains slow |
|
Mind this is a very old issue with penultimate comment from 2023. Publishing to BOTH DHT+PubSub looks (afaik) sensible these days (not 5 minutes, just a few seconds): $ time ipfs name publish /ipfs/cid --ttl 1m
# < 5s
$ time ipfs resolve /ipns/{name}
# <1s If you see slow update propagation it is usually due to default TTL of IPNS record set to 1h. It is no different than TTL of DNS records. It controls how long result is cached before checking for an update. If you need less caching and faster propagation, you can lower it down to 1 minute like I did in example above. DNS TTL and IPNS TTL serve the same function. See https://discuss.ipfs.tech/t/ipns-best-setting-for-fast-update/18103/3?u=lidel for additional guidance for Publishers and Gateway Operators. I'm closing this as it has a lot of outdated stats and design discussions that no longer apply in 2025:
If someone is looking for up-to-date info:
|
|
I see. Thank you for the detailed explanation. |
and others
Hi, is it normal, that ipns loading very slow?
I tried to make something like cms with dynamic content, but ipns to slow, when i load site via ipns, first loading is very slow, if after that i reload page it's load quickly. But if i reload after few minutes, it's again load slow.
The text was updated successfully, but these errors were encountered: