RPC accounts_pending inconsistent with 15.2

[Srayman]

Description of bug:

In version 14.2 I was using RPC accounts_pending with an array of accounts to identify all pending blocks and then process them for a given wallet. It would consistent return pending blocks in 14.2, but after upgrading to version 15.2 it rarely shows pending blocks. The classic wallet seems to handle processing receive/pending blocks. I was able to switch to using RPC: pending for a specific account and that seems to work more consistently.

Steps to reproduce the issue:

1. Send Nano to an account
2. For the receiving account do the following. a) Unlock wallet with password_enter b) run search_pending c) run account_list for wallet ID d) use account list for accounts_pending
3. After performing the steps above it rarely shows any pending blocks. However if I change to use "pending" instead it works every time.

Describe the results you received:
Results of "search_pending"
{"started":"true"}

Results of account_info with pending=true {"frontier":"143E872BE5E9DB889ED25741EC44FEAC2A49F963EC2352FE1D73C6324B3BCDBA","open_block":"5E18786A572EE6CC09F2E9FCD1279AC45742BF0F0BB2F527276889C79CBE9ACE","representative_block":"143E872BE5E9DB889ED25741EC44FEAC2A49F963EC2352FE1D73C6324B3BCDBA","balance":"3163591742600000000000000000000","modified_timestamp":"1535408272","block_count":"740","account_version":"0","pending":"474681100000000000000000000"}

Results of "accounts_pending"
{"blocks":{"xrb_34prihdxwz3u4ps8qjnn14p7ujyewkoxkwyxm3u665it8rg5rdqw84qrypzk":""}}

Results of "pending"
{"blocks":["A59212765D1F9A4C163D86DFD1B6AA5B7C9BC5E553F7D1DBD23685CADA58C10D"]}

Describe the results you expected:
Expect a list of pending blocks to eventually show up from "accounts_pending"

Additional information you deem important (e.g. issue happens only occasionally):
Happens consistently. But on occasion accounts_pending does show the pending blocks. Could it be related to the new "include_active" option?

Environment:

• Windows 10
• Version 15.2 (classic wallet)

logs

[renesq]

The auto-receive feature has always been quirky from time to time AFAIK, even after using search_pending and search_pending_all. Just keep in mind that if these commands do work correctly, then accounts_pending will not return anything because all the pendings were already received. You also have to keep in mind that the automatic receive is dependent on the config.json settings - tiny amounts will be skipped by default. But I presume you did everything right here. I'm just not sure if you had auto receives activated in the first place (default is to auto-receive incoming transactions of at least 1 µNANO), and if the malfunctioning accounts_pending is related to the leftover pendings from a malfunctioning auto-receive.

When having pendings stuck, the workaround for us was to periodically use wallet_pending and then receive each single pending block that's left. We normally use the send block discovery to credit funds, so a delay in the receive will not cause any harm for our use-case.

The pending you used is the single-account-equivalent to wallet_pending it seems, because both tend to work correctly.

I'm not exactly sure about which node versions are affected, but it has been a problem for months.

[Srayman]

I'm using a script to process pending blocks. I have not had anything "auto-received" yet, have had amounts as large as 2 NANO stay pending until I either unlock the nano_wallet GUI or process them with an RPC call. I have "receive_minimum": "1" in config.json. I'm ok with a script, just curious that the RPC command I was using "accounts_pending" seems not be as consistent as "pending" after upgrading to 15.2.

By "pending" i mean this RPC. https://github.com/nanocurrency/raiblocks/wiki/RPC-protocol#pending. That one seems to work more consistently to return pending blocks instead of https://github.com/nanocurrency/raiblocks/wiki/RPC-protocol#accounts-pending, which often doesn't show anything even if "pending" does.

In 14.2 accounts_pending was working consistently for me and it wasn't until upgrading to 15.2 that I started noticing issues and tried switching to "pending" which appears to be working consistently now.

[argakiig]

Take a look at #1028 though if your not seeing pending like you expect, this patch fixed the pending rpc calls from showing blocks that weren't confirmed yet as well

[Srayman]

Thanks, I saw the new Optional "include_active" setting in both "pending" and "accounts_pending". I had not tried adding that as I'm ok waiting some time for them to be confirmed, however "pending" and "accounts_pending" are not showing the same pending blocks without that setting checked. So either "pending" is not filtering out active blocks like described in that patch or something else is going on.

How long does it take typically for a block to be confirmed and to no longer show as active?

[Srayman]

I think I see the difference in #1028. Looking at the code changes in that issue it only updated accounts_pending and wallet_pending, and not the "pending" call. So does that mean "pending" is looking at active still that haven't been confirmed, but the other two are looking for confirmed pending blocks?

[Srayman]

I added some more details to my original post.

I tried adding include_active=true and that resulted in the block showing up, but after attempting block_confirm and checking confirmation_history I'm not seeing it show up as confirmed.

I'm able to process the receive block for the pending blocks, using either "pending" to get the blocks or "accounts_pending" with include_active=true, however is this the right approach? What's causing pending blocks to not be confirmed?

[argakiig]

So, most likely what you are seeing is that votes are being dropped before reaching your node to confirm the block, the changes to the network going live sept 1 should help alleviate that

[rkeene]

I think the current behavior of filtering out active blocks is the right approach. At this point I believe this is really a documentation and communication issue.

We have taken steps to better identify changes to RPC semantics and communicate them out earlier in releases that do so.

[Srayman]

After the comment from @argakiig I dug into the logs a bit more and found this after a pending block.

[2018-08-29 14:43:18.233250]: Broadcasting confirm req for block F25E5EB9390EADF63085DEDD7EE68966073C8239B33A3825E0B1CBB2AE2A2705 to 8 representatives
[2018-08-29 14:43:18.271250]: Unable to receive, wallet locked

I had my script setup to unlock the wallet at the beginning and then lock at the end and it was locking before the responses would come back. I added a sleep for several seconds in between to provide enough time to receive responses and then it auto-received the block without my script needing to receive it manually.

I still think there is an issue where "accounts_pending" and "wallet_pending" filter out active blocks, but "pending" is not filtering active blocks. Shouldn't "pending" be filtering active also or is the RPC documentation not accurate for that action?

[rkeene]

We will look into why "pending" is not filtering out active -- it is expected that it should. Thank you for the follow up !

[rkeene]

I created issue #1122 to investigate the "pending" RPC call's deviance from the documentation.

I believe this issue can now be resolved.

[Srayman]

Yep! I think I figured out my issue was due to timing with the wallet being locked before receiving confirmation responses. I'll go ahead and close it.