Implement new Solana watcher

[leoluk]

In Wormhole v1, we were able to get away using program accounts as a queue. Unprocessed lockups were added as new program accounts, and would be deleted after they were either executed or stored on Solana.

This was a very nice primitive, since it would allow us to do a cheap getProgramAccounts call (at least with --account-index program-id on the RPC node) to get all pending accounts.

https://github.com/certusone/wormhole/blob/ddf180a5eb467f1d0162f081d09c4abbe6b441e7/bridge/pkg/solana/client.go#L115

Unfortunately, this is not a workable approach for v2: We no longer use Solana for data availability, so the Solana contract won't know when a transfer is complete and therefore, the queue approach is not working anymore – getProgramAccounts is an O(n) operation, with n = number of program accounts, or even n = all accounts on nodes without --account-index.

We discussed a couple of possible approaches:

1. Switch to a slot-based approach, using the getConfirmedBlock API to fetch every single slot, filter for Wormhole transactions client-side and then request all the individual accounts. This is what Solana themselves recommends exchanges to do. Requires the expensive --enable-rpc-transaction-history flag on the RPC node. Requesting a mainnet-beta block is currently ~300ms with ~800KiB of output, which isn't great but also not terrible and in the same ballpark latency-wise as full scans.

2. Do (1), but instead of creating accounts, emit log messages instead. Requires --enable-cpi-and-log-storage in addition to --enable-rpc-transaction-history. This may or may not be cheaper than using accounts, but most importantly, it causes the messages to be part of the transaction - they'll be in getConfirmedBlock output, which avoids having to make extra RPC calls - and can be requested and replayed for any historic slot depending on a node's retention config. Account state isn't currently persisted (see Ability to query historic account state solana-labs/solana#18197).

3. Keep using getProgramAccounts, but limit n by reducing rent, effectively implementing a sliding window. We can request a subset of all accounts within the limit by doing a memcmp on a slot or timestamp prefix. This has the advantage of only relying on account state, which all nodes are guaranteed to hold, requiring only --account-index program-id, but once we're beyond the sliding window it won't be possible to retrieve state at all. What's nice about this option is doing fewer RPC calls operating only on Wormhole state, at the expense of potentially having to re-process a lot of txs within the sliding window.

CC @hendrikhofstadt @Reisen @calebrate

[leoluk]

It appears that (2) cannot be implemented safely - there's no way to determine what program originated a log, and the caller could spoof the Wormhole program.

[leoluk]

We implemented (1)