Optimize total_installs for /plugin/index

[manasaV3]

Description

Relates to: #1042

Changes introduced:

• Adds a new index to install-activity table on the plugin_name, and is_total field

• Adds the is_total attribute only to the records of type=TOTAL in data workflow

• In backend, updates the data fetch from batch get to scan of the table.

Notes:

• On deploy of this changes, the activity workflow needs to be executed from start of time.

[richaagarwal]

Approving for now to help get this change in, but I would recommend not passing around an additional variable is_total for instantiating InstallActivity, since it's redundant with an existing variable activity_type that we can utilize. The mix of positional and keyword arguments here is a little confusing as well, ideally we would only mix them if one of the arguments was optional.

[manasaV3]

I would recommend not passing around an additional variable is_total for instantiating InstallActivity, since it's redundant with an existing variable activity_type that we can utilize.

With how the pynamo Model's are initialized, refactoring that to be evaluated in the __init__ method from the granularity attribute will add more complexity to the code. The code would look something like:

  def __init__(
            self, hash_key=None, range_key=None,  _user_instantiated: bool = True, **attributes: Any,
    ) -> None:
        if attributes.get('granularity') == InstallActivityType.TOTAL.name:
            attributes['is_total'] = 'true'

        super(InstallActivity, self).__init__(
            hash_key=hash_key,
            range_key=range_key,
            _user_instantiated=_user_instantiated,
            **attributes
        )

So, I am more in favor of computing it outside and passing it on as an attribute. As an added advantage, we compute just once to determine the value of is_total.

The mix of positional and keyword arguments here is a little confusing as well, ideally we would only mix them if one of the arguments was optional.

I agree, we should avoid mixing keyword and positional arguments and lean towards keyword arguments in such cases. Updating the code to reflect that.

[richaagarwal]

@manasaV3 Sorry I missed this before, but let's make sure to update the logging statements too that still reference BatchGet.