Android O Background Scanning

[davidgyoung]

This adds an alternate means of background scanning compatible with Android O that does not rely on long running background services (newly prohibited by Android O). The new technique relies on the JobScheduler which allows an app that isn't in the foreground to schedule periodic activities.

The goals of this design are as follows:

1. Retain backward compatibility with existing public APIs of this library.
2. Detect beacons in the foreground and background with Android versions 4.3.x, 4.4.x, 5-7 and Android O (API 18-26). This is a challenge since API 26 does not allow long-running background services and JobScheduler was not introduced until API 21.

This change retains the legacy use of a long-running BeaconService on Android 7 and earlier, but instead uses a ScanJob triggered by the JobScheduler on devices with Android O. The ScanJob may also be manually configured on Android 5+, but it is not made the default on these devices because it does not allow detecting beacons with low-power scans when the ScanJob is not running without new scanning APIs introduced in Android O.

This change works as follows when ScanJobs are enabled on Android O:

1. In the foreground or when the betweenScanPeriod is 0, a ScanJob will be scheduled immediately, and run for 5 minutes, at which time it will end and a new one will immediately begin. The ScanJob uses a CycledLeScanner in much the same way that the BeaconService does.

2. In the background, or when the betweenScanPeriod is nonzero, a ScanJob will be scheduled to periodically run every scanPeriod+betweenScanPeriod milliseconds. It will run for the scanPeriod then stop.

3. Due to OS limits, periodic jobs may only be scheduled at a maximum frequency of 15 minutes, meaning the default backgroundBetweenScanPeriod of 30000ms (5 minutes) will not be achieved. The consequence of this is minor due the following point:

4. When the betweenScanPeriod is nonzero and a ScanJob completes, a low-power filtered scan will be started with the results to be delivered via Intent using new APIs available in Android O. These Intents, if received, will trigger the library's StartupBroadcastReceiver which will then schedule an immediate ScanJob to process the beacon detections. In testing on a Nexus Player, this ensures that a newly detected beacon causes a didEnterRegion callback within seconds of being received, even if the ScanJob is not running.

If the ScanJob is enabled on pre-Android O devices, points 1-3 apply, but point 4 does not, as these APIs are unavailable.

The main difference in the library behavior on Android O is as follows:

1. In the background, full high-power scans will not be scheduled more than once every 15 minutes, regardless of what the betweenScanPeriod is set to.

2. These high-power scans will not run exactly every 15 minutes. On pre-Android O devices using the BeaconService, timers are used to schedule the high-power scans at nearly exact intervals of 310000 ms using default background scan settings. But because the JobScheduler allows
   In overnight tests, the variation in the actual time the job was scheduled ranged from 10.2 to 25.7 minutes, with an average of 15.5 minutes. If for some reason a device running Android O fails to detect a beacon with a low-powered filtered scan (e.g. all the bluetooth hardware filters are used) then it may take up to 25 minutes to detect the beacon based on these tests.

3. If beacons have been detected recently with the app in the background, no low-powered filtered scans will be performed, because they will immediately yield results. Instead, periodic high-powered scans are used. This is the same as when using a long-running BeaconService. But because these high powered scans will be less frequent and less regular, it will typically take longer on Android O devices for background didExitRegion events to fire, or for didRangeBeaconsInRegion events to fire when the app is in the background. These will come only after a period of 10-25 minutes.

[davidgyoung]

Updates include:

• Updated strategy of JobScheduler usage that starts a CycledScanner for a variable period of time for each ScanJob:
  • For 5 minutes per job if in the foreground, recurring every 5 minutes (effectively scanning forever with scan cycles like in Android 4.x-7.x)
  • For the background scan period if in the background, recurring every background scan period + background between scan period
  • Starting immediately if in the foreground once ranging or monitoring is started
• Preliminary background scanning on Android O, using a filtered scan that sends an Intent to restart the app if a beacon is detected when it is no longer running.

The new JobScheduler strategy is much more stable and less resource intensive than the previous strategy.

This resolves the first two open issues regarding failure to detect in the background between scan cycles on Android O, and the serialization being to heavy-weight for rapid scan cycles.

[cupakromer]

This is only a minor review. I'm still trying to parse through the major changes with the JobScheduler but won't be able to do that for a bit.

[cupakromer]

Copy paste error duplicating these blocks?

[davidgyoung]

Yep, other one removed

[cupakromer]

Any particular reason for the this. here? Just asking as it's use has been mixed but it seems the majority of cases don't included it.

[davidgyoung]

since we are prefixing members with m per Android style, I agree that this adds nothing.

[cupakromer]

What happens if someone enables this after the manager has already been started?

[davidgyoung]

I can throw an IllegalStateException if the BeaconService has already been started

[cupakromer]

Since there are four places related to beacon updates where you needed to make this change, maybe a single internal helper would be good to have to centralize this logic?

Maybe something like:

@TargetApi(18)
private void beaconServiceUpdate(int type, @NonNull Region region) throws RemoteException {
    if (mScheduledScanJobsEnabled) {
        ScanJob.applySettingsToScheduledJob(mContext, this);
        return;
    }
    if (serviceMessenger == null) {
        throw new RemoteException("The BeaconManager is not bound to the service.  Call beaconManager.bind(BeaconConsumer consumer) and wait for a callback to onBeaconServiceConnect()");
    }
    Message msg = Message.obtain(null, type, 0, 0);
    msg.setData(new StartRMData(region, callbackPackageName(), getScanPeriod(), getBetweenScanPeriod(), mBackgroundMode).toBundle());
    serviceMessenger.send(msg);
}

[cupakromer]

Thoughts on resolveInfo.isEmpty() instead of resolveInfo.size() == 0? isEmpty feels a bit more expressive of the intent.

[cupakromer]

While I haven't had a chance to get ahold of a Android O preview device to test directly, I was curious why the original bound service would be affected. The following is from the Android O behavior changes:

Bound services are not affected

These rules do not affect bound services in any way. If your app defines a bound service, other components can bind to that service whether or not your app is in the foreground.

-- https://developer.android.com/preview/features/background.html#services

From what I can tell it seems that only if the app / service is initially started in the background (i.e. a device reboot or power connection event) would there be a potential issue. I saw the report from #504 which seems to indicate an issue, but I didn't see anything in the actual documentation which states this should be the case.

Is there something I'm missing with regards to this?

[davidgyoung]

Android O plans to kill long running background services and also prevent them from starting up on phone boot using android.intent.action.BOOT_COMPLETED. The ability to start beacon detection on boot is a library feature that will be broken on Android O by these changes, but can be retained by using the JobScheduler.

I'm not 100% sure what the "Bound services are not affected" clause means. My guess is that it means foreground apps can use bound services provided by background apps, and if they do, the background service is allowed to run indefinitely. My guess is also that background apps using their own bound services will be subject to the same restrictions of non-bound services. If this interpretation is true, this does not apply to this library because the service is not exported for other apps to use -- it is only used internally. Testing will ultimately show which is true. Regardless, the JobScheduler approach is needed for detection at boot.

[herow-io]

1. Have you think about using the https://github.com/evernote/android-job library to unify the ScanJob over the different android version? (Just an idea, I never test it)

2. I have got the impression that the minimum time interval that is scheduled is 5 minutes, while from Android 7+, the minimum time interval accepted by the JobScheduler is 15 minutes.

Here the logs that are generated on Android 7+
JobInfo: Specified interval for 42 is +5m0s0ms. Clamped to +15m0s0ms -> And In fact the jobService is called each 15 minutes

[davidgyoung]

My understanding of https://github.com/evernote/android-job is that it provides a shim for using the JobScheduler on Android versions that do not support (pre API 21). My plan on using the JobScheduler is to do so only and Android O and above.

The reason is that prior to Android O, there is no way to get callbacks for BLE detections if your app is not running. This means a service must be continually running to scan for and detect beacons in low power mode, effectively making it not possible to use a JobScheduler as intended. It means the long-running BeaconService will continue to be used for devices running OS versions earlier than Android O.

Yes, you are correct that the minimum time for the JobScheduler is now 15 minutes. What this means for Android O is that:

• A full low latency scan can run at most once every 15 minutes. (The library's default backgroundBetweenScanPeriod of 5 minutes will therefore be downgraded to 15 minutes on Android O.)

• Low-power scans will still be allowed continually between these low-latency scans, and results delivered via Intent using new APIs in Android O. This is a small addition has been added to a second branch including only those Android O API changes: https://github.com/AltBeacon/android-beacon-library/tree/scheduled-job-scanning-android-o This exists as a second branch to ease testing on pre-Android O devices using the branch for this pull request.

[davidgyoung]

This PR has been rolled into a beta release below so that @Ch3D and others may use it for testing apps targeting Android O.

https://github.com/AltBeacon/android-beacon-library/releases/tag/2.12-beta1

See updated release beta 2 below.

[davidgyoung]

New beta2 release fixes crashes when ranging callbacks are made:

https://github.com/AltBeacon/android-beacon-library/tree/2.12-beta2

See updated beta 3 release below.

[Ch3D]

@davidgyoung every background scan I see the following error message in logcat:
E/Callback: Failed attempting to start service: com.app.client.staging/org.altbeacon.beacon.BeaconIntentProcessor and I never receive didRangeBeaconsInRegion callback.
What this could be?

[davidgyoung]

@Ch3D, see #547 for a description of this issue. I will post a beta update to fix this here: https://github.com/AltBeacon/android-beacon-library/tree/2.12-beta3

See updated beta 4 release below.

[Ch3D]

@davidgyoung yeah, I saw it, thanks David. Will try it out later today

[shliama]

Hi, everyone. I want to report an issue with the 2.12-beta3 version #551

[Ch3D]

Reported another issue here: #550

[davidgyoung]

Thanks, @shliama for the report on the crash with PREVIEW_SDK_INT. I had intended to remove all those references, but apparently forgot one. This is fixed in 0dac329

[davidgyoung]

Here is a beta 4 release that fixes:

• Possible corruption of the status preservation reported in java.io.OptionalDataException when trying to start ScanJob in background #550
• References to PREVIEW_SDK_INT that caused crashes on Android 5 and earlier targets. java.lang.NoSuchFieldError: No static field PREVIEW_SDK_INT on ScanJob start #551

https://github.com/AltBeacon/android-beacon-library/tree/2.12-beta4

[themaincharacter]

@davidgyoung Pardon me if this is dumb but any help/documentation in how to implement the new ScanJob class? Do I use it just like any other job in a JobScheduler?

[davidgyoung]

The ScanJob class is designed to be an internal component of the library that you don't have to worry about. If you are running the library in an app on Android 8.0, the library will use it by default to schedule scans for beacons. If you are running the library in an app on Android 4.3-7.x, it will keep using a long running BeaconService to do the scanning.

[themaincharacter]

Thanks for the quick response! Thats pretty convenient. So just so I understand you correctly my current implementation with a service implementing the RangeNotifier and persisting in the background will not need to be refactored?

[davidgyoung]

You may need to refactor for changes in Android 8, even if not for changes in this library, as Android 8 blocks long-running background services. Read more here: http://www.davidgyoungtech.com/2017/08/07/beacon-detection-with-android-8

[chintan-mishra]

@davidgyoung would it be feasible to schedule an exact job using JobScheduler and scheduling same job within the scheduled exact job?

[davidgyoung]

@chintan-mishra are you suggesting this as a way of getting around the 15 minute limit on Android 8?

[chintan-mishra]

Yes, @davidgyoung that's exactly what I am trying to say.