15 minute cron.php magic value?

[guruz]

I noticedwhen trying announcement_center that announcements are only shipped when cron is run(https://github.com/owncloud/announcementcenter/blob/master/lib/BackgroundJob.php @tomneedham )

Which is every 15 minutes when you configure according to documentation, but maybe more when webcron or ajax cron is used. https://doc.owncloud.org/server/10.0/admin_manual/configuration/server/background_jobs_configuration.html#cron
So when using system cron the magic 15 minute value is also in cron.php: https://github.com/owncloud/core/blob/master/cron.php#L110

My gut feeling would say that background jobs should be run much more often (once per minute), then my announcement notification would have showed up much earlier...

But @butonic would disagree as jobs could take ages: #25350 (comment)
Although we support concurrent cron.php invocations though so maybe it's not an issue as a job would get locked while it is executed, so in the next minute would run another job?)

(thanks @PVince81 for the chat)

Maybe this issue is a duplicate/overlap with #23544

[DeepDiver1975]

My gut feeling would say that background jobs should be run much more often (once per minute), then my announcement notification would have showed up much earlier...

absolutly - docs need to be adjusted @settermjd

[settermjd]

So, it's agreed that the value's to be adjusted to every minute?

[PVince81]

Will system cron automatically run the command in parallel in case invocation 1 is taking much longer ? Or will cron skip the next runs until the first invocation returns ?

[guruz]

@PVince81 Implementation detail I guess.. (and since you say parallel invocations are OK it's fine)

[PVince81]

I'm talking about system cron, the stuff that runs on a Linux system, not ownCloud cron.

[guruz]

@PVince81 me too. system cron invoking php cron.php in parallel or serial is both oK?

[PVince81]

Yes, but might influence the expectation of the admin. If we are sure that all versions of system cron behave the same then the recommendation will be accurate, no surprises.

[GrendelOnCampus]

We're running ownCloud in bigger installations. cron.php is running every 15 minutes. In our biggest installations with more than 10.000 users cron.php is running sometimes really long, sometimes for several hours. (6-day activity with 900k entries). So it won't make sense for us, to run cron.php every minute. For others, it might be useful.
Maybe, you should make it possible to send announcements via occ script separately, when this is the reason calling cron.php more often than every 15 minutes.
So, each admin will have to decide, which frequency will be the best for his installation.

@PVince81: As far, as I know, the Linux cron daemon is executing cron jobs from crontab every minute, if you configured * * * * * as time/date specs. */15 * * * is executing every 15 minutes.

[cdamken]

@GrendelWWU the jobs will automatically run when is needed, not all jobs will start each minute (for example email-activity starts only every 15 minutes) and if you have many server it will bock that job until it finished.

some other jobs like trashbin or versions would be better if they start at different times.

As you say, the admin at the end of the day has the last word, but I find the idea good that the server manages their working times better.

I will test it and I will provide more feedback later.

[tomneedham]

@GrendelWWU could you elaborate on "6-day activity with 900k entries" I am interested about this as I have been touching this code recently to make some improvements. Separate ticket? Or central?

[tomneedham]

So whilst we have job locking ( so parallel cron runs is ok ), some jobs take longer than their interval, which means they will be running in parallel and the number of parallel running jobs can gradually increase out of control.

In reality, we need to optimise our jobs, so that they are not taking such long times. I would be interested in any large jobs which take > 15 minutes to see where we can make improvements => new tickets.

[GrendelOnCampus]

Hello Tom, we're storing our activity for 6 days, but sometimes cleaning will fail. We discussed that some time ago. Cleaning in slices would be a great idea.

Look at this:
MariaDB [rwth_aachen]> select from_unixtime(timestamp) from oc_activity order by timestamp limit 1;
+--------------------------+
| from_unixtime(timestamp) |
+--------------------------+
| 2018-03-01 22:23:01 |
+--------------------------+
1 row in set (0.02 sec)

MariaDB [rwth_aachen]> select from_unixtime(timestamp) from oc_activity order by timestamp desc limit 1;
+--------------------------+
| from_unixtime(timestamp) |
+--------------------------+
| 2018-03-09 13:15:50 |
+--------------------------+
1 row in set (0.00 sec)

MariaDB [rwth_aachen]> select count() from oc_activity;
+----------+
| count() |
+----------+
| 2683162 |
+----------+
1 row in set (0.36 sec)

[tomneedham]

Thanks @GrendelWWU . Addressed here - owncloud/activity#610 :) Chunking used on mysql where DELETE with LIMIT is supported - and the only case at the moment where the large row number delete is known to cause issues.

[tomneedham]

We're running ownCloud in bigger installations. cron.php is running every 15 minutes. In our biggest installations with more than 10.000 users cron.php is running sometimes really long, sometimes for several hours. (6-day activity with 900k entries). So it won't make sense for us, to run cron.php every minute. For others, it might be useful.

As @cdamken mentioned this should not matter. Internally, our 'Jobs' have their own intervals. Some could be defined as per-minute in which case they would be triggered with each cron.php execution, others are hourly so despite cron.php being executed every minute, they would only run once per hour.

Yes - we still face issues where jobs overlap their own intervals, but this is the same as we face right now.

[tomneedham]

@PVince81 I propose making 1 minute the default. Internally I believe we have the necessary mechanisms for this to work correctly.

[GrendelOnCampus]

Ok, but we're having up to 10 ownCloud/sciebo Installations on one server. So we configured our crontab, so run cron.php every 15 minutes. Every call for each instance is coded separately.
So, not all jobs will start at the same time...
cronphp.txt

[ownclouders]

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 7 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)

[cdamken]

Ok, but we're having up to 10 ownCloud/sciebo Installations on one server. So we configured our crontab, so run cron.php every 15 minutes. Every call for each instance is coded separately.
So, not all jobs will start at the same time...
cronphp.txt

Thomas, it will be better that you configure on all your servers. then the load of the process is not on one:

for example:

Server1:

# m h dom mon dow command
*/5 * * * * /usr/bin/php -f /var/www/owncloud/cron.php
8 0 * * * /usr/bin/php -f /var/www/owncloud/occ dav:cleanup-chunks
13 0 * * * /usr/bin/php -f /var/www/owncloud/occ dav:sync-system-addressbook
18 0 * * * /usr/bin/php -f /var/www/owncloud/occ federation:sync-addressbooks
23 0 * * * /usr/bin/php -f /var/www/owncloud/occ occ trashbin:expire
28 0 * * * /usr/bin/php -f /var/www/owncloud/occ occ versions:expire

Server2:

# m h dom mon dow command
*/5+1 * * * * /usr/bin/php -f /var/www/owncloud/cron.php

Server3:

# m h dom mon dow command
*/5+2 * * * * /usr/bin/php -f /var/www/owncloud/cron.php

[ownclouders]

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 7 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)

[ownclouders]

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 7 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.