Fenzo does not account for the resources of a custom executor

[zonybob]

I am running tasks with a custom dockerized executor that needs 0.5 cpus.
If (let's say) all my tasks need 0.1 cpus , and Fenzo gets a lease for 1.0 cpu, what normally happens is that "scheduleOnce" tries to pair up ten tasks against that lease... so I schedule those ten.

• Task 1 launches fine, but because it's the first time the executor runs on that agent, there are actually 0.6 CPUs used in the process (0.5 + 0.1)
• Task 2 launches fine (now we're at 0.7 cpus)
• Task 3 launches fine (now we're at 0.8 cpus)
• Task 4 launches fine (now we're at 0.9 cpus)
• Task 5 launches fine (now we're at 1.0 cpus)
• Tasks 6-10 fail with TASK_ERROR - effectively saying the task resources are more that what is left in the offer.

I can work around it by checking if my custom executor is not part of the offer and then manually summing resources and not scheduling tasks that would cause a TASK_ERROR, but of course this is somewhat duplicating what I hope Fenzo would do for me, and I am bound to screw it up.

[spodila]

There isn't a direct way to do this at this time. However, it would be fairly simple to achieve this via a custom hard constraint that you would attach for every task. The hard constraint would be an implementation of the interface ConstraintEvaluator. In the evaluate() method implementation of the interface, you would do this:

• Check if targetVM.getRunningTasks() already contains a task of the executor you are going to launch. If so, the constraint passes
• If not, subtract the resources that your executor needs from the total resources available: look it up using targetVM.getCurrAvailableResources().
• Fail the constraint if total resources would exceed the modified total.

[zonybob]

The lazy part of me was hoping that there'd be a direct way, but I think the method you presented is still much cleaner than what I was doing.

It was a bit more work than I'd expected... I had to effectively check if
(targetVM.getCurrAvailableResources - sum(resources from targetVM.getTasksCurrentlyAssigned()) - executorInfoResources) < taskRequestResources.

Seems to be working though. Thank you for the suggestion!

[spodila]

Closing this since you confirmed that the suggestions works for you.