Click or drag to resize

Dynamically control the resources allocated to task

Problem

You want to dynamically change the consumed resources of the task.

Solution

Use the IJobSchedulerContext interface and call yieldResource(java.lang.String, long) to decrease the resources consumed by the task and reserveResource(java.lang.String, long) to increase the resources consumed by the task.

Java
package stkparallelcomputingserversdk.howto;

import agi.parallel.client.ClusterJobScheduler;
import agi.parallel.client.IJobScheduler;
import agi.parallel.client.Job;
import agi.parallel.infrastructure.ConsumableResources;
import agi.parallel.infrastructure.IJobSchedulerContext;
import agi.parallel.infrastructure.Task;
import agi.parallel.infrastructure.TaskProperties;

public class DynamicallyControlConsumedResources {
    public static void main(String[] args) {
        try (IJobScheduler scheduler = new ClusterJobScheduler("localhost")) {
            scheduler.connect();

            Task task = new YieldResourceDemo();
            Job job = scheduler.createJob();
            job.addTask(task);

            job.submit();
            job.waitUntilDone();

            System.out.print(task.getStandardOutput());
        }

        /*
         * The output of the application should resemble:
         * Initial: Consuming 1 Cores
         * After YieldResource: Consuming 0 Cores
         * After ReserveResource: Consuming 1 Cores
         */
    }

    public static class YieldResourceDemo extends Task {
        @Override
        public void execute() {
            IJobSchedulerContext jobSchedulerContext = (IJobSchedulerContext) this.getProperty(TaskProperties.JOB_SCHEDULER_CONTEXT);

            // You can pass 0 to query that amount the task is currently consuming.
            long currentConsumption = jobSchedulerContext.yieldResource(ConsumableResources.CORES, 0);
            System.out.format("Initial: Consuming %s Cores%n", currentConsumption);

            // Let's yield the resource to the global resource poll
            currentConsumption = jobSchedulerContext.yieldResource(ConsumableResources.CORES, 1);
            System.out.format("After YieldResource: Consuming %s Cores%n", currentConsumption);

            // We can also dynamically increase our resource consumption.
            currentConsumption = jobSchedulerContext.reserveResource(ConsumableResources.CORES, 1);
            System.out.format("After ReserveResource: Consuming %s Cores%n", currentConsumption);
        }
    }
}
Discussion

Accurate resource tracking is important for efficiently using all available resources. A high degree of flexibility is allowed for more advanced cases where more control is needed for specifying which task is consuming which resource. Resource yielding comes in handy when a known task is not doing any real work and the system should consider its resources available to the system. The most common use case for this is when the task is waiting for work to be done. Resource reserving is much less common. Reserve a resource when more work is about to be done and the system should be notified.

Resources also have a concept of private and global availability. A global resource is the most common and the default. Global resources are available to all tasks--there are no restrictions. A private resource is a special kind of resource that restricts who can consume it, most commonly children of the task. Make a resource private by specifying true to the restrictToChildren parameter of yieldResource(java.lang.String, long, boolean). Resources from private resources are consumed first before consuming global resources. The most common reason for using private resources over global resources is to ensure child tasks will have a pool of resources available only to them. That is, using private resources can guarantee that a child task will not have to wait in the queue because it does not have the necessary resources.

Important note Important

When a task submits a child job all of its resources are yielded by default. You will only have to yield resources if you choose to manually do so.

See Also

STK Parallel Computing Server 2.9 API for Java