Problem

You need fine grain control over how many cores a task is consuming.

Solution

On the task, call setProperty(java.lang.String, java.lang.Object) and pass the resource to be configured as the second parameter. There are currently two resources that can be configured for consumption: cores and memory. These can be set by specifying TaskProperties.MAX_CORES_FOR_TASK and TaskProperties.MAX_MEMORY_FOR_TASK respectively.

Java  Copy imageCopy
task.setProperty(TaskProperties.MIN_CORES_FOR_TASK, 2);
task.setProperty(TaskProperties.MAX_CORES_FOR_TASK, 4);

A task can inspect the following properties when it executes to retrieve the resource amounts allocated to it: TaskProperties.ALLOCATED_CORES_FOR_TASK and TaskProperties.ALLOCATED_MEMORY_FOR_TASK.

Java  Copy imageCopy
long coresAllocated = (Long) this.getProperty(TaskProperties.ALLOCATED_CORES_FOR_TASK);

Below is a code example that demonstrates how to configure a task to consume between 2 and 4 cores.

Java  Copy imageCopy
package stkscalabilitysdk.howto;


import agi.parallel.client.ClusterJobScheduler;
import agi.parallel.client.IJobScheduler;
import agi.parallel.client.Job;
import agi.parallel.infrastructure.Task;
import agi.parallel.infrastructure.TaskProperties;

public class ControlResourcesGivenToTask {
    public static void main(String[] args) {
        IJobScheduler scheduler = new ClusterJobScheduler("localhost");
        try {
            scheduler.connect();

            Job job = scheduler.createJob();
            Task task = new ReturnsNumberOfAllocatedCores();

            // Configure the task to use at least 2 cores, but no more than 4.
            task.setProperty(TaskProperties.MIN_CORES_FOR_TASK, 2);
            task.setProperty(TaskProperties.MAX_CORES_FOR_TASK, 4);

            job.addTask(task);
            job.submit();
            job.waitUntilDone();
            System.out.println("Cores allocated: " + task.getResult());
        } finally {
            scheduler.dispose();
        }

        /*
         * The output of the application should resemble:
         * Cores allocated: 4
         */
    }

    public static class ReturnsNumberOfAllocatedCores extends Task {
        @Override
        public void execute() {
            // You can get the number of cores allocated by inspecting the TaskProperties.ALLOCATED_CORES_FOR_TASK property
            // You can use this value to decide how many threads you can spawn in your task.
            long coresAllocated = (Long) this.getProperty(TaskProperties.ALLOCATED_CORES_FOR_TASK);
            this.setResult(coresAllocated);
        }
    }
}

Discussion

When a task executes, what is happening internally is that the task is consuming a resource. The most common of these resources is "Cores". By default a task consumes one resource of type "Cores" on an agent. On a machine with eight cores an agent can run eight concurrent tasks at a time until its "Cores" resource is exhausted. The amount of resources a task can consume can be optionally specified. For instance, if each task is configured to consume at least two cores, the eight core machine can only run four of these tasks concurrently.

Using another example, imagine a task believed to consume 2GB of memory. On an agent with 6GB of memory but eight cores, the agent would ideally only execute three of these tasks concurrently to prevent the agent from swapping massive amounts of memory. In this case, configure the task to consume 2GB of memory.

Note that tasks are not restricted from only using the resources allocated to them. That is, a task with two resources of type "Cores" can spawn as many threads as it wants. The task will not be stopped from over consuming its declared resources. Efficient use of resource consumption requires that tasks are coded to be honest and only use the resources they are allocated.

See Also