Submitting sub or child jobs

You want to submit sub-jobs within a task.

Get a reference to the IJobSchedulerContext interface by calling getProperty(java.lang.String) and passing in TaskProperties.JOB_SCHEDULER_CONTEXT.

IJobSchedulerContext schedulerContext = (IJobSchedulerContext) this.getProperty(TaskProperties.JOB_SCHEDULER_CONTEXT);

With a reference to the IJobSchedulerContext interface, call createChildJob() to create a job.

IJob job = schedulerContext.createChildJob();

package stkparallelcomputingserversdk.howto;

import agi.parallel.client.ClusterJobScheduler;
import agi.parallel.client.IJobScheduler;
import agi.parallel.client.Job;
import agi.parallel.infrastructure.IJob;
import agi.parallel.infrastructure.IJobSchedulerContext;
import agi.parallel.infrastructure.Task;
import agi.parallel.infrastructure.TaskProperties;

public class SubmittingSubJobs {
    public static void main(String[] args) {
        try (IJobScheduler scheduler = new ClusterJobScheduler("localhost")) {
            scheduler.connect();

            Job job = scheduler.createJob();
            job.addTask(new ParentTask());

            job.submit();
            job.waitUntilDone();

            System.out.println(job.getTasks().get(0).getResult());
        }

        /*
         * The output of the application should resemble:
         * From Parent -> From Child
         */
    }

    public static class ParentTask extends Task {
        @Override
        public void execute() {
            // Get the instance of the job scheduler context
            IJobSchedulerContext schedulerContext = (IJobSchedulerContext) this.getProperty(TaskProperties.JOB_SCHEDULER_CONTEXT);
            // Create a child job by calling CreateChildJob.
            IJob job = schedulerContext.createChildJob();
            Task task = new ChildTask();
            job.addTask(task);
            job.submit();
            job.waitUntilDone();
            this.setResult("From Parent -> " + task.getResult());
        }
    }

    public static class ChildTask extends Task {
        @Override
        public void execute() {
            this.setResult("From Child");
        }
    }
}

Along with being submitted by a client, jobs can also be submitted in a task. Submitting sub-jobs can be a powerful tool used to handle dynamic workloads such as divide and conquer computations. Sub-jobs have all the same fields and properties as a standard job, e.g. events, monitoring, status reporting, etc. When sub-jobs are submitted they are submitted back to the original client's job scheduler and tracked the same way a task submitted from a regular client is tracked. Sub-jobs can also have sub-jobs of their own, which enables tasks to recursively spawn work to child jobs.

Note
createChildJob() returns an IJob. This is just a Job object and can be cast to the Job class if needed.

There are some advanced details about sub-jobs that are useful to know when debugging an issue. The most important concept to understand is how sub-jobs deal with allocated resources. Internally, a sub-job is treated just like a regular job; before a job can run, it needs available resources to consume. For both jobs and sub-jobs, cores are the most common of these resources.

To explain how they are different is best done through an example. If a regular job is submitted to a job scheduler that has no cores available, the job gets queued and waits until a core becomes available. When a sub-job is submitted to a job scheduler that has no cores available, the child job gets immediately executed. This happens because internally the parent task yields all its resources to the child job. This works transparently without the user doing anything. If this is not the desired behavior, there is an overload to createChildJob(boolean) that allows a yieldAllResourcesToChildren parameter. If the yieldAllResourcesToChildren parameter is false, resource yielding will not be automatic. The sub-job waits in the queue like a regular job.

Note
When submitting sub-jobs, the most common use case is that the child job handles the workload of its parent while the parent waits for the work to finish. By default, the reason all resources are provided to the children is to try to make this use case as easy as possible.

Other Resources