Library Reference

This Library Reference contains information on all API types. To learn more about the contents of the library, browse the table of contents. You can also find information on a specific type in that way, or by entering its name in the Search field.

Packages

Modules

Modules

Description

Client

Provides types that are used to submit and monitor jobs to a job scheduler.

Constants

Provides constants and enumerations used with the AppServer.

Client

The client module is responsible for submitting jobs and monitoring the cluster.

class agiparallel.client.ClusterJobScheduler(hostname=None, port=9090, use_ssl=False, allow_self_signed_certificates=False, ssl_client_authentication_timeout=30, provide_client_certificate=False, client_certificate_certfile=None, client_certificate_keyfile=None, client_certificate_password=None, coordinator_thumbprint=None)

Bases: object

Submits tasks to the coordinator. Accepts client_certificate_certfile as the certificate, and client_certificate_keyfile as the certificate’s private key. If these are in one file that file may be supplied for both arguments.

cancel_job(job_id, listen_for_cancellation_event=False)

Cancels the job with the specified id.

cancel_task(task_id)

Cancels the task with the specified id.

connect()

Connects to the coordinator.

create_job()

Creates a job with access to task submission.

disconnect()

Disconnects from the coordinator.

get_coordinator_snapshot()

Returns snapshot of coordinator state.

get_server_maximum_host_count()

Returns the maximum number of hosts available to the coordinator.

property id

Gets the client’s current connection ID.

is_connected()

Gets a value indicating whether the job scheduler is connected.

on_coordinator_disconnected()

Called when the job scheduler connection with the coordinator is interrupted. Attempts to re-establish a connection with the coordinator until either a connection is obtained or the maximum number of attempts to reconnect is reached.

reconnect()

Reconnects to the coordinator.

submit_job(job, task_completed_callback, task_state_changed_callback, job_submitted_callback, job_completed_callback, task_progress_callback)

Submits a job to the job scheduler so the job scheduler can add the job to its queue.

wait_until_done(job_id, seconds_timeout, heartbeat_callback, seconds_heartbeat)

Blocks until all tasks in this job complete or the operation times out. The callback passed in for the heartbeat is called with the period specified by the secondsHeartbeat argument.

exception agiparallel.client.ClusterJobSchedulerException

Bases: Exception

An exception thrown when a client-side error occurs.

class agiparallel.client.CoordinatorProxy(connection, connection_id)

Bases: object

Proxy to AppServer coordinator. Knows how to submit tasks and receive updates.

cancel_job(job_id)

Cancels the job previously submitted to the coordinator.

property disconnected

Occurs when the Coordinator connection is disconnected.

get_coordinator_snapshot()

Waits until all tasks of job are done.

property last_error_message

The last error message.

restart(connection, connection_id)

Restarts the CoordinatorProxy using a new connection. Should only be called if the previous coordinator connection was disconnectedand the heartbeat and message threads stopped executing (i.e. stop has been called).

start()

Starts the heartbeat and message loops.

stop()

Stops the client from communicating with the Coordinator via the proxy

submit_job(job, job_submitted_callback, task_completed_callback, task_updated_callback, task_progress_updated_callback)

Submits job to Coordinator.

property terminated_gracefully

Indicates whether the coordinator proxy was terminated gracefully as part of the system’s behavior (true) or due to an interruption in the connection to the coordinator (false).

class agiparallel.client.DefaultTaskEnvironment

Bases: TaskEnvironment

Default task environment used in a job.

setup()

Called by Host before tasks are executed.

teardown()

Called by Host after tasks are executed.

class agiparallel.client.FutureTaskResult(thread, task)

Bases: object

join(timeout=None)
exception agiparallel.client.InfrastructureException

Bases: Exception

An exception thrown when an infrastructure error occurs.

class agiparallel.client.Job(client)

Bases: object

A job is a collection of tasks with the correct context for receiving updates from and being submitted to the coordinator.

add_dependency(dependency)

Adds an additional dependency sent with the Job

add_task(task, params=None)

Adds a task to job.

add_task_preconditions(resource, operation, resource_value)

Adds the task precondition.

agent_selection_preference

The scheduling algorithm to use when selecting which agent should execute a task. Default value = AgentSelectionType.DEFAULT

cancel(listen_for_cancellation_event=False)

Cancels all tasks of job.

cancel_on_client_disconnection

Whether to cancel all tasks if the client disconnects. Appropriate for interactive jobs. Default value = False

cancel_on_task_failure

Whether to cancel all tasks if any task in the job fails. Appropriate if the result of the job depends on all the tasks completing successfully. Default value = False

description

Description of the job displayed in the monitoring applications. Default value = None

max_interrupted_retry_attempts

Number of times the job’s tasks will be retried if the task gets interrupted. Default value = 5

name

Name of the job displayed in the monitoring applications. Default value = Job #

priority

Priority of the job’s tasks. Higher priority tasks are selected to be executed before lower priority tasks. Default value = JobPriority.NORMAL

set_task_environment(task_environment)

Sets the TaskEnvironment for the job

submit()

Submits the job to the Coordinator.

task_execution_timeout

List of conditions that must be met by an agent machine before a task is assigned to it.

task_preconditions

List of conditions that must be met by an agent machine before a task is assigned to it. Default value = []

wait_until_done(seconds_timeout=None, heartbeat_callback=None, seconds_heartbeat=None)

Waits until all tasks of job are in a completed state.

class agiparallel.client.JobDispatcher(client_id, job_id, job_submitter, task_completed_callback, task_state_changed_callback, job_submitted_callback, job_completed_callback, task_progress_callback)

Bases: object

Provides job operations.

EVENT_COORDINATOR_DISCONNECTED = 3
EVENT_COORDINATOR_LOST = 4
EVENT_JOB_CANCELED_HANDLE = 2
EVENT_PENDING_ALL_TASKS_DONE_HANDLE = 0
EVENT_PENDING_HANDLE = 1
cancel(listen_for_cancellation_event=False)

Cancels this current job request.

on_coordinator_disconnected()

Called when the job scheduler connection with the coordinator is interrupted.

on_coordinator_lost()

Called when the job scheduler connection with the coordinator is lost.

submit(job)

Asynchronously submits all tasks to the Coordinator.

wait_until_done(timeout, heartbeat_callback, seconds_heartbeat)

Waits until all tasks associated with this client are finished.

class agiparallel.client.TaskCallbacksContext(task, task_completed_callback, task_state_updated_callback, task_progress_updated_callback)

Bases: object

Wraps tasks with their callbacks

property task

Gets the task of this wrapper.

property task_completed_callback

Gets the callback to invoke when the task is completed.

property task_progress_updated_callback

Gets the callback to invoke when the task progress is updated.

property task_state_updated_callback

Gets the callback to invoke when the task state is updated.

class agiparallel.client.TaskStatusChangedEventArgs(previous_status, new_status, task, task_state)

Bases: object

Data for task status change event.

agiparallel.client.get_task_name(task)

High level way to get a task name.

agiparallel.client.submit(coordinator, task, task_environment=None, params=None, port=9090)

High level way to submit task.

Constants

Common constants for AppServer

class agiparallel.constants.AgentSelectionType(value)

Bases: IntEnum

Controls which agent is selected to execute a task.

DEFAULT = 0
LEAST_RECENTLY_SELECTED = 2
MOST_RECENTLY_SELECTED = 3
PRIORITY = 1
RANDOM = 4
class agiparallel.constants.CommonResources

Bases: object

Common resource values.

AVAILABLE_CORES = 'AvailableCores'
AVAILABLE_HARD_DISK = 'AvailableHardDisk'
AVAILABLE_MEMORY = 'AvailableMemory'
CURRENT_WORKLOAD = 'CurrentWorkload'
HOSTNAME = 'Hostname'
IS_64BIT_PROCESS = 'Is64BitProcess'
OS_PLATFORM = 'OSPlatform'
TOTAL_CORES = 'TotalCores'
class agiparallel.constants.ConsumableResources

Bases: object

Consumable Resource values.

CORES = 'Cores'
ESTIMATED_MEMORY_BUDGET = 'EstimatedMemoryBudget'
MEMORY = 'Memory'
class agiparallel.constants.CoordinatorCommand(value)

Bases: IntEnum

Commands to the Coordinator.

CANCEL_TASK = 6
CHANGE_AGENT_PRIORITY = 5
CLEAR_LOG = 7
DISABLE_AGENT = 4
ENABLE_AGENT = 3
HEARTBEAT_RESPONSE = 8
LICENSE_INFO = 9
NOTHING = 0
POLL = 1
REQUEST_LOG = 2
class agiparallel.constants.CpuArchitecture(value)

Bases: IntEnum

Specifies the target CPU type of the host process.

ANY = 0
X64 = 2
X86 = 1
class agiparallel.constants.ExecutionModeType(value)

Bases: IntEnum

The execution mode of a job.

BATCH = 0
INTERACTIVE = 1
class agiparallel.constants.JobPriority(value)

Bases: IntEnum

Job priority provides control on order of jobs in the queue.

ABOVE_NORMAL = 3
BELOW_NORMAL = 1
HIGH = 4
LOW = 0
NORMAL = 2
class agiparallel.constants.LoggingLevel(value)

Bases: IntEnum

The different logging levels.

DEBUG = 0
ERROR = 3
FATAL = 4
INFO = 1
WARN = 2
class agiparallel.constants.OSPlatform(value)

Bases: IntEnum

OS platform values.

LINUX = 2

Linux platform.

UNSPECIFIED = 0

Unspecified platform. Indicates the agent’s version is before OS Platform was implemented. Update the agent to 2.6 or greater.

WINDOWS = 1

Windows platform.

class agiparallel.constants.Operator(value)

Bases: IntEnum

Represents an operation code.

EQUAL_TO = 2
GREATER_THAN = 0
LESS_THAN = 1
MEMBER_OF = 4
NOT_EQUAL_TO = 3
class agiparallel.constants.TaskCancellationReason(value)

Bases: IntEnum

Reason for the cancellation of a task.

CONSEQUENTIAL = 2
DIRECT = 1
NONE = 0
class agiparallel.constants.TaskExecutionType(value)

Bases: IntEnum

Types of task execution host.

DOTNET = 1
PYTHON = 2
UNSET = 0
class agiparallel.constants.TaskProperties

Bases: object

Common task properties.

ENVIRONMENT = 'Environment'
GRACEFUL_CANCELLATION_TIMEOUT = 'GracefulCancellationTimeOut'
HOST_END_TIME = 'HostEndTime'
HOST_START_TIME = 'HostStartTime'
INTERRUPTED_RETRY_ATTEMPTS = 'InterruptedRetryAttempts'
JOB_SCHEDULER_CONTEXT = 'JobSchedulerContext'
LOGGER = 'Logger'
SUBMITTER_ID = 'SubmitterId'
class agiparallel.constants.TaskStatus(value)

Bases: IntEnum

Status of a task.

ASSIGNED = 2

Task is assigned but not run yet.

CANCELED = 6

Task is canceled.

CANCELING = 4

Task is in the process of canceling. is_cancelling flag is set to true.

COMPLETED = 8

Task completed successfully.

ENVIRONMENT_ERROR = 7

Task failed to run because of an uncaught exception in setup().

FAILED = 10

Task failed because of an uncaught exception while running the task.

INTERRUPTED = 5

Task encountered a system exception. Examples of system exceptions include if the agent disconnects or the host process exits unexpectedly.

NOT_SUBMITTED = 0

Task is not submitted yet.

RUNNING = 3

Task is currently running.

SUBMITTED = 1

Task is submitted but not assigned yet.

TIMED_OUT = 9

Task timed-out because it ran longer than the specified TASK_EXECUTION_TIMEOUT value.