AWS adds instance preference lists to SageMaker AI jobs

AWS has added ordered instance preference lists to Amazon SageMaker AI training and processing jobs. Each job configuration can rank between one and five acceptable ML compute instance types.

SageMaker works through the list in order, launching the job on the first instance type with available capacity and moving to the next preference when capacity is insufficient. It selects exactly one instance type for the job.

That puts approved alternatives into a single submission. A caller no longer needs to submit a separate job just to try the next listed type. The API references define this selection behavior, but they do not establish shorter wait times or guarantee that capacity will be available.

For training jobs, instance preferences support on-demand capacity and Flexible Training Plans. The list cannot be combined with the existing single InstanceType field, instance groups, managed spot training or instance placement configuration. For processing jobs, the list is mutually exclusive with the single InstanceType field.

Instance counts can be set on every entry or once at the job level. AWS says it rejects configurations that provide counts for only some preferences, set them at both levels or omit them at both levels. The rules are documented separately for training preferences and processing preferences.

A training preference can also point to matching Training Plan capacity; a preference without a TrainingPlanArns entry uses on-demand capacity. The per-preference setting cannot be combined with a job-level TrainingPlanArn.

The job-level setting is distinct from SageMaker’s prefix-aware routing for real-time endpoints, which concerns inference traffic rather than instance selection for a training or processing job.

When a training preference list includes at least one accelerated instance type, MaxPendingTimeInSeconds limits the total time SageMaker spends working through the list; it is not applied separately to each preference. AWS says the setting has no effect on jobs that request only CPU instance types.