Api-Sagemakertrainingsessionruntime-2025-02-10

Client: Aws\SageMakerTrainingSessionRuntime\SageMakerTrainingSessionRuntimeClient

Service ID: sagemakertrainingsessionruntime

Version: 2025-02-10

This page describes the parameters and results for the operations of the SageMaker Training Session Runtime (2025-02-10), and shows how to use the Aws\SageMakerTrainingSessionRuntime\SageMakerTrainingSessionRuntimeClient object to call the described operations. This documentation is specific to the 2025-02-10 API version of the service.

Operation Summary

Each of the following operations can be created from a client using $client->getCommand('CommandName'), where "CommandName" is the name of one of the following operations. Note: a command is a value that encapsulates an operation and the parameters used to create an HTTP request.

You can also create and send a command immediately using the magic methods available on a client object: $client->commandName(/* parameters */). You can send the command asynchronously (returning a promise) by appending the word "Async" to the operation name: $client->commandNameAsync(/* parameters */).

ExecuteSamplingOperation ( array $params = [] ): Submits an asynchronous sampling operation to generate text or compute log-probabilities from the model associated with the specified training session.
ExecuteTrainingOperation ( array $params = [] ): Submits an asynchronous training operation to the specified training session.
GetSamplingOperationResult ( array $params = [] ): Retrieves the current status and result of a previously submitted sampling operation.
GetTrainingOperationResult ( array $params = [] ): Retrieves the current status and result of a previously submitted training operation.
SampleTrainingSession ( array $params = [] ): Sends an inference request to the model associated with the specified training session and returns the response synchronously.
SampleTrainingSessionWithResponseStream ( array $params = [] ): Streams inference responses from the model associated with the specified training session.

Operations

ExecuteSamplingOperation

$result = $client->executeSamplingOperation([/* ... */]);
$promise = $client->executeSamplingOperationAsync([/* ... */]);

Submits an asynchronous sampling operation to generate text or compute log-probabilities from the model associated with the specified training session. The session must be in Running or Blocked state. Use GetSamplingOperationResult to poll for the result.

Parameter Syntax

$result = $client->executeSamplingOperation([
    'operationConfig' => [ // REQUIRED
        'computeLogprobs' => [
            'modelPackageArn' => '<string>',
            'prompt' => [ // REQUIRED
                'chunks' => [ // REQUIRED
                    [
                        'tokens' => [<integer>, ...], // REQUIRED
                        'type' => 'encoded_text', // REQUIRED
                    ],
                    // ...
                ],
            ],
        ],
        'loadWeightsForSampling' => [
            'modelPackageArn' => '<string>', // REQUIRED
        ],
        'sample' => [
            'includePromptLogprobs' => true || false,
            'modelPackageArn' => '<string>',
            'numSamples' => <integer>, // REQUIRED
            'prompt' => [ // REQUIRED
                'chunks' => [ // REQUIRED
                    [
                        'tokens' => [<integer>, ...], // REQUIRED
                        'type' => 'encoded_text', // REQUIRED
                    ],
                    // ...
                ],
            ],
            'samplingParams' => [ // REQUIRED
                'maxTokens' => <integer>, // REQUIRED
                'seed' => <integer>,
                'stop' => ['<string>', ...],
                'temperature' => <float>,
                'topK' => <integer>,
                'topP' => <float>,
            ],
            'topkPromptLogprobs' => <integer>,
        ],
    ],
    'trainingSessionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

operationConfig

: Required: Yes
: Type: SampleOperationConfig structure

The configuration for the sample operation. Specify either a sample configuration to generate text or a computeLogprobs configuration to compute token log-probabilities.

trainingSessionName

: Required: Yes
: Type: string

The name of the training session to sample from.

Result Syntax

[
    'operationId' => '<string>',
]

Result Details

Members

operationId

: Required: Yes
: Type: string

The unique identifier for the submitted sample operation. Use this identifier with GetSamplingOperationResult to retrieve the operation status and result.

Errors

ConflictException:: The training session is in a state that cannot accept new operations. The session must be in Running or Blocked state to accept requests.
ResourceNotFoundException:: The specified training session or operation does not exist, or the operation does not belong to the specified training session.
ThrottlingException:: The request was throttled. Retry the request after a brief wait using exponential backoff.
InternalServerException:: An internal service error occurred. Retry the request using exponential backoff.
ValidationException:: The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Examples

Example 1: Generate samples from a prompt

Generates completions for a tokenized prompt using the session's current weights. Generation behavior is controlled by `samplingParams`, and the operation is dispatched asynchronously.

$result = $client->executeSamplingOperation([
    'operationConfig' => [
        'sample' => [
            'numSamples' => 4,
            'prompt' => [
                'chunks' => [
                    [
                        'type' => 'encoded_text',
                        'tokens' => [
                            791,
                            6864,
                            315,
                            9822,
                            374,
                        ],
                    ],
                ],
            ],
            'samplingParams' => [
                'maxTokens' => 1024,
                'temperature' => 0.7,
                'topP' => 0.9,
            ],
        ],
    ],
    'trainingSessionName' => 'my-training-session',
]);

Result syntax:

[
    'operationId' => 'b2c3d4e5-6789-01bc-def0-234567890abc',
]

ExecuteTrainingOperation

$result = $client->executeTrainingOperation([/* ... */]);
$promise = $client->executeTrainingOperationAsync([/* ... */]);

Submits an asynchronous training operation to the specified training session. The service executes operations in sequence number order. The session must be in Running or Blocked state. Use GetTrainingOperationResult to poll for the result. Supported operation types: forward, forwardBackward, optimizerStep, saveWeights, saveState, and loadState.

Parameter Syntax

$result = $client->executeTrainingOperation([
    'clientToken' => '<string>',
    'operationConfig' => [ // REQUIRED
        'forward' => [
            'datumList' => [ // REQUIRED
                [
                    'lossFnInputs' => [
                        '<String>' => [
                            'data' => [ // REQUIRED
                            ],
                            'dtype' => '<string>', // REQUIRED
                            'shape' => [<integer>, ...],
                        ],
                        // ...
                    ],
                    'modelInput' => [ // REQUIRED
                        'chunks' => [ // REQUIRED
                            [
                                'tokens' => [<integer>, ...], // REQUIRED
                                'type' => 'encoded_text', // REQUIRED
                            ],
                            // ...
                        ],
                    ],
                ],
                // ...
            ],
            'lossFnConfig' => [
            ],
            'lossFnName' => 'cross_entropy|importance_sampling|ppo|cispo', // REQUIRED
        ],
        'forwardBackward' => [
            'datumList' => [ // REQUIRED
                [
                    'lossFnInputs' => [
                        '<String>' => [
                            'data' => [ // REQUIRED
                            ],
                            'dtype' => '<string>', // REQUIRED
                            'shape' => [<integer>, ...],
                        ],
                        // ...
                    ],
                    'modelInput' => [ // REQUIRED
                        'chunks' => [ // REQUIRED
                            [
                                'tokens' => [<integer>, ...], // REQUIRED
                                'type' => 'encoded_text', // REQUIRED
                            ],
                            // ...
                        ],
                    ],
                ],
                // ...
            ],
            'lossFnConfig' => [
            ],
            'lossFnName' => 'cross_entropy|importance_sampling|ppo|cispo', // REQUIRED
        ],
        'loadState' => [
            'loadOptimizer' => true || false, // REQUIRED
            'modelPackageArn' => '<string>', // REQUIRED
        ],
        'optimizerStep' => [
            'adamParamsConfig' => [ // REQUIRED
                'beta1' => <float>,
                'beta2' => <float>,
                'eps' => <float>,
                'gradClipNorm' => <float>,
                'learningRate' => <float>, // REQUIRED
                'loraPlusLrRatio' => <float>,
                'weightDecay' => <float>,
            ],
        ],
        'saveState' => [
            'description' => '<string>',
        ],
        'saveWeights' => [
            'description' => '<string>',
            'mergeWeights' => true || false,
        ],
    ],
    'sequenceNumber' => <integer>, // REQUIRED
    'trainingSessionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

clientToken

: Type: string

A unique, case-sensitive identifier that you provide to ensure that the operation completes no more than one time. If you retry a request with the same client token and the same parameters, the service ignores the request but does not return an error.

operationConfig

: Required: Yes
: Type: OperationConfig structure

The configuration that specifies the type and parameters of the training operation. Exactly one member of the union must be set.

sequenceNumber

: Required: Yes
: Type: int

The monotonically increasing sequence number that determines execution order. The training session processes operations strictly in sequence number order. If a gap exists, the session blocks until the missing operation arrives or times out. Must be at least 1.

trainingSessionName

: Required: Yes
: Type: string

The name of the training session to submit the operation to.

Result Syntax

[
    'operationId' => '<string>',
]

Result Details

Members

operationId

: Required: Yes
: Type: string

The unique identifier for the submitted operation. Use this identifier with GetTrainingOperationResult to retrieve the operation status and result.

Errors

ConflictException:: The training session is in a state that cannot accept new operations. The session must be in Running or Blocked state to accept requests.
ThrottlingException:: The request was throttled. Retry the request after a brief wait using exponential backoff.
ResourceNotFoundException:: The specified training session or operation does not exist, or the operation does not belong to the specified training session.
InternalServerException:: An internal service error occurred. Retry the request using exponential backoff.
ValidationException:: The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Examples

Example 1: Run a forward and backward pass

Computes gradients for a batch of tokenized prompts and accumulates them into the session. Operations are applied in `sequenceNumber` order, so each call must supply the next number in the sequence.

$result = $client->executeTrainingOperation([
    'operationConfig' => [
        'forwardBackward' => [
            'datumList' => [
                [
                    'modelInput' => [
                        'chunks' => [
                            [
                                'type' => 'encoded_text',
                                'tokens' => [
                                    791,
                                    6864,
                                    315,
                                    9822,
                                    374,
                                    12366,
                                ],
                            ],
                        ],
                    ],
                ],
            ],
            'lossFnName' => 'cross_entropy',
        ],
    ],
    'sequenceNumber' => 1,
    'trainingSessionName' => 'my-training-session',
]);

Result syntax:

[
    'operationId' => 'a1b2c3d4-5678-90ab-cdef-1234567890ab',
]

GetSamplingOperationResult

$result = $client->getSamplingOperationResult([/* ... */]);
$promise = $client->getSamplingOperationResultAsync([/* ... */]);

Retrieves the current status and result of a previously submitted sampling operation. Poll this operation until the status reaches a terminal state (Completed or Failed).

Parameter Syntax

$result = $client->getSamplingOperationResult([
    'operationId' => '<string>', // REQUIRED
    'trainingSessionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

operationId

: Required: Yes
: Type: string

The unique identifier of the sample operation to retrieve, returned by ExecuteSamplingOperation.

trainingSessionName

: Required: Yes
: Type: string

The name of the training session that owns the sample operation.

Result Syntax

[
    'errorDetails' => '<string>',
    'operationResult' => [
        'computeLogprobs' => [
            'logprobs' => [<float>, ...],
        ],
        'loadWeightsForSampling' => [
        ],
        'sample' => [
            'promptLogprobs' => [<float>, ...],
            'sequences' => [
                [
                    'logprobs' => [<float>, ...],
                    'stopReason' => 'stop|length',
                    'tokenUsage' => [
                        'decode' => <integer>,
                        'prefill' => <integer>,
                    ],
                    'tokens' => [<integer>, ...],
                ],
                // ...
            ],
            'topkPromptLogprobs' => [
                [
                    [
                        'logprob' => <float>,
                        'tokenId' => <integer>,
                    ],
                    // ...
                ],
                // ...
            ],
        ],
    ],
    'status' => 'Pending|InProgress|Completed|Failed|Skipped',
]

Result Details

Members

errorDetails

: Type: string

A message describing why the sample operation failed. The service populates this field only when the status is Failed.

operationResult

: Type: SampleOperationResult structure

The result payload of the sample operation. The service populates this field only when the status is Completed.

status

: Required: Yes
: Type: string

The current status of the sample operation. Possible values are Pending, InProgress, Completed, or Failed.

Errors

ResourceNotFoundException:: The specified training session or operation does not exist, or the operation does not belong to the specified training session.
ThrottlingException:: The request was throttled. Retry the request after a brief wait using exponential backoff.
InternalServerException:: An internal service error occurred. Retry the request using exponential backoff.
ValidationException:: The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Examples

Example 1: Get the result of a sampling operation

Retrieves the status and generated sequences for a sampling operation submitted with ExecuteSamplingOperation.

$result = $client->getSamplingOperationResult([
    'operationId' => 'a1b2c3d4-5678-90ab-cdef-1234567890ab',
    'trainingSessionName' => 'my-training-session',
]);

Result syntax:

[
    'operationResult' => [
        'sample' => [
            'sequences' => [
                [
                    'stopReason' => 'stop',
                    'tokenUsage' => [
                        'decode' => 7,
                        'prefill' => 5,
                    ],
                    'tokens' => [
                        791,
                        6864,
                        315,
                        9822,
                        374,
                        12366,
                        13,
                    ],
                ],
            ],
        ],
    ],
    'status' => 'Completed',
]

GetTrainingOperationResult

$result = $client->getTrainingOperationResult([/* ... */]);
$promise = $client->getTrainingOperationResultAsync([/* ... */]);

Retrieves the current status and result of a previously submitted training operation. Poll this operation until the status reaches a terminal state (Completed, Failed, or Skipped).

Parameter Syntax

$result = $client->getTrainingOperationResult([
    'operationId' => '<string>', // REQUIRED
    'trainingSessionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

operationId

: Required: Yes
: Type: string

The unique identifier of the operation to retrieve, returned by ExecuteTrainingOperation.

trainingSessionName

: Required: Yes
: Type: string

The name of the training session that owns the operation.

Result Syntax

[
    'errorDetails' => '<string>',
    'operationResult' => [
        'forward' => [
            'lossFnOutputType' => '<string>',
            'lossFnOutputs' => [
                [
                    '<String>' => [
                        'data' => [
                        ],
                        'dtype' => '<string>',
                        'shape' => [<integer>, ...],
                    ],
                    // ...
                ],
                // ...
            ],
            'metrics' => [<float>, ...],
            'tokenUsage' => [
                'trainingTokens' => <integer>,
            ],
        ],
        'forwardBackward' => [
            'lossFnOutputType' => '<string>',
            'lossFnOutputs' => [
                [
                    '<String>' => [
                        'data' => [
                        ],
                        'dtype' => '<string>',
                        'shape' => [<integer>, ...],
                    ],
                    // ...
                ],
                // ...
            ],
            'metrics' => [<float>, ...],
            'tokenUsage' => [
                'trainingTokens' => <integer>,
            ],
        ],
        'loadState' => [
        ],
        'optimizerStep' => [
            'metrics' => [<float>, ...],
        ],
        'saveState' => [
            'modelPackageArn' => '<string>',
        ],
        'saveWeights' => [
            'modelPackageArn' => '<string>',
        ],
    ],
    'status' => 'Pending|InProgress|Completed|Failed|Skipped',
    'tokenUsage' => [
        'trainingTokens' => <integer>,
    ],
]

Result Details

Members

errorDetails

: Type: string

A message describing why the operation failed. The service populates this field only when the status is Failed.

operationResult

: Type: OperationResult structure

The result payload of the operation. The service populates this field only when the status is Completed. The populated member of the union corresponds to the operation type that was submitted.

status

: Required: Yes
: Type: string

The current status of the operation. Possible values are Pending, InProgress, Completed, Failed, or Skipped.

tokenUsage

: Type: TrainingTokenUsage structure

The token usage statistics for metering purposes. The service populates this field for forward and forwardBackward operations upon completion.

Errors

ResourceNotFoundException:: The specified training session or operation does not exist, or the operation does not belong to the specified training session.
ThrottlingException:: The request was throttled. Retry the request after a brief wait using exponential backoff.
InternalServerException:: An internal service error occurred. Retry the request using exponential backoff.
ValidationException:: The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Examples

Example 1: Get the result of a training operation

Retrieves the status and result of a training operation submitted with ExecuteTrainingOperation.

$result = $client->getTrainingOperationResult([
    'operationId' => 'a1b2c3d4-5678-90ab-cdef-1234567890ab',
    'trainingSessionName' => 'my-training-session',
]);

Result syntax:

[
    'operationResult' => [
        'forwardBackward' => [
            'lossFnOutputType' => 'logprobs',
            'metrics' => [
                'loss' => 1.8342,
            ],
        ],
    ],
    'status' => 'Completed',
    'tokenUsage' => [
        'trainingTokens' => 4096,
    ],
]

SampleTrainingSession

$result = $client->sampleTrainingSession([/* ... */]);
$promise = $client->sampleTrainingSessionAsync([/* ... */]);

Sends an inference request to the model associated with the specified training session and returns the response synchronously. The service passes the request and response bodies to and from the model without modification.

Parameter Syntax

$result = $client->sampleTrainingSession([
    'accept' => '<string>',
    'body' => <string || resource || Psr\Http\Message\StreamInterface>,
    'contentType' => '<string>',
    'modelPackageArn' => '<string>',
    'trainingSessionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

accept

: Type: string

The desired MIME type of the inference body in the response.

body

: Type: blob (string|resource|Psr\Http\Message\StreamInterface)

The raw inference request body.

contentType

: Type: string

The MIME type of the input data in the request body.

modelPackageArn

: Type: string

The ARN of the model package whose checkpoint to invoke.

trainingSessionName

: Required: Yes
: Type: string

The name of the training session whose model to invoke.

Result Syntax

[
    'body' => <string || resource || Psr\Http\Message\StreamInterface>,
    'contentType' => '<string>',
]

Result Details

Members

body

: Required: Yes
: Type: blob (string|resource|Psr\Http\Message\StreamInterface)

The raw inference response body from the model.

contentType

: Type: string

The MIME type of the inference result in the response body.

Errors

ConflictException:: The training session is in a state that cannot accept new operations. The session must be in Running or Blocked state to accept requests.
ResourceNotFoundException:: The specified training session or operation does not exist, or the operation does not belong to the specified training session.
ThrottlingException:: The request was throttled. Retry the request after a brief wait using exponential backoff.
InternalServerException:: An internal service error occurred. Retry the request using exponential backoff.
ValidationException:: The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Examples

Example 1: Sample from a training session

Sends a chat completion request to the session's current weights and receives the complete response in a single payload.

$result = $client->sampleTrainingSession([
    'accept' => 'application/json',
    'body' => <BLOB>,
    'contentType' => 'application/json',
    'trainingSessionName' => 'my-training-session',
]);

Result syntax:

[
    'body' => <BLOB>,
    'contentType' => 'application/json',
]

SampleTrainingSessionWithResponseStream

$result = $client->sampleTrainingSessionWithResponseStream([/* ... */]);
$promise = $client->sampleTrainingSessionWithResponseStreamAsync([/* ... */]);

Streams inference responses from the model associated with the specified training session. Use this operation for real-time token-by-token generation. The service passes the request body to the model without modification.

Parameter Syntax

$result = $client->sampleTrainingSessionWithResponseStream([
    'accept' => '<string>',
    'body' => <string || resource || Psr\Http\Message\StreamInterface>,
    'contentType' => '<string>',
    'modelPackageArn' => '<string>',
    'trainingSessionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

accept

: Type: string

The desired MIME type of the inference body in the response.

body

: Type: blob (string|resource|Psr\Http\Message\StreamInterface)

The raw inference request body.

contentType

: Type: string

The MIME type of the input data in the request body.

modelPackageArn

: Type: string

The ARN of the model package whose checkpoint to invoke.

trainingSessionName

: Required: Yes
: Type: string

The name of the training session whose model to invoke.

Result Syntax

[
    'body' => <string || resource || Psr\Http\Message\StreamInterface>,
    'contentType' => '<string>',
]

Result Details

Members

body

: Required: Yes
: Type: blob (string|resource|Psr\Http\Message\StreamInterface)

The streaming inference response from the model, delivered as a series of chunks.

contentType

: Type: string

The MIME type of the streaming inference result.

Errors

ConflictException:: The training session is in a state that cannot accept new operations. The session must be in Running or Blocked state to accept requests.
ResourceNotFoundException:: The specified training session or operation does not exist, or the operation does not belong to the specified training session.
ThrottlingException:: The request was throttled. Retry the request after a brief wait using exponential backoff.
InternalServerException:: An internal service error occurred. Retry the request using exponential backoff.
ValidationException:: The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Examples

Example 1: Stream samples from a training session

Sends a chat completion request to the session's current weights and receives the response as a stream of payload chunks.

$result = $client->sampleTrainingSessionWithResponseStream([
    'accept' => 'application/json',
    'body' => <BLOB>,
    'contentType' => 'application/json',
    'trainingSessionName' => 'my-training-session',
]);

Shapes

AdamParamsConfig

Description

The Adam optimizer hyperparameters used during an optimizer step.

Members

beta1

: Type: double

The exponential decay rate for the first moment estimates.

beta2

: Type: double

The exponential decay rate for the second moment estimates.

eps

: Type: double

A small constant added to the denominator for numerical stability.

gradClipNorm

: Type: double

The maximum gradient norm for gradient clipping. Gradients are scaled down if their norm exceeds this value.

learningRate

: Required: Yes
: Type: double

The learning rate for the optimizer. Must be greater than 0.

loraPlusLrRatio

: Type: double

The learning rate multiplier for LoRA+ B matrices relative to A matrices. When set, B matrices use learningRate * loraPlusLrRatio while A matrices use the base learning rate.

weightDecay

: Type: double

The weight decay (L2 regularization) coefficient.

ComputeLogprobsConfig

Description

The configuration for computing log-probabilities of prompt tokens without generating new text.

Members

modelPackageArn

: Type: string

The ARN of the model package whose checkpoint this operation should run against. Must be a valid SageMaker model package ARN in the format arn:aws:sagemaker:REGION:ACCOUNT:model-package/PACKAGE_NAME/VERSION. When absent, the operation runs against the session's latest checkpoint.

prompt

: Required: Yes
: Type: ModelInput structure

The input prompt whose token log-probabilities will be computed.

ComputeLogprobsResult

Description

The result of a compute-logprobs operation.

Members

logprobs

: Required: Yes
: Type: Array of doubles

The per-token log-probabilities for the input prompt.

ConflictException

Description

The training session is in a state that cannot accept new operations. The session must be in Running or Blocked state to accept requests.

Members

message

: Required: Yes
: Type: string

Datum

Description

A single training datum containing model input and optional loss function inputs.

Members

lossFnInputs

: Type: Associative array of custom strings keys (String) to TensorData structures

The tensor inputs for the loss function, keyed by name. Unsupported keys are rejected. Supported keys: target_tokens, weights, advantages, logprobs, ref_log_prob, temperature. Required keys depend on the loss function type: SFT (cross_entropy) requires target_tokens and weights; RL (ppo, cispo, importance_sampling) requires target_tokens, weights, advantages, and logprobs.

modelInput

: Required: Yes
: Type: ModelInput structure

The tokenized input to feed to the model.

Document

Members

EncodedTextChunk

Description

A chunk of pre-tokenized text represented as a list of token IDs.

Members

tokens

: Required: Yes
: Type: Array of ints

The token IDs representing the encoded text.

type

: Required: Yes
: Type: string

The type of the chunk encoding.

ForwardBackwardConfig

Description

The configuration for a forward and backward pass through the model. This computes both loss values and gradients for use in subsequent optimizer steps.

Members

datumList

: Required: Yes
: Type: Array of Datum structures

The list of input data to feed through the model. Must contain at least one datum, and each datum must have a non-empty modelInput with at least one chunk.

lossFnConfig

: Type: document (null|bool|string|numeric) or an (array|associative array) whose members are all valid documents

Additional loss function configuration passed to the engine. Supported keys are clip_low_threshold (valid range: 0 to 1) and clip_high_threshold (valid range: 1 to 10). Unsupported keys are rejected.

lossFnName

: Required: Yes
: Type: string

The name of the loss function to evaluate during the forward-backward pass.

ForwardBackwardResult

Description

The result of a forward and backward pass through the model, including computed gradients.

Members

lossFnOutputType

: Type: string

The data type of the loss function output tensors.

lossFnOutputs

: Type: Array of TensorData structuress

The loss function output tensors, keyed by name.

metrics

: Type: Associative array of custom strings keys (String) to doubles

The training metrics emitted by the engine, such as loss values.

tokenUsage

: Type: TrainingTokenUsage structure

The token usage statistics for metering.

ForwardConfig

Description

The configuration for a forward-only pass through the model. This computes loss values without computing gradients.

Members

datumList

: Required: Yes
: Type: Array of Datum structures

The list of input data to feed through the model. Must contain at least one datum, and each datum must have a non-empty modelInput with at least one chunk.

lossFnConfig

: Type: document (null|bool|string|numeric) or an (array|associative array) whose members are all valid documents

lossFnName

: Required: Yes
: Type: string

The name of the loss function to evaluate during the forward pass.

ForwardResult

Description

The result of a forward-only pass through the model.

Members

lossFnOutputType

: Type: string

The data type of the loss function output tensors.

lossFnOutputs

: Type: Array of TensorData structuress

The loss function output tensors, keyed by name.

metrics

: Type: Associative array of custom strings keys (String) to doubles

The training metrics emitted by the engine, such as loss values.

tokenUsage

: Type: TrainingTokenUsage structure

The token usage statistics for metering.

InternalServerException

Description

An internal service error occurred. Retry the request using exponential backoff.

Members

message

: Required: Yes
: Type: string

LoadStateConfig

Description

The configuration for loading training state from a previously saved checkpoint. This restores model weights and optionally optimizer state.

Members

loadOptimizer

: Required: Yes
: Type: boolean

Specifies whether to also restore the optimizer state alongside model weights. When true, optimizer state is restored to resume training from where it left off.

modelPackageArn

: Required: Yes
: Type: string

The ARN of the model package containing the checkpoint to load. Must be a valid SageMaker model package ARN in the format arn:aws:sagemaker:REGION:ACCOUNT:model-package/PACKAGE_NAME/VERSION. Model package group ARNs are not accepted.

LoadStateResult

Description

The result of loading training state. An empty response indicates success.

Members

LoadWeightsForSamplingConfig

Description

The configuration for loading model weights for inference and sampling without restoring optimizer state.

Members

modelPackageArn

: Required: Yes
: Type: string

The ARN of the model package containing the weights to load for sampling. Must be a valid SageMaker model package ARN in the format arn:aws:sagemaker:REGION:ACCOUNT:model-package/PACKAGE_NAME/VERSION. The model package must belong to the same AWS account as the training session.

LoadWeightsForSamplingResult

Description

The result of loading weights for sampling. An empty response indicates success.

Members

ModelInput

Description

The tokenized model input composed of ordered encoded text chunks.

Members

chunks

: Required: Yes
: Type: Array of EncodedTextChunk structures

The ordered list of encoded text chunks forming the input sequence.

OperationConfig

Description

The configuration for a training operation. Exactly one member must be set to specify the operation type and its parameters.

Members

forward

: Type: ForwardConfig structure

The configuration for a forward-only pass through the model without gradient computation.

forwardBackward

: Type: ForwardBackwardConfig structure

The configuration for a forward and backward pass that computes gradients.

loadState

: Type: LoadStateConfig structure

The configuration for loading training state from a previously saved checkpoint.

optimizerStep

: Type: OptimizerStepConfig structure

The configuration for an optimizer step that updates model weights using accumulated gradients.

saveState

: Type: SaveStateConfig structure

The configuration for saving full training state, including model weights and optimizer state.

saveWeights

: Type: SaveWeightsConfig structure

The configuration for saving model weights as a checkpoint.

OperationResult

Description

The result of a completed training operation. Exactly one member is populated, corresponding to the operation type that was submitted.

Members

forward

: Type: ForwardResult structure

The result from a forward-only pass.

forwardBackward

: Type: ForwardBackwardResult structure

The result from a forward and backward pass.

loadState

: Type: LoadStateResult structure

The result from loading training state.

optimizerStep

: Type: OptimizerStepResult structure

The result from an optimizer step.

saveState

: Type: SaveStateResult structure

The result from saving full training state.

saveWeights

: Type: SaveWeightsResult structure

The result from saving model weights.

OptimizerStepConfig

Description

The configuration for an optimizer step that updates model weights using gradients accumulated from previous forward-backward passes.

Members

adamParamsConfig

: Required: Yes
: Type: AdamParamsConfig structure

The Adam optimizer hyperparameters for this step.

OptimizerStepResult

Description

The result of an optimizer step.

Members

metrics

: Type: Associative array of custom strings keys (String) to doubles

The training metrics emitted by the engine after the optimizer step.

ResourceNotFoundException

Description

The specified training session or operation does not exist, or the operation does not belong to the specified training session.

Members

message

: Required: Yes
: Type: string

SampleConfig

Description

The configuration for generating text samples from the model.

Members

includePromptLogprobs

: Type: boolean

Specifies whether to include per-token log-probabilities for the input prompt in the response.

modelPackageArn

: Type: string

The ARN of the model package whose checkpoint this sample should run against. Must be a valid SageMaker model package ARN in the format arn:aws:sagemaker:REGION:ACCOUNT:model-package/PACKAGE_NAME/VERSION. When absent, the sample runs against the session's latest checkpoint.

numSamples

: Required: Yes
: Type: int

The number of independent samples to generate for the prompt.

prompt

: Required: Yes
: Type: ModelInput structure

The input prompt to send to the model for text generation.

samplingParams

: Required: Yes
: Type: SamplingParams structure

The parameters controlling the sampling strategy.

topkPromptLogprobs

: Type: int

The number of top-k log-probabilities to return per prompt token.

SampleOperationConfig

Description

The configuration for a sampling operation. Exactly one member must be set to specify the operation type.

Members

computeLogprobs

: Type: ComputeLogprobsConfig structure

The configuration for computing log-probabilities of prompt tokens without generating new text.

loadWeightsForSampling

: Type: LoadWeightsForSamplingConfig structure

The configuration for loading model weights for inference and sampling without restoring optimizer state.

sample

: Type: SampleConfig structure

The configuration for generating text samples from the model.

SampleOperationResult

Description

The result of a completed sampling operation. Exactly one member is populated, corresponding to the operation type that was submitted.

Members

computeLogprobs

: Type: ComputeLogprobsResult structure

The computed log-probabilities for the prompt tokens.

loadWeightsForSampling

: Type: LoadWeightsForSamplingResult structure

The result from loading weights for sampling.

sample

: Type: SampleResponse structure

The generated text samples from the model.

SampleResponse

Description

The response containing generated samples and optional log-probabilities.

Members

promptLogprobs

: Type: Array of doubles

The per-token log-probabilities for the input prompt. Populated only when includePromptLogprobs is true.

sequences

: Required: Yes
: Type: Array of SampledSequence structures

The list of generated sequences.

topkPromptLogprobs

: Type: Array of TokenLogprob structuress

The top-k log-probabilities per prompt token, as a list of lists. Populated only when topkPromptLogprobs is greater than 0.

SampleTokenUsage

Description

The token usage statistics for a sample generation.

Members

decode

: Required: Yes
: Type: int

The number of tokens generated (decode phase).

prefill

: Required: Yes
: Type: int

The number of tokens in the input prompt (prefill phase).

SampledSequence

Description

A single generated sequence from the model.

Members

logprobs

: Type: Array of doubles

The per-token log-probabilities for the generated tokens.

stopReason

: Required: Yes
: Type: string

The reason why generation stopped. A value of stop indicates a stop sequence was encountered. A value of length indicates the maximum token limit was reached.

tokenUsage

: Type: SampleTokenUsage structure

The token usage statistics for this individual sample.

tokens

: Required: Yes
: Type: Array of ints

The token IDs of the generated sequence.

SamplingParams

Description

The parameters controlling the text generation sampling strategy.

Members

maxTokens

: Required: Yes
: Type: int

The maximum number of tokens to generate per sample.

seed

: Type: int

The random seed for reproducible sampling. When set, the same seed with the same input produces deterministic outputs.

stop

: Type: Array of strings

The list of stop sequences that terminate generation when encountered in the output.

temperature

: Type: double

The sampling temperature controlling randomness. Higher values produce more diverse outputs.

topK

: Type: int

The number of highest-probability vocabulary tokens to keep for top-k filtering. Set to -1 to disable.

topP

: Type: double

The cumulative probability threshold for nucleus (top-p) sampling. Only tokens with cumulative probability below this threshold are considered.

SaveStateConfig

Description

The configuration for saving full training state, including model weights and optimizer state. Use this to create resumable checkpoints.

Members

description

: Type: string

A human-readable description for the saved state checkpoint.

SaveStateResult

Description

The result of saving full training state.

Members

modelPackageArn

: Required: Yes
: Type: string

The ARN of the model package created for the saved state checkpoint.

SaveWeightsConfig

Description

The configuration for saving model weights as a checkpoint.

Members

description

: Type: string

A human-readable description for the saved checkpoint.

mergeWeights

: Type: boolean

Specifies whether to merge LoRA adapter weights into the base model before saving. When true, the saved checkpoint contains the full merged model weights.

SaveWeightsResult

Description

The result of saving model weights.

Members

modelPackageArn

: Type: string

The ARN of the model package created for the saved weights.

TensorData

Description

A tensor with data, data type, and shape metadata.

Members

data

: Required: Yes
: Type: document (null|bool|string|numeric) or an (array|associative array) whose members are all valid documents

The raw tensor data as JSON. The format is determined by the dtype field: integer arrays for int64, floating-point arrays for float32.

dtype

: Required: Yes
: Type: string

The data type of the tensor elements, such as int64 or float32.

shape

: Type: Array of ints

The dimensions of the tensor, such as [batch_size, seq_length].

ThrottlingException

Description

The request was throttled. Retry the request after a brief wait using exponential backoff.

Members

message

: Required: Yes
: Type: string

TokenLogprob

Description

The log-probability for a single token.

Members

logprob

: Required: Yes
: Type: double

The log-probability of the token.

tokenId

: Required: Yes
: Type: int

The vocabulary token ID.

TrainingTokenUsage

Description

The token usage statistics for training operations.

Members

trainingTokens

: Type: int

The number of training tokens consumed by the operation.

ValidationException

Description

The request is not valid. Check the request syntax and parameters. Examples include invalid sequence numbers, malformed client tokens, missing required fields, or invalid operation configuration.

Members

message

: Required: Yes
: Type: string

SageMaker Training Session Runtime 2025-02-10

Operation Summary

Operations

ExecuteSamplingOperation

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

Examples

Example 1: Generate samples from a prompt

ExecuteTrainingOperation

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

Examples

Example 1: Run a forward and backward pass

GetSamplingOperationResult

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

Examples

Example 1: Get the result of a sampling operation

GetTrainingOperationResult

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

Examples

Example 1: Get the result of a training operation

SampleTrainingSession

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

Examples

Example 1: Sample from a training session

SampleTrainingSessionWithResponseStream

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

Examples

Example 1: Stream samples from a training session

Shapes

AdamParamsConfig

Description

Members

ComputeLogprobsConfig

Description

Members

ComputeLogprobsResult

Description

Members

ConflictException

Description

Members

Datum

Description

Members

Document