Package software.amazon.awscdk.services.glue

Class PySparkEtlJobProps.Jsii$Proxy

java.lang.Object
software.amazon.jsii.JsiiObject
software.amazon.awscdk.services.glue.PySparkEtlJobProps.Jsii$Proxy
All Implemented Interfaces:
JobProps, PySparkEtlJobProps, SparkJobProps, software.amazon.jsii.JsiiSerializable
Enclosing interface:
PySparkEtlJobProps
@Stability(Stable) @Internal public static final class PySparkEtlJobProps.Jsii$Proxy extends software.amazon.jsii.JsiiObject implements PySparkEtlJobProps
An implementation for PySparkEtlJobProps

  • Nested Class Summary

    Nested classes/interfaces inherited from class software.amazon.jsii.JsiiObject

    software.amazon.jsii.JsiiObject.InitializationMode

    Nested classes/interfaces inherited from interface software.amazon.awscdk.services.glue.PySparkEtlJobProps

    PySparkEtlJobProps.Builder, PySparkEtlJobProps.Jsii$Proxy

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    Jsii$Proxy(PySparkEtlJobProps.Builder builder)
    Constructor that initializes the object based on literal property values passed by the PySparkEtlJobProps.Builder.
    protected
    Jsii$Proxy(software.amazon.jsii.JsiiObjectRef objRef)
    Constructor that initializes the object based on values retrieved from the JsiiObject.

  • Method Summary

    Modifier and Type
    Method
    Description
    com.fasterxml.jackson.databind.JsonNode
    $jsii$toJson()
     
    final boolean
    equals(Object o)
     
    final List<IConnection>
    getConnections()
    Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC.
    final ContinuousLoggingProps
    getContinuousLogging()
    Enables continuous logging with the specified props.
    final Map<String,String>
    getDefaultArguments()
    Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs.
    final String
    getDescription()
    Description (optional) Developer-specified description of the Glue job.
    final Boolean
    getEnableMetrics()
    Enable profiling metrics for the Glue job.
    final Boolean
    getEnableObservabilityMetrics()
    Enable observability metrics for the Glue job.
    final List<Code>
    getExtraFiles()
    Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
    final List<Code>
    getExtraJars()
    Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located.
    final Boolean
    getExtraJarsFirst()
    Setting this value to true prioritizes the customer's extra JAR files in the classpath.
    final List<Code>
    getExtraPythonFiles()
    Extra Python Files S3 URL (optional) S3 URL where additional python dependencies are located.
    final GlueVersion
    getGlueVersion()
    Glue Version The version of Glue to use to execute this job.
    final String
    getJobName()
    Name of the Glue job (optional) Developer-specified name of the Glue job.
    final Boolean
    getJobRunQueuingEnabled()
    Specifies whether job run queuing is enabled for the job runs for this job.
    final Number
    getMaxConcurrentRuns()
    Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run.
    final Number
    getMaxRetries()
    Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails.
    final Duration
    getNotifyDelayAfter()
    Specifies configuration properties of a notification (optional).
    final IRole
    getRole()
    IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.
    final Code
    getScript()
    Script Code Location (required) Script to run when the Glue job executes.
    final ISecurityConfiguration
    getSecurityConfiguration()
    Security Configuration (optional) Defines the encryption options for the Glue job.
    final SparkUIProps
    getSparkUI()
    Enables the Spark UI debugging and monitoring with the specified props.
    final Map<String,String>
    getTags()
    Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources.
    final Duration
    getTimeout()
    Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
    final WorkerConfiguration
    getWorkerConfiguration()
    The worker type and the number of workers allocated when a job runs.
    final int
    hashCode()
     

    Methods inherited from class software.amazon.jsii.JsiiObject

    jsiiAsyncCall, jsiiAsyncCall, jsiiCall, jsiiCall, jsiiGet, jsiiGet, jsiiSet, jsiiStaticCall, jsiiStaticCall, jsiiStaticGet, jsiiStaticGet, jsiiStaticSet, jsiiStaticSet

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • Jsii$Proxy

      protected Jsii$Proxy(software.amazon.jsii.JsiiObjectRef objRef)
      Constructor that initializes the object based on values retrieved from the JsiiObject.
      Parameters:
      objRef - Reference to the JSII managed object.

    • Jsii$Proxy

      protected Jsii$Proxy(PySparkEtlJobProps.Builder builder)
      Constructor that initializes the object based on literal property values passed by the PySparkEtlJobProps.Builder.

  • Method Details

    • getExtraFiles

      public final List<Code> getExtraFiles()
      Description copied from interface: PySparkEtlJobProps
      Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.

      Default: - no extra files specified.

      Specified by:
      getExtraFiles in interface PySparkEtlJobProps
      See Also:

    • getExtraJars

      public final List<Code> getExtraJars()
      Description copied from interface: PySparkEtlJobProps
      Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located.

      Default: - no extra jar files

      Specified by:
      getExtraJars in interface PySparkEtlJobProps

    • getExtraJarsFirst

      public final Boolean getExtraJarsFirst()
      Description copied from interface: PySparkEtlJobProps
      Setting this value to true prioritizes the customer's extra JAR files in the classpath.

      Default: false - priority is not given to user-provided jars

      Specified by:
      getExtraJarsFirst in interface PySparkEtlJobProps
      See Also:

    • getExtraPythonFiles

      public final List<Code> getExtraPythonFiles()
      Description copied from interface: PySparkEtlJobProps
      Extra Python Files S3 URL (optional) S3 URL where additional python dependencies are located.

      Default: - no extra files

      Specified by:
      getExtraPythonFiles in interface PySparkEtlJobProps

    • getJobRunQueuingEnabled

      public final Boolean getJobRunQueuingEnabled()
      Description copied from interface: PySparkEtlJobProps
      Specifies whether job run queuing is enabled for the job runs for this job.

      A value of true means job run queuing is enabled for the job runs. If false or not populated, the job runs will not be considered for queueing. If this field does not match the value set in the job run, then the value from the job run field will be used. This property must be set to false for flex jobs. If this property is enabled, maxRetries must be set to zero.

      Default: false

      Specified by:
      getJobRunQueuingEnabled in interface PySparkEtlJobProps

    • getNotifyDelayAfter

      public final Duration getNotifyDelayAfter()
      Description copied from interface: PySparkEtlJobProps
      Specifies configuration properties of a notification (optional).

      After a job run starts, the number of minutes to wait before sending a job run delay notification.

      Default: - undefined

      Specified by:
      getNotifyDelayAfter in interface PySparkEtlJobProps

    • getEnableMetrics

      public final Boolean getEnableMetrics()
      Description copied from interface: SparkJobProps
      Enable profiling metrics for the Glue job.

      When enabled, adds '--enable-metrics' to job arguments.

      Default: true

      Specified by:
      getEnableMetrics in interface SparkJobProps

    • getEnableObservabilityMetrics

      public final Boolean getEnableObservabilityMetrics()
      Description copied from interface: SparkJobProps
      Enable observability metrics for the Glue job.

      When enabled, adds '--enable-observability-metrics': 'true' to job arguments.

      Default: true

      Specified by:
      getEnableObservabilityMetrics in interface SparkJobProps

    • getSparkUI

      public final SparkUIProps getSparkUI()
      Description copied from interface: SparkJobProps
      Enables the Spark UI debugging and monitoring with the specified props.

      Default: - Spark UI debugging and monitoring is disabled.

      Specified by:
      getSparkUI in interface SparkJobProps
      See Also:

    • getWorkerConfiguration

      public final WorkerConfiguration getWorkerConfiguration()
      Description copied from interface: SparkJobProps
      The worker type and the number of workers allocated when a job runs.

      Default: - the job runs with the G_1X worker type and 10 workers.

      Specified by:
      getWorkerConfiguration in interface SparkJobProps

    • getRole

      public final IRole getRole()
      Description copied from interface: JobProps
      IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.

      Specified by:
      getRole in interface JobProps
      See Also:

    • getScript

      public final Code getScript()
      Description copied from interface: JobProps
      Script Code Location (required) Script to run when the Glue job executes.

      Can be uploaded from the local directory structure using fromAsset or referenced via S3 location using fromBucket

      Specified by:
      getScript in interface JobProps

    • getConnections

      public final List<IConnection> getConnections()
      Description copied from interface: JobProps
      Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC.

      Default: [] - no connections are added to the job

      Specified by:
      getConnections in interface JobProps

    • getContinuousLogging

      public final ContinuousLoggingProps getContinuousLogging()
      Description copied from interface: JobProps
      Enables continuous logging with the specified props.

      Default: - continuous logging is enabled.

      Specified by:
      getContinuousLogging in interface JobProps
      See Also:

    • getDefaultArguments

      public final Map<String,String> getDefaultArguments()
      Description copied from interface: JobProps
      Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs.

      This map is the escape hatch for Glue job arguments that this construct does not model. It MUST NOT be used to set arguments that already have a dedicated prop — configure those through the corresponding prop instead (continuousLogging, enableMetrics, enableObservabilityMetrics, sparkUI, className, extraJars, extraJarsFirst, extraPythonFiles, extraFiles). Passing a construct-managed argument (e.g. --enable-continuous-cloudwatch-log, --enable-metrics, --enable-spark-ui, --job-language) or a Glue-reserved argument (--debug, --mode, --JOB_NAME, --endpoint) here throws at synthesis time, so there is exactly one way to express each intent.

      Also note that these are emitted verbatim into the CloudFormation template, so avoid placing secrets here in plaintext. Pass secrets to the job at runtime through AWS Secrets Manager instead. A synthesis-time warning is emitted when an argument key looks like a credential and holds a plaintext literal.

      Default: - no arguments

      Specified by:
      getDefaultArguments in interface JobProps
      See Also:

    • getDescription

      public final String getDescription()
      Description copied from interface: JobProps
      Description (optional) Developer-specified description of the Glue job.

      Default: - no value

      Specified by:
      getDescription in interface JobProps

    • getGlueVersion

      public final GlueVersion getGlueVersion()
      Description copied from interface: JobProps
      Glue Version The version of Glue to use to execute this job.

      Default: - determined by the job type: 4.0 for ETL and Streaming, 5.0 for Flex, 3.0 for Python Shell

      Specified by:
      getGlueVersion in interface JobProps

    • getJobName

      public final String getJobName()
      Description copied from interface: JobProps
      Name of the Glue job (optional) Developer-specified name of the Glue job.

      Default: - a name is automatically generated

      Specified by:
      getJobName in interface JobProps

    • getMaxConcurrentRuns

      public final Number getMaxConcurrentRuns()
      Description copied from interface: JobProps
      Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run.

      An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.

      Default: 1

      Specified by:
      getMaxConcurrentRuns in interface JobProps

    • getMaxRetries

      public final Number getMaxRetries()
      Description copied from interface: JobProps
      Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails.

      Default: 0

      Specified by:
      getMaxRetries in interface JobProps

    • getSecurityConfiguration

      public final ISecurityConfiguration getSecurityConfiguration()
      Description copied from interface: JobProps
      Security Configuration (optional) Defines the encryption options for the Glue job.

      Default: - no security configuration.

      Specified by:
      getSecurityConfiguration in interface JobProps

    • getTags

      public final Map<String,String> getTags()
      Description copied from interface: JobProps
      Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources.

      Default: {} - no tags

      Specified by:
      getTags in interface JobProps

    • getTimeout

      public final Duration getTimeout()
      Description copied from interface: JobProps
      Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.

      Specified in minutes.

      Default: - no value set; Glue applies its service default (2880 minutes for non-streaming jobs)

      Specified by:
      getTimeout in interface JobProps

    • $jsii$toJson

      @Internal public com.fasterxml.jackson.databind.JsonNode $jsii$toJson()
      Specified by:
      $jsii$toJson in interface software.amazon.jsii.JsiiSerializable

    • equals

      public final boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public final int hashCode()
      Overrides:
      hashCode in class Object