Package software.amazon.awscdk.services.glue

Class ScalaSparkStreamingJob

java.lang.Object
software.amazon.jsii.JsiiObject
software.constructs.Construct
software.amazon.awscdk.Resource
software.amazon.awscdk.services.glue.JobBase
software.amazon.awscdk.services.glue.Job
software.amazon.awscdk.services.glue.SparkJob
software.amazon.awscdk.services.glue.ScalaSparkStreamingJob
All Implemented Interfaces:
IJobRef, IEnvironmentAware, IResource, IJob, IGrantable, software.amazon.jsii.JsiiSerializable, software.constructs.IConstruct, software.constructs.IDependable
@Generated(value="jsii-pacmak/1.139.0 (build 26a6b54)", date="2026-09-17T14:54:05.092Z") @Stability(Stable) public class ScalaSparkStreamingJob extends SparkJob
Scala Streaming Jobs class.

A Streaming job is similar to an ETL job, except that it performs ETL on data streams using the Apache Spark Structured Streaming framework. This job runs on the Scala runtime.

Similar to ETL jobs, streaming job supports Scala and Python languages. Streaming jobs default to the G_1X worker type and Glue version 4.0, both of which developers can override with any supported WorkerType or GlueVersion. We will enable --enable-metrics, --enable-continuous-cloudwatch-log. The Spark UI (--enable-spark-ui) is off by default; enable it by setting the sparkUI prop.

Example: 

 // The code below shows an example of how to instantiate this type.
 // The values are placeholders you should change.
 import software.amazon.awscdk.*;
 import software.amazon.awscdk.services.glue.*;
 import software.amazon.awscdk.services.iam.*;
 import software.amazon.awscdk.services.logs.*;
 import software.amazon.awscdk.services.s3.*;
 Bucket bucket;
 Code code;
 Connection connection;
 LogGroup logGroup;
 Role role;
 SecurityConfiguration securityConfiguration;
 ScalaSparkStreamingJob scalaSparkStreamingJob = ScalaSparkStreamingJob.Builder.create(this, "MyScalaSparkStreamingJob")
         .className("className")
         .role(role)
         .script(code)
         // the properties below are optional
         .connections(List.of(connection))
         .continuousLogging(ContinuousLoggingProps.builder()
                 .enabled(false)
                 // the properties below are optional
                 .conversionPattern("conversionPattern")
                 .logGroup(logGroup)
                 .logStreamPrefix("logStreamPrefix")
                 .quiet(false)
                 .build())
         .defaultArguments(Map.of(
                 "defaultArgumentsKey", "defaultArguments"))
         .description("description")
         .enableMetrics(false)
         .enableObservabilityMetrics(false)
         .extraFiles(List.of(code))
         .extraJars(List.of(code))
         .extraJarsFirst(false)
         .glueVersion(GlueVersion.V0_9)
         .jobName("jobName")
         .jobRunQueuingEnabled(false)
         .maxConcurrentRuns(123)
         .maxRetries(123)
         .securityConfiguration(securityConfiguration)
         .sparkUI(SparkUIProps.builder()
                 .bucket(bucket)
                 .prefix("prefix")
                 .build())
         .tags(Map.of(
                 "tagsKey", "tags"))
         .timeout(Duration.minutes(30))
         .workerConfiguration(WorkerConfiguration.builder()
                 .numberOfWorkers(123)
                 .workerType(WorkerType.STANDARD)
                 .build())
         .build();

  • Field Details

    • PROPERTY_INJECTION_ID

      @Stability(Stable) public static final String PROPERTY_INJECTION_ID
      Uniquely identifies this class.

  • Constructor Details

    • ScalaSparkStreamingJob

      protected ScalaSparkStreamingJob(software.amazon.jsii.JsiiObjectRef objRef)

    • ScalaSparkStreamingJob

      protected ScalaSparkStreamingJob(software.amazon.jsii.JsiiObject.InitializationMode initializationMode)

    • ScalaSparkStreamingJob

      @Stability(Stable) public ScalaSparkStreamingJob(@NotNull software.constructs.Construct scope, @NotNull String id, @NotNull ScalaSparkStreamingJobProps props)
      ScalaSparkStreamingJob constructor.

      Parameters:
      scope - This parameter is required.
      id - This parameter is required.
      props - This parameter is required.

  • Method Details