Skip to content

Latest commit

 

History

History
3394 lines (2013 loc) · 96.7 KB

API.md

File metadata and controls

3394 lines (2013 loc) · 96.7 KB

API Reference

Structs

ApplyMappingField

Apply Mapping field.

Initializer 

import { ApplyMappingField } from 'cdk-glue-job-builder'

const applyMappingField: ApplyMappingField = { ... }

Properties

Name Type Description
source @aws-cdk/aws-glue-alpha.Column Source field.
target @aws-cdk/aws-glue-alpha.Column Target field.
sourceRequired 
public readonly source: Column;
  • Type: @aws-cdk/aws-glue-alpha.Column

Source field.

targetRequired 
public readonly target: Column;
  • Type: @aws-cdk/aws-glue-alpha.Column

Target field.

ApplyMappingProps

Props of ApplyMapping.

Initializer 

import { ApplyMappingProps } from 'cdk-glue-job-builder'

const applyMappingProps: ApplyMappingProps = { ... }

Properties

Name Type Description
mappings ApplyMappingField[] Field you want to select.
name string The node name.
mappingsRequired 
public readonly mappings: ApplyMappingField[];

Field you want to select.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'Apply Mapping'

The node name.

CatalogSourceProps

Props of CatalogSource.

Initializer 

import { CatalogSourceProps } from 'cdk-glue-job-builder'

const catalogSourceProps: CatalogSourceProps = { ... }

Properties

Name Type Description
database @aws-cdk/aws-glue-alpha.IDatabase The database to read from.
table @aws-cdk/aws-glue-alpha.ITable The table in the database to read from.
granter IGranter Granter the necessary authorization to a node.
name string The name of your data source.
databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase

The database to read from.

tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

The table in the database to read from.

granterOptional 
public readonly granter: IGranter;

Granter the necessary authorization to a node.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'AWS Glue Data Catalog'

The name of your data source.

CatalogTargetProps

Props of CatalogTarget.

Initializer 

import { CatalogTargetProps } from 'cdk-glue-job-builder'

const catalogTargetProps: CatalogTargetProps = { ... }

Properties

Name Type Description
database @aws-cdk/aws-glue-alpha.IDatabase The database that contains the table you want to use as the target.
table @aws-cdk/aws-glue-alpha.ITable The table that defines the schema of your output data.
granter IGranter Granter the necessary authorization to a node.
name string The name of the data target.
databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase

The database that contains the table you want to use as the target.

This database must already exist in the Data Catalog.

tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

The table that defines the schema of your output data.

This table must already exist in the Data Catalog.

granterOptional 
public readonly granter: IGranter;

Granter the necessary authorization to a node.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'AWS Glue Data Catalog'

The name of the data target.

CodeFragment

Fragment of code.

Initializer 

import { CodeFragment } from 'cdk-glue-job-builder'

const codeFragment: CodeFragment = { ... }

Properties

Name Type Description
body string[] The body is output between the head and tail sections.
head string[] Set the code to be written between the import section and body.
imports string[] Import section of CodeFragment.
tail string[] The tail is output just before job.commit().
bodyOptional 
public readonly body: string[];
  • Type: string[]
  • Default: No body codes.

The body is output between the head and tail sections.

The ETL is written here.

headOptional 
public readonly head: string[];
  • Type: string[]
  • Default: No head codes.

Set the code to be written between the import section and body.

Functions must be defined here.

importsOptional 
public readonly imports: string[];
  • Type: string[]
  • Default: No imports.

Import section of CodeFragment.

tailOptional 
public readonly tail: string[];
  • Type: string[]
  • Default: No tail codes.

The tail is output just before job.commit().

DropFieldsProps

Props of DropFields.

Initializer 

import { DropFieldsProps } from 'cdk-glue-job-builder'

const dropFieldsProps: DropFieldsProps = { ... }

Properties

Name Type Description
fields string[] Field you want to drop.
name string The node name.
fieldsRequired 
public readonly fields: string[];
  • Type: string[]

Field you want to drop.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'Drop Fields'

The node name.

DropNullFieldsProps

Props of DropNullFields.

Initializer 

import { DropNullFieldsProps } from 'cdk-glue-job-builder'

const dropNullFieldsProps: DropNullFieldsProps = { ... }

Properties

Name Type Description
fields RepresentNullValue[] Field you want to drop.
name string The node name.
fieldsRequired 
public readonly fields: RepresentNullValue[];

Field you want to drop.

Example

Transform.dropNullFields({
 fields: [
   RepresentNullValue.emptyString(),
   RepresentNullValue.nullString(),
 ]
})
nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'Drop Null Fields'

The node name.

S3CatalogSourceProps

Props of S3CatalogSource.

Initializer 

import { S3CatalogSourceProps } from 'cdk-glue-job-builder'

const s3CatalogSourceProps: S3CatalogSourceProps = { ... }

Properties

Name Type Description
bucket aws-cdk-lib.aws_s3.IBucket The bucket of table.
database @aws-cdk/aws-glue-alpha.IDatabase The database to read from.
table @aws-cdk/aws-glue-alpha.ITable The table in the database to read from.
name string The name of your data source.
bucketRequired 
public readonly bucket: IBucket;
  • Type: aws-cdk-lib.aws_s3.IBucket

The bucket of table.

databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase

The database to read from.

tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

The table in the database to read from.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'Amazon S3'

The name of your data source.

S3CatalogTargetProps

Props of S3CatalogTarget.

Initializer 

import { S3CatalogTargetProps } from 'cdk-glue-job-builder'

const s3CatalogTargetProps: S3CatalogTargetProps = { ... }

Properties

Name Type Description
bucket aws-cdk-lib.aws_s3.IBucket The bucket of table.
database @aws-cdk/aws-glue-alpha.IDatabase The database that contains the table you want to use as the target.
table @aws-cdk/aws-glue-alpha.ITable The table that defines the schema of your output data.
name string The name of your data source.
bucketRequired 
public readonly bucket: IBucket;
  • Type: aws-cdk-lib.aws_s3.IBucket

The bucket of table.

databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase

The database that contains the table you want to use as the target.

This database must already exist in the Data Catalog.

tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

The table that defines the schema of your output data.

This table must already exist in the Data Catalog.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'Amazon S3'

The name of your data source.

SelectFieldsProps

Props of SelectFields.

Initializer 

import { SelectFieldsProps } from 'cdk-glue-job-builder'

const selectFieldsProps: SelectFieldsProps = { ... }

Properties

Name Type Description
fields string[] Field you want to select.
name string The node name.
fieldsRequired 
public readonly fields: string[];
  • Type: string[]

Field you want to select.

nameOptional 
public readonly name: string;
  • Type: string
  • Default: 'Select Fields'

The node name.

Classes

ApplyMapping

Map fields to new names and types of your chice.

Initializers 

import { ApplyMapping } from 'cdk-glue-job-builder'

new ApplyMapping(id: string, props: ApplyMappingProps)
Name Type Description
id string No description.
props ApplyMappingProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(_job: IGrantable): Grant

Granting Glue job permissions.

_jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

CatalogSource

Specifies a data store in the Glue Data Catalog.

Initializers 

import { CatalogSource } from 'cdk-glue-job-builder'

new CatalogSource(id: string, props: CatalogSourceProps)
Name Type Description
id string No description.
props CatalogSourceProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
database @aws-cdk/aws-glue-alpha.IDatabase No description.
table @aws-cdk/aws-glue-alpha.ITable No description.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase
tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

CatalogTarget

Specifies a data target that writes to AWS Glue Data Catalog using the Glue Data Catalog.

Initializers 

import { CatalogTarget } from 'cdk-glue-job-builder'

new CatalogTarget(id: string, props: CatalogTargetProps)
Name Type Description
id string No description.
props CatalogTargetProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
database @aws-cdk/aws-glue-alpha.IDatabase No description.
table @aws-cdk/aws-glue-alpha.ITable No description.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase
tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

Codenizer

Initializers 

import { Codenizer } from 'cdk-glue-job-builder'

new Codenizer()
Name Type Description

Static Functions

Name Description
python No description.
scala No description.
python 
import { Codenizer } from 'cdk-glue-job-builder'

Codenizer.python(node: INode, bucket?: IBucket, objectKey?: string)
nodeRequired

The Node you want to build.

bucketOptional
  • Type: aws-cdk-lib.aws_s3.IBucket
  • Default: generate a new Bucket

Built code save S3 bucket.

objectKeyOptional
  • Type: string
  • Default: generate a object key

Built code save S3 object key.

scala 
import { Codenizer } from 'cdk-glue-job-builder'

Codenizer.scala(node: INode, bucket?: IBucket, objectKey?: string)
nodeRequired

The Node you want to build.

bucketOptional
  • Type: aws-cdk-lib.aws_s3.IBucket
  • Default: generate a new Bucket

Built code save S3 bucket.

objectKeyOptional
  • Type: string
  • Default: generate a object key

Built code save S3 object key.

CodenizerBase

Initializers 

import { CodenizerBase } from 'cdk-glue-job-builder'

new CodenizerBase(node: INode, bucket?: IBucket, objectKey?: string)
Name Type Description
node INode The Node you want to build.
bucket aws-cdk-lib.aws_s3.IBucket Built code save S3 bucket.
objectKey string Built code save S3 object key.
nodeRequired

The Node you want to build.

bucketOptional
  • Type: aws-cdk-lib.aws_s3.IBucket
  • Default: generate a new Bucket

Built code save S3 bucket.

objectKeyOptional
  • Type: string
  • Default: generate a object key

Built code save S3 object key.

Methods

Name Description
bind Called when the Job is initialized to allow this object to bind.
codenize No description.
bind 
public bind(scope: Construct, grantable: IGrantable): CodeConfig

Called when the Job is initialized to allow this object to bind.

scopeRequired
  • Type: constructs.Construct
grantableRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
codenize 
public codenize(): string

Static Functions

Name Description
fromAsset Job code from a local disk path.
fromBucket Job code as an S3 object.
fromAsset 
import { CodenizerBase } from 'cdk-glue-job-builder'

CodenizerBase.fromAsset(path: string, options?: AssetOptions)

Job code from a local disk path.

pathRequired
  • Type: string

code file (not a directory).

optionsOptional
  • Type: aws-cdk-lib.aws_s3_assets.AssetOptions
fromBucket 
import { CodenizerBase } from 'cdk-glue-job-builder'

CodenizerBase.fromBucket(bucket: IBucket, key: string)

Job code as an S3 object.

bucketRequired
  • Type: aws-cdk-lib.aws_s3.IBucket

The S3 bucket.

keyRequired
  • Type: string

The object key.

DataSource

Data sources.

Initializers 

import { DataSource } from 'cdk-glue-job-builder'

new DataSource()
Name Type Description

Static Functions

Name Description
catalog Create a new {@link CatalogSource}.
s3Catalog Create a new {@link S3CatalogSource}.
catalog 
import { DataSource } from 'cdk-glue-job-builder'

DataSource.catalog(id: string, props: CatalogSourceProps)

Create a new {@link CatalogSource}.

idRequired
  • Type: string
propsRequired
s3Catalog 
import { DataSource } from 'cdk-glue-job-builder'

DataSource.s3Catalog(id: string, props: S3CatalogSourceProps)

Create a new {@link S3CatalogSource}.

idRequired
  • Type: string
propsRequired

DataTarget

Data targets.

Initializers 

import { DataTarget } from 'cdk-glue-job-builder'

new DataTarget()
Name Type Description

Static Functions

Name Description
catalog Create a new {@link CatalogTarget}.
s3Catalog Create a new {@link S3CatalogTarget}.
catalog 
import { DataTarget } from 'cdk-glue-job-builder'

DataTarget.catalog(id: string, props: CatalogTargetProps)

Create a new {@link CatalogTarget}.

idRequired
  • Type: string
propsRequired
s3Catalog 
import { DataTarget } from 'cdk-glue-job-builder'

DataTarget.s3Catalog(id: string, props: S3CatalogTargetProps)

Create a new {@link S3CatalogTarget}.

idRequired
  • Type: string
propsRequired

DropFields

Remove selected fields from your data.

Initializers 

import { DropFields } from 'cdk-glue-job-builder'

new DropFields(id: string, props: DropFieldsProps)
Name Type Description
id string No description.
props DropFieldsProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(_job: IGrantable): Grant

Granting Glue job permissions.

_jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

DropNullFields

Remove fields or columns where all the values are the null objects.

Initializers 

import { DropNullFields } from 'cdk-glue-job-builder'

new DropNullFields(id: string, props: DropNullFieldsProps)
Name Type Description
id string No description.
props DropNullFieldsProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(_job: IGrantable): Grant

Granting Glue job permissions.

_jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

NodeBase

Base of node.

Initializers 

import { NodeBase } from 'cdk-glue-job-builder'

new NodeBase(id: string, name: string)
Name Type Description
id string No description.
name string No description.
idRequired
  • Type: string
nameRequired
  • Type: string

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

PythonCodenizer

Build Python code from nodes.

Initializers 

import { PythonCodenizer } from 'cdk-glue-job-builder'

new PythonCodenizer(node: INode, bucket?: IBucket, objectKey?: string)
Name Type Description
node INode The Node you want to build.
bucket aws-cdk-lib.aws_s3.IBucket Built code save S3 bucket.
objectKey string Built code save S3 object key.
nodeRequired

The Node you want to build.

bucketOptional
  • Type: aws-cdk-lib.aws_s3.IBucket
  • Default: generate a new Bucket

Built code save S3 bucket.

objectKeyOptional
  • Type: string
  • Default: generate a object key

Built code save S3 object key.

Methods

Name Description
bind Called when the Job is initialized to allow this object to bind.
codenize No description.
bind 
public bind(scope: Construct, grantable: IGrantable): CodeConfig

Called when the Job is initialized to allow this object to bind.

scopeRequired
  • Type: constructs.Construct
grantableRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
codenize 
public codenize(): string

Static Functions

Name Description
fromAsset Job code from a local disk path.
fromBucket Job code as an S3 object.
fromAsset 
import { PythonCodenizer } from 'cdk-glue-job-builder'

PythonCodenizer.fromAsset(path: string, options?: AssetOptions)

Job code from a local disk path.

pathRequired
  • Type: string

code file (not a directory).

optionsOptional
  • Type: aws-cdk-lib.aws_s3_assets.AssetOptions
fromBucket 
import { PythonCodenizer } from 'cdk-glue-job-builder'

PythonCodenizer.fromBucket(bucket: IBucket, key: string)

Job code as an S3 object.

bucketRequired
  • Type: aws-cdk-lib.aws_s3.IBucket

The S3 bucket.

keyRequired
  • Type: string

The object key.

RepresentNullValue

Static Functions

Name Description
emptyString Empty String ("" or '').
integer Create a new {@link RepresentNullValue} from number value.
minusOne -1 Integer.
nullString "null" String.
string Create a new {@link RepresentNullValue} from string value.
emptyString 
import { RepresentNullValue } from 'cdk-glue-job-builder'

RepresentNullValue.emptyString()

Empty String ("" or '').

integer 
import { RepresentNullValue } from 'cdk-glue-job-builder'

RepresentNullValue.integer(value: number)

Create a new {@link RepresentNullValue} from number value.

valueRequired
  • Type: number

represent null value.

minusOne 
import { RepresentNullValue } from 'cdk-glue-job-builder'

RepresentNullValue.minusOne()

1 Integer.

nullString 
import { RepresentNullValue } from 'cdk-glue-job-builder'

RepresentNullValue.nullString()

"null" String.

string 
import { RepresentNullValue } from 'cdk-glue-job-builder'

RepresentNullValue.string(value: string)

Create a new {@link RepresentNullValue} from string value.

valueRequired
  • Type: string

represent null value.

Properties

Name Type Description
type RepresentNullType No description.
value string No description.
typeRequired 
public readonly type: RepresentNullType;
valueRequired 
public readonly value: string;
  • Type: string

S3CatalogSource

Specifies an Amazon S3 data store in the Glue Data Catalog.

Initializers 

import { S3CatalogSource } from 'cdk-glue-job-builder'

new S3CatalogSource(id: string, props: S3CatalogSourceProps)
Name Type Description
id string No description.
props S3CatalogSourceProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
database @aws-cdk/aws-glue-alpha.IDatabase No description.
table @aws-cdk/aws-glue-alpha.ITable No description.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase
tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

S3CatalogTarget

Specifies a data target that writes to Amazon S3 using the Glue Data Catalog.

Initializers 

import { S3CatalogTarget } from 'cdk-glue-job-builder'

new S3CatalogTarget(id: string, props: S3CatalogTargetProps)
Name Type Description
id string No description.
props S3CatalogTargetProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
database @aws-cdk/aws-glue-alpha.IDatabase No description.
table @aws-cdk/aws-glue-alpha.ITable No description.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

databaseRequired 
public readonly database: IDatabase;
  • Type: @aws-cdk/aws-glue-alpha.IDatabase
tableRequired 
public readonly table: ITable;
  • Type: @aws-cdk/aws-glue-alpha.ITable

ScalaCodenizer

Build Scala code from nodes.

Initializers 

import { ScalaCodenizer } from 'cdk-glue-job-builder'

new ScalaCodenizer(node: INode, bucket?: IBucket, objectKey?: string)
Name Type Description
node INode The Node you want to build.
bucket aws-cdk-lib.aws_s3.IBucket Built code save S3 bucket.
objectKey string Built code save S3 object key.
nodeRequired

The Node you want to build.

bucketOptional
  • Type: aws-cdk-lib.aws_s3.IBucket
  • Default: generate a new Bucket

Built code save S3 bucket.

objectKeyOptional
  • Type: string
  • Default: generate a object key

Built code save S3 object key.

Methods

Name Description
bind Called when the Job is initialized to allow this object to bind.
codenize No description.
bind 
public bind(scope: Construct, grantable: IGrantable): CodeConfig

Called when the Job is initialized to allow this object to bind.

scopeRequired
  • Type: constructs.Construct
grantableRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
codenize 
public codenize(): string

Static Functions

Name Description
fromAsset Job code from a local disk path.
fromBucket Job code as an S3 object.
fromAsset 
import { ScalaCodenizer } from 'cdk-glue-job-builder'

ScalaCodenizer.fromAsset(path: string, options?: AssetOptions)

Job code from a local disk path.

pathRequired
  • Type: string

code file (not a directory).

optionsOptional
  • Type: aws-cdk-lib.aws_s3_assets.AssetOptions
fromBucket 
import { ScalaCodenizer } from 'cdk-glue-job-builder'

ScalaCodenizer.fromBucket(bucket: IBucket, key: string)

Job code as an S3 object.

bucketRequired
  • Type: aws-cdk-lib.aws_s3.IBucket

The S3 bucket.

keyRequired
  • Type: string

The object key.

SelectFields

Choose which fields you want from your data.

Initializers 

import { SelectFields } from 'cdk-glue-job-builder'

new SelectFields(id: string, props: SelectFieldsProps)
Name Type Description
id string No description.
props SelectFieldsProps No description.
idRequired
  • Type: string
propsRequired

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired
grant 
public grant(_job: IGrantable): Grant

Granting Glue job permissions.

_jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable
python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string No description.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string
nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Generation of node.

Transform

Transforms.

Initializers 

import { Transform } from 'cdk-glue-job-builder'

new Transform()
Name Type Description

Static Functions

Name Description
applyMapping Create a new {@link ApplyMapping}.
dropFields Create a new {@link DropFields}.
dropNullFields Create a new {@link DropNullFields}.
selectFields Create a new {@link SelectFields}.
applyMapping 
import { Transform } from 'cdk-glue-job-builder'

Transform.applyMapping(id: string, props: ApplyMappingProps)

Create a new {@link ApplyMapping}.

idRequired
  • Type: string
propsRequired
dropFields 
import { Transform } from 'cdk-glue-job-builder'

Transform.dropFields(id: string, props: DropFieldsProps)

Create a new {@link DropFields}.

idRequired
  • Type: string
propsRequired
dropNullFields 
import { Transform } from 'cdk-glue-job-builder'

Transform.dropNullFields(id: string, props: DropNullFieldsProps)

Create a new {@link DropNullFields}.

idRequired
  • Type: string
propsRequired
selectFields 
import { Transform } from 'cdk-glue-job-builder'

Transform.selectFields(id: string, props: SelectFieldsProps)

Create a new {@link SelectFields}.

idRequired
  • Type: string
propsRequired

Protocols

IGranter

Granter the necessary authorization to a node.

Methods

Name Description
grant Granting Glue job permissions.
grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable

The Glue job.

INode

A node of job.

Methods

Name Description
from Chain from previous node.
grant Granting Glue job permissions.
python Build Python code fragment.
scala Build Scala code fragment.
to Chain to the next node.
from 
public from(node: INode): INode

Chain from previous node.

nodeRequired

previous node.

grant 
public grant(job: IGrantable): Grant

Granting Glue job permissions.

jobRequired
  • Type: aws-cdk-lib.aws_iam.IGrantable

The Glue job.

python 
public python(): CodeFragment

Build Python code fragment.

scala 
public scala(): CodeFragment

Build Scala code fragment.

to 
public to(node: INode): INode

Chain to the next node.

nodeRequired

next node.

Properties

Name Type Description
inputs INode[] Input nodes (i.e parent nodes).
name string Name of node.
nodeId string ID of node.
outputs INode[] Output nodes (i.e child nodes).
generation number Generation of node.
inputsRequired 
public readonly inputs: INode[];

Input nodes (i.e parent nodes).

nameRequired 
public readonly name: string;
  • Type: string

Name of node.

nodeIdRequired 
public readonly nodeId: string;
  • Type: string

ID of node.

outputsRequired 
public readonly outputs: INode[];

Output nodes (i.e child nodes).

generationRequired 
public readonly generation: number;
  • Type: number

Generation of node.

Enums

RepresentNullType

Type representing a null value.

Members

Name Description
STRING No description.
INTEGER No description.
STRING
INTEGER