Airbyte Server

  • Commercial Systems
Channel Revision Published Runs on
latest/stable 5 17 Oct 2024
Ubuntu 22.04
latest/edge 5 17 Oct 2024
Ubuntu 22.04
juju deploy airbyte-k8s
Show information

Learn to deploy on juju >

Platform:

Learn about configurations >

  • aws-access-key | string

    Defines the aws_access_key_id from the AWS credentials to use for AWS Secret Manager.

  • aws-kms-key-arn | string

    Optional param that defines the KMS Encryption key used for the AWS Secret Manager.

  • aws-secret-access-key | string

    Defines aws_secret_access_key to use for the AWS Secret Manager.

  • aws-secret-manager-secret-tags | string

    Defines the tags that will be included to all writes to the AWS Secret Manager.

    The format should be "key1=value1,key2=value2".

  • check-job-kube-annotations | string

    Defines one or more pod annotations for the check job.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2

  • check-job-kube-node-selectors | string

    Defines one or more pod node selectors for the check job.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2.

  • discover-job-kube-annotations | string

    Defines one or more pod annotations for the discover job.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2

  • discover-job-kube-node-selectors | string

    Defines one or more pod node selectors for the discover job.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2.

  • job-kube-annotations | string

    Defines one or more Job pod annotations.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2. It is the pod annotations of the "sync" job. It also serves as the default pod annotations fallback for other jobs.

  • job-kube-busybox-image | string

    Defines the Job pod busybox image. e.g. busybox:1.28

  • job-kube-curl-image | string

    Defines the Job pod curl image. e.g. curlimages/curl:7.83.1

  • job-kube-main-container-image-pull-policy | string

    Default: IfNotPresent

    Defines the Job pod connector image pull policy.

  • job-kube-main-container-image-pull-secret | string

    Defines the Job pod connector image pull secret. Useful when hosting private images.

  • job-kube-namespace | string

    Defines the Kubernetes namespace Job pods are created in.

    Defaults to the current namespace.

  • job-kube-node-selectors | string

    Defines one or more Job pod node selectors.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2. It is the pod node selectors of the "sync" job. It also serves as the default pod node selectors fallback for other jobs.

  • job-kube-sidecar-container-image-pull-policy | string

    Default: IfNotPresent

    Defines the image pull policy on the sidecar containers in the Job pod.

    Useful when there are cluster policies enforcing to always pull.

  • job-kube-socat-image | string

    Defines the Job pod socat image. e.g. alpine/socat:1.7.4.3-r0

  • job-kube-tolerations | string

    Defines one or more Job pod tolerations.

    Tolerations are separated by ';'. Each toleration contains k=v pairs mentioning some/all of key, effect, operator and value and separated by ','.

  • job-main-container-cpu-limit | string

    Job container's maximum CPU usage. Defaults to none.

  • job-main-container-cpu-request | string

    Job container's minimum CPU usage. Defaults to none.

  • job-main-container-memory-limit | string

    Job container's maximum RAM usage. Defaults to none.

  • job-main-container-memory-request | string

    Job container's minimum RAM usage. Defaults to none.

  • log-level | string

    Default: INFO

    Configures the log level.

    Acceptable values are: "INFO", "DEBUG", "WARNING", "ERROR" and "FATAL"

  • logs-ttl | int

    Default: 30

    Number of days until logs are purged from object storage.

  • max-check-workers | int

    Default: 5

    Maximum number of Check workers each Airbyte Worker container can support. Defaults to 5.

  • max-days-of-only-failed-jobs-before-connection-disable | int

    Default: 14

    Number of consecuative days of only failed jobs before the connection is disabled.

  • max-discover-workers | int

    Default: 5

    Maximum number of Discover workers each Airbyte Worker container can support. Defaults to 5.

  • max-failed-jobs-in-a-row-before-connection-disable | int

    Default: 20

    Number of consecuative failed jobs before the connection is disabled.

  • max-fields-per-connections | int

    Default: 20000

    Maximum number of fields able to be selected for a single connection.

  • max-spec-workers | int

    Default: 5

    Maximum number of Spec workers each Airbyte Worker container can support. Defaults to 5.

  • max-sync-workers | int

    Default: 5

    Maximum number of Sync workers each Airbyte Worker container can support. Defaults to 5.

  • pod-running-ttl-minutes | int

    Default: 240

    Number of minutes until a running job pod is removed.

  • pod-successful-ttl-minutes | int

    Default: 30

    Number of minutes until a successful job pod is removed.

  • pod-unsuccessful-ttl-minutes | int

    Default: 1440

    Number of minutes until an unsuccessful job pod is removed.

  • secret-persistence | string

    Defines the Secret Persistence type.

    If set, acceptable values are: "GOOGLE_SECRET_MANAGER", "AWS_SECRET_MANAGER", "TESTING_CONFIG_DB_TABLE", "VAULT"

  • secret-store-gcp-credentials | string

    Defines the JSON credentials used to read/write Airbyte Configuration to Google Secret Manager.

    These credentials must have Secret Manager Read/Write access. Alpha support.

  • secret-store-gcp-project-id | string

    Defines the GCP Project to store secrets in. Alpha support.

  • spec-job-kube-annotations | string

    Defines one or more pod annotations for the spec job.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2

  • spec-job-kube-node-selectors | string

    Defines one or more pod node selectors for the spec job.

    Each k=v pair is separated by a ','. For example: key1=value1,key2=value2.

  • storage-bucket-activity-payload | string

    Default: airbyte-payload-storage

    Name of activity payload storage bucket.

  • storage-bucket-logs | string

    Default: airbyte-dev-logs

    Name of logs storage bucket.

  • storage-bucket-state | string

    Default: airbyte-state-storage

    Name of state storage bucket.

  • storage-bucket-workload-output | string

    Default: airbyte-state-storage

    Name of workload output storage bucket.

  • storage-type | string

    Default: MINIO

    Storage type for logs.

    Acceptable values are: "MINIO", "S3" (AWS)

  • sync-job-max-timeout-days | int

    Default: 3

    Number of days a sync job will execute for before timing out.

  • sync-job-retries-complete-failures-backoff-base | int

    Default: 3

    Exponential base of the backoff interval between failed attempts in which no data was synchronized.

  • sync-job-retries-complete-failures-backoff-max-interval-s | int

    Default: 1800

    Maximum backoff interval in seconds between failed attempts in which no data was synchronized.

  • sync-job-retries-complete-failures-backoff-min-interval-s | int

    Default: 10

    Minimum backoff interval in seconds between failed attempts in which no data was synchronized.

  • sync-job-retries-complete-failures-max-successive | int

    Default: 5

    Max number of successive attempts in which no data was synchronized before failing the job.

  • sync-job-retries-complete-failures-max-total | int

    Default: 10

    Max number of attempts in which no data was synchronized before failing the job.

  • sync-job-retries-partial-failures-max-successive | int

    Default: 1000

    Max number of successive attempts in which some data was synchronized before failing the job.

  • sync-job-retries-partial-failures-max-total | int

    Default: 20

    Max number of attempts in which some data was synchronized before failing the job.

  • temporal-history-retention-in-days | int

    Default: 30

    Retention period of the job history in Temporal, defaults to 30 days.

  • temporal-host | string

    Default: temporal-k8s:7233

    Temporal server host.

  • vault-address | string

    Defines the vault address to read/write Airbyte Configuration to Hashicorp Vault. Alpha Support.

  • vault-auth-method | string

    Default: token

    How vault will perform authentication. Currently, only supports Token auth. Defaults to "token". Alpha Support.

  • vault-auth-token | string

    The token used for vault authentication. Alpha Support.

  • vault-prefix | string

    Defines the vault path prefix. Empty by default. Alpha Support.

  • webapp-url | string

    Default: http://airbyte-ui-k8s:8080

    URL for the Airbyte webapp.