Skip to main content
Version: v5.3

Release 5.3.0

Reference Deployment Versions​

Release 5.3.0 was validated on the following infrastructure. The minimum versions listed in the requirements are unchanged. MongoDB 8.1 and later are not supported; use MongoDB 8.0.x at most.

Component5.2.1 reference5.3.0 reference
Kubernetes (EKS)1.351.35, 1.36
Kubernetes (OKE)v1.35.2v1.36.1
Calico (OKE)v3.30.3v3.30.3, v3.32.0
Gateway API CRDs (standard channel)v1.5.1v1.5.1
Istio1.30.21.30.2
Kafka (Strimzi operator 0.49.1)4.1.04.1.0
PostgreSQL1515
MongoDB8.08.0
Redis (Valkey)8.1, 8.28.1, 8.2
Neo4j4.44.4
ActiveMQ Classic5.185.18
External Secrets Operator (chart)0.9.20, 0.10.40.9.20, 0.10.4
cert-managerv1.17.2v1.17.2
kube-prometheus-stack (chart)45.28.145.28.1
prometheus-adapter (chart)4.2.04.2.0

The only infrastructure change from 5.2.1 is the Kubernetes minor version (and Calico on OKE). No changes to data services are required for the upgrade.

New Services​

The following new services are added to the DT Platform Helm Chart:

  • aosvc-api - The REST API for the Application Ontology Service, routed at /aosvc on the API host
  • aosvc-backend - The worker backend for the Application Ontology Service
  • aosvc-migration - Job that runs the Application Ontology Service MongoDB migrations
  • aosvc-postgres-migration - Job that runs the Application Ontology Service PostgreSQL migrations (job queue tables)
  • eventviewersvc - The Event Viewer Service, routed at /eventviewersvc on the API host

Both Application Ontology Service migration jobs must complete before aosvc-api and aosvc-backend start.

eventviewersvc runs a single replica by default. Each pod holds its own rolling window of events and all pods share one consumer group, so additional replicas would answer reads from a partial window. Do not scale it beyond one replica.

The Application Ontology Service is enabled by default. To install without it, set the following to false:

deployment.aosvc-api.enabled
deployment.aosvc-backend.enabled
job.aosvc-migration.enabled
job.aosvc-postgres-migration.enabled
service.aosvc-api.enabled
service.aosvc-backend.enabled
ingress.aosvc.enabled

Disabling these removes only the Application Ontology Service workloads, services, and ingress. As with any other disabled service, the aosvc-* configmaps and secrets are still rendered.

New External Services Configuration​

The new services use the following resources in the external services.

ServiceResourceName
PostgreSQLDatabaseaosvcdb
PostgreSQLUseraosvc_app
MongoDBDatabaseaosvcdb
MongoDBUseraosvc_app
KafkaUserstrimzi-kafka-aosvc-api
KafkaUserstrimzi-kafka-aosvc-backend
KafkaUserstrimzi-kafka-eventviewersvc
RedisUsereventviewersvc

The names are configured at config.external.postgres.database.aosvc, config.external.postgres.user.aosvc, config.external.mongodb.database.aosvc, config.external.mongodb.user.aosvc, config.external.kafka.user.aosvc-api, config.external.kafka.user.aosvc-backend, config.external.kafka.user.eventviewersvc, and config.external.redis.user.eventviewersvc.

The aosvc-postgres-migration job creates the PostgreSQL database and user with the administrator account, as the other services' migration jobs do. Only POSTGRES_AOSVC_PASSWORD needs to be provisioned.

The Kafka and Redis users are created the same way as the existing services' users.

The MongoDB user must be created before deploying. Connect to the MongoDB cluster with an administrative user and run the following, replacing XXXXX with the password stored as MONGODB_AOSVC_PASSWORD:

db.getSiblingDB("admin").createUser(
{
user: "aosvc_app",
pwd: "XXXXX",
roles: [
{
role: "readWrite",
db: "aosvcdb"
}
]
}
);

On MongoDB Atlas, create the database user in the Atlas UI or API with the readWrite role on aosvcdb.

New External Secrets​

New external secrets must be created with the corresponding environment variables below. None of the new variables have defaults.

aosvc-external​

  • AOSVC_ENCRYPTION_KEY - New. Encrypts the submitter credential stored with each queued Application Ontology Service job
  • AOSVC_ENCRYPTION_SALT - New. Salt used with AOSVC_ENCRYPTION_KEY
  • KAFKA_ADMINISTRATOR_PASSWORD
  • KAFKA_AOSVC_API_PASSWORD - New
  • KAFKA_AOSVC_BACKEND_PASSWORD - New
  • MONGODB_AOSVC_PASSWORD - New
  • POSTGRES_AOSVC_PASSWORD - New

eventviewersvc-external​

  • KAFKA_ADMINISTRATOR_PASSWORD
  • KAFKA_EVENTVIEWERSVC_PASSWORD - New
  • REDIS_EVENTVIEWERSVC_PASSWORD - New

Updating Existing External Secrets​

filesvc-external​

  • FILE_DOWNLOAD_PRESIGNED_SECRET - New. HMAC signing secret for File Service download tokens. Required when config.global.cloud.filesvc.file_download_mode is proxy (see File Download Mode below). Provisioning it on every installation is recommended so the mode can be changed later without a secret update.

New Internal Secrets​

The following passwords are generated automatically by the chart and require no action:

  • PASSPORTSVC_AOSVC_PASSWORD - Allows aosvc to access passportsvc (in passportsvc-internal, aosvc-api-internal, aosvc-backend-internal)
  • PASSPORTSVC_EVENTVIEWERSVC_PASSWORD - Allows eventviewersvc to access passportsvc (in passportsvc-internal, eventviewersvc-internal)
  • ITEMSVC_AOSVC_PASSWORD - Allows aosvc to access itemsvc (in itemsvc-internal, aosvc-api-internal, aosvc-backend-internal)

The existing INITIAL_OWNER_PASSWORD and PLATFORMKAFKACONNECT_DEFAULT_PASSWORD generated passwords are now also included in the new aosvc-internal secret. It is shared by all four Application Ontology Service workloads, which also mount their own aosvc-api-internal or aosvc-backend-internal secret where they have one.

New Object Storage Bucket​

The Application Ontology Service requires its own object storage bucket for queued job payloads (import request bodies). Create it according to the normal documentation for AWS or OCI, and set its name at config.global.cloud.bucket.aosvc (default XXXXX-dtplatform-prod1-aosvc).

The reference deployment enables versioning on the bucket, expires noncurrent versions after 365 days, and aborts incomplete multipart uploads after 7 days.

New Object Storage Access Policies​

The following Kubernetes service accounts must be granted access to the corresponding buckets:

Kubernetes Service AccountPolicy
aosvc-apikafka, aosvc
aosvc-backendkafka, aosvc

On AWS, the aosvc policy uses the same bucket and object actions as the other bucket policies. If the bucket is encrypted with a customer managed KMS key, the policy also needs kms:GenerateDataKey and kms:Decrypt on that key.

File Download Mode​

The File Service can now issue download URLs in one of three ways, selected by the new config.global.cloud.filesvc.file_download_mode setting:

ValueBehaviorClouds
cloudfrontPre-signed CloudFront URLaws only
nativePre-signed URL from the object storeoci only
proxyThe File Service streams the file through a signed token URLaws and oci

The setting is unset by default, and the File Service selects cloudfront on aws and native on oci, so existing installations need no change. Set it only to choose a non-default mode. Setting cloudfront on a non-aws installation is rejected at startup. The value is validated by the chart's values schema.

proxy mode signs its tokens with config.global.cloud.filesvc.presigned_secret, which defaults to ${FILE_DOWNLOAD_PRESIGNED_SECRET} from filesvc-external.

Application Ontology Service Configuration​

New config.internal.aosvc-api, config.internal.aosvc-backend, and config.internal.aosvc-migration blocks have been added to values.yaml. They hold only how the Application Ontology Service reaches other services. Application settings such as log levels, leases, and limits use the application's own defaults, and config.override.aosvc-api, config.override.aosvc-backend, and config.override.aosvc-migration default to empty.

A new config.internal.workflowsvc block (host, port, and scheme of workflowsvc-api) has been added for use by the Application Ontology Service.

Application Settings Removed From config.internal​

The config.internal.* blocks now carry only wiring between services. The following application settings have been removed from values.yaml, and the applications now default to the same values:

Removed keyApplication default
config.internal.workflowsvc-api.logging.levelINFO
config.internal.workflowsvc-backend.logging.levelINFO
config.internal.workflowwkr-backend.logging.levelINFO
config.internal.scriptworker.script_execution_timeout_ms7200000
config.internal.scriptworker.isolate_memory_limit_mb2048

These blocks do not accept additional properties, so an override file that still sets any of these keys under config.internal fails schema validation on upgrade. Remove the keys, or, if the installation needs a value other than the default, move it to config.override.<service>:

config:
override:
workflowsvc-api:
internal:
workflowsvc-api:
logging:
level: DEBUG
scriptworker:
SCRIPT_EXECUTION_TIMEOUT: 3600000
ISOLATE_MEMORY_LIMIT_MB: 4096

Autoscaler Metrics Scraped Through the Istio Sidecar​

The scriptmanager and workflowsvc-api ServiceMonitors, which feed the scriptworker and workflowwkr-backend autoscalers, now scrape Istio's merged metrics endpoint (port 15020, path /stats/prometheus) by default instead of the application's own port. Port 15020 is exempt from mTLS, so an unmeshed Prometheus can keep scraping after the mesh is placed in STRICT mode.

  1. A new serviceMonitor.scrapeVia setting selects the scrape target. istio (the default) scrapes the merged endpoint and adds an http-envoy-prom port (15020) to the scriptmanager and workflowsvc-api services. direct scrapes the application's http-web port as in 5.2 and omits the extra service port.
  2. istio requires an Istio sidecar on the scriptmanager and workflowsvc-api pods. Without one nothing listens on port 15020, the scrape target is down, and the scriptworker autoscaler loses its metric. Set serviceMonitor.scrapeVia to direct on installations without sidecar injection.
  3. deployment.scriptmanager.podAnnotations and deployment.workflowsvc-api.podAnnotations now default to prometheus.io/scrape, prometheus.io/path, and prometheus.io/port annotations that point at each application's metrics endpoint (/metrics on 5000 and /workflowsvc/api/v1/metrics on 4567). Istio uses them to merge application metrics into port 15020. Override files that set podAnnotations for these deployments replace the defaults and must include these annotations.

Pod Template Labels​

Pod templates in every deployment, statefulset, and job no longer carry the helm.sh/chart and app.kubernetes.io/version labels. These labels change with every chart version, so they caused a new ReplicaSet and a full rolling restart on every upgrade even when no image changed. The labels remain on the parent Deployment, StatefulSet, and Job resources.

The upgrade to 5.3.0 restarts every pod once because of this change. Network policies, monitoring rules, or scripts that select pods by helm.sh/chart or app.kubernetes.io/version must select on app.kubernetes.io/name or app.kubernetes.io/instance instead.

Resource Changes​

DeploymentSetting5.2.15.3.0
platform-notificationsvc-workermemory limit750Mi1.5Gi

Reference Terraform Modules​

The example base modules under devops/modules for AWS and OCI have been updated:

  1. An aosvc bucket is created (S3 with a dedicated KMS key on AWS, Object Storage on OCI), and the aosvc-api and aosvc-backend service accounts are granted access to it and to the kafka bucket.
  2. External secrets are created for aosvc-external (including a generated AOSVC_ENCRYPTION_KEY and AOSVC_ENCRYPTION_SALT) and eventviewersvc-external, and FILE_DOWNLOAD_PRESIGNED_SECRET is generated and added to filesvc-external.
  3. The namespace label that lets the shared gateway accept the namespace's ListenerSet is now applied with a dedicated server-side apply field manager (terraform-shared-gateway). With the default field manager, another Terraform resource writing labels to the same namespace removed the label on apply and the ListenerSet was rejected as NotAllowed. Any other automation that labels the release namespace with server-side apply should use its own field manager.
  4. The shell debug deployment has been removed.
  5. The legacy notificationsvc-external ExternalSecret and its generated POSTGRES_NOTIFICATIONSVC_PASSWORD have been removed. The chart has not used them since platform-notificationsvc replaced notificationsvc.