Skip to content

[CIEXE-2681] Use published Docker CI images with internal Buildx - #882

Open
ericdog wants to merge 10 commits into
DataDog:mainfrom
ericdog:ericlim/internal-buildx-modern-docker-20261103
Open

ericdog wants to merge 10 commits into
DataDog:mainfrom
ericdog:ericlim/internal-buildx-modern-docker-20261103

Conversation

@ericdog

@ericdog ericdog commented Nov 3, 2026 •

Copy link
Copy Markdown

Use maintained docker:27.3.1 and docker:29.4.0-noble-py3 tags in CI instead of pipeline-specific images or fixed digests. Keep the Python virtual-environment setup and SKIP_PIP_INSTALL=true in layer publishing jobs so Noble does not attempt a system-pip install.

Removed the temporary validate-python-ci-tools job and all four branch-specific deployment guards following review. Existing CI rules, including SKIP_E2E_TESTS, are restored.

images#12612 is merged and both maintained shared images were rebuilt and published from master with internal Buildx v0.38.2-dd.3.

Validation: rendered the Go template locally and parsed the generated YAML using GitLab-compatible reused-anchor handling; all 28 publishing jobs retain the Python setup and pip skip flag, and the validation-only job is absent. All 44 affected integration, publishing and E2E jobs retain their original skip rules without a hardcoded branch exception. Shell syntax and diff checks pass. The new commit is signed and GitHub-verified; CI is enabled. Earlier candidate runs passed layer builds, size checks and unit tests.

Integration CI follow-up: install the existing test dependencies from tests/integration (the old integration_tests directory no longer exists). Preserve aws sts assume-role failures instead of swallowing them inside export, so runner proxy/authentication failures stop immediately. No tests are skipped and Fabric proxy settings are unchanged.

Validation: Bash syntax and diff checks pass; local credential-success and credential-failure checks confirm exports and failure propagation. The pipeline template renders successfully. Fresh CI is enabled; the commit is signed and GitHub-verified.

@ericdog
ericdog force-pushed the ericlim/internal-buildx-modern-docker-20261103 branch from edda1eb to 0970e69 Compare November 3, 2026 15:42
@ericdog
ericdog marked this pull request as ready for review November 4, 2026 16:26
@ericdog
ericdog requested review from a team as code owners November 4, 2026 16:26
@ericdog
ericdog requested a review from lym953 November 4, 2026 16:26
@ericdog

ericdog commented Nov 4, 2026

Copy link
Copy Markdown
Author

/ddci trigger

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-11-04 16:27:34 UTC ℹ️ Start processing command /ddci trigger


2026-11-04 16:27:37 UTC

Tasks sourcing running asynchronously in workflow devflow:4924896d-a00c-476c-a539-b3c721df0f9f_47:01a1a763-25c5-7b53-9ea7-7b5e2b14c881 under run id 01a1a763-2e2d-7a17-b0cb-f742a5242fe9

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

With SKIP_PIP_INSTALL unset, the Noble-based layer-publishing jobs still invoke system pip and abort under PEP 668 before publishing any layers.

Open Bits AI session

🤖 Bits Code Review · Commit 0970e69

Comment thread ci/input_files/build.yaml.tpl Resolved
@ericdog
ericdog force-pushed the ericlim/internal-buildx-modern-docker-20261103 branch from 1ff3e17 to 628602f Compare November 5, 2026 11:36
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: da42843 | Docs | View more details | Give us feedback!

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

CI jobs now use digest-pinned Docker images with internal Buildx, apt-installed AWS tools, and Python virtual environments. Publishing exports SKIP_PIP_INSTALL=true, resolving the earlier Noble system-pip failure. No findings remain open. Full container and CI integration validation remains outstanding.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 97c3648

@ericdog ericdog added the buildx-internal-rollout Internal Buildx CI adoption label Nov 12, 2026
Comment thread ci/input_files/build.yaml.tpl Outdated
rules:
# Image-maintenance validation has no authorization to deploy Lambda services.
- if: '$CI_COMMIT_BRANCH == "ericlim/internal-buildx-modern-docker-20261103" || $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "ericlim/internal-buildx-modern-docker-20261103"'
when: never

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to commit these?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed not thanks for the catch !

@ericdog

ericdog commented Nov 12, 2026

Copy link
Copy Markdown
Author

/gitlab resync-job-status

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-11-12 23:54:56 UTC ℹ️ Start processing command /gitlab resync-job-status


2026-11-12 23:55:44 UTC

47 jobs status updated and 0 skipped for a total of 47 jobs

@ericdog
ericdog requested a review from purple4reina November 17, 2026 13:42
@ericdog

ericdog commented Nov 17, 2026

Copy link
Copy Markdown
Author

/ddci trigger

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-11-17 13:58:32 UTC ℹ️ Start processing command /ddci trigger


2026-11-17 13:58:37 UTC

Tasks sourcing running asynchronously in workflow devflow:c5593916-57f2-4b84-acd2-7682b01516c8_47:01a1e9cd-63ad-74b8-a2c4-503048e1a5b3 under run id 01a1e9cd-6d51-7c87-bfd6-8ff029fb845c

@lucaspimentel

Copy link
Copy Markdown
Member

@DataDog review

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The credential assignment preserves failed role assumptions, while CI tool installation uses a virtual environment for Noble’s managed Python. Execution of the image-based GitLab jobs remains unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit da42843

@purple4reina purple4reina left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seem to be failures in the integration tests. https://gitlab.ddbuild.io/DataDog/datadog-lambda-python/-/jobs/2264147211

It's showing aws sts assume-role failed with Failed to connect to proxy URL: "http://127.0.0.1:15002"

@ericdog

ericdog commented Nov 19, 2026

Copy link
Copy Markdown
Author

Reopened as #894 from a branch in DataDog/datadog-lambda-python so repository CI can run. It retains SKIP_PIP_INSTALL=true and the removal of temporary validation/branch-guard changes. It also retains the corrected tests/integration path and fail-fast AWS role-assumption handling. The cited integration job could not connect to the runner Fabric proxy at 127.0.0.1:15002; proxy configuration is unchanged, and fresh CI will show whether that infrastructure failure persists. Leaving this PR open to preserve the review history.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

buildx-internal-rollout Internal Buildx CI adoption

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants