[Enhancement]: PortForwardingContainer breaks reuse hash

[flynny75]

Problem

I am using a message broker container with reuse.

I am using other transient containers that need to be able to reach the host.

I am requesting TestContainers expose the ports after creating the reuse container, but before the transient containers. When ran in isolation this works well.

I am using TestContainers to run multiple integration tests at the same time, sharing the reuse container between multiple tests.

The issue I am facing is the second test runs, attempts to start the reuse container (which is already running, and was started before the PortForwardingContainer), but the builder now has ExtraHosts configured because the PortForwardingContainer is now running. Thus the hash calculation is different and the reuse container is now a clash.

This is because the default implementation of container builder init is to apply the port forwarding if the (static) PortForwardingContainer is running.

Solution

Provide the ability for a container builder to clear the ExtraHosts

OR expose WithPortForwarding such that a container builder can 'opt in' to using the PortForwardingContainer

Benefit

Container reuse would cooperate with transient containers that require port forwarding

Alternatives

N/A

Would you like to help contributing this enhancement?

Yes

[coderabbitai]

🔗 Related PRs

#1555 - chore: Don't create a static field in a generic class [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[flynny75]

0001-Add-more-control-over-automatic-PortForwarding.txt

I have attached a patch on 4.11.0 that resolves my issue.

I set TestcontainersSettings.AutoPortForwarding=false, enable port forwarding via ExposeHostPortsAsync, then for the transient contains that require the functionality I call WithPortForwarding on the container builder.

[HofmeisterAn]

Thanks for creating the issue and looking into how to support and fix it. I was thinking of using something similar to what we're using for labels. There, we use a custom JSON converter that skips (ignores) TC runtime labels.

testcontainers-dotnet/src/Testcontainers/Configurations/Commons/ResourceConfiguration.cs

Line 73 in 524bade

[JsonConverter(typeof(JsonIgnoreRuntimeResourceLabels))]

I was thinking of using a similar approach for the extra host. But I need to think about whether it causes other issues. WDYT?

[flynny75]

Hmm, I feel that that approach would be opinionated in that ExtraHosts would never be included in the reuse hash calculation. If I wanted to have multiple reuse containers of almost identical configuration apart from the ExtraHosts I wouldn't be able to do it. I don't have an example use case.

I think the most flexible approach would be to add a WithReuseHashFunction to the builder that takes a Func<string, byte[]> , where the caller can supply a delegate that is provided with the json serialised configuration and returns the hash.

The caller can deserialise and add/remove fields before calculating the hash. (I only suggest a string so as not to tie callers to a particular JSON API).

It's a little cumbersome, but in my opinion this issue is probably a niche scenario and the effort of making an elegant but comprehensive API (such that container builders can influence the inclusion of every field in the reuse hash) is not worth the initial investment.

Alternatively, this new property could be declared on the TestcontainersSettings type so that it would not require updates from module packages to expose the new API through the builder

[HofmeisterAn]

I feel that that approach would be opinionated in that ExtraHosts would never be included in the reuse hash calculation. If I wanted to have multiple reuse containers of almost identical configuration apart from the ExtraHosts I wouldn't be able to do it. I don't have an example use case.

I'm not sure I understood this correctly. It won't ignore ExtraHosts entirely, just the host.testcontainers.internal entry. Basically, you're running into the same issue as our CI:

testcontainers-dotnet/tests/Testcontainers.Platform.Linux.Tests/ReusableResourceTest.cs

Lines 3 to 9 in 470e975

	// We cannot run these tests in parallel because they interfere with the port
	// forwarding tests. When the port forwarding container is running, Testcontainers
	// automatically inject the necessary extra hosts into the builder configuration
	// using `WithPortForwarding()` internally. Depending on when the test framework
	// starts the port forwarding container, these extra hosts can lead to flakiness.
	// This happens because the reuse hash changes, resulting in two containers with
	// the same labels running instead of one.

Including host.testcontainers.internal in the reuse hash doesn't really make sense. Port forwarding won't work reliably with reuse anyway, because the IP of the port forwarding container can change. That's why I think it's fine to ignore it (and it will be better for our CI as well).

I think the most flexible approach would be to add a WithReuseHashFunction to the builder that takes a Func<string, byte[]> , where the caller can supply a delegate that is provided with the json serialised configuration and returns the hash.

I was thinking about something similar, allowing developers to override the default hash implementation.

[flynny75]

It won't ignore ExtraHosts entirely, just the host.testcontainers.internal entry.

Ah ok, I misunderstood. I think your suggestion of only excluding host.testcontainers.internal is viable and would solve my problem, and is probably the shortest path to short term success without worrying about a new builder API yet

[HofmeisterAn]

It won't ignore ExtraHosts entirely, just the host.testcontainers.internal entry.

Ah ok, I misunderstood. I think your suggestion of only excluding host.testcontainers.internal is viable and would solve my problem, and is probably the shortest path to short term success without worrying about a new builder API yet

Yes, exactly. I looked into this today, and I think we can add a delegate that allows developers to set their own reuse hash.