2026-05-07 Engineering Team

[joecastiglione]

Agenda

• Random numbers generation
• Generalized check in on progress

[jpn--]

ActivitySim Meeting Notes

Meeting Overview

Agenda

The meeting was an open check-in with no formal agenda. The primary focus was coordinating on random number generation (RNG) work and reviewing profiling results to understand performance implications for different sampling approaches.

Participants

• Jeff Newman (@jpn--) — Driftless
• Jan Zill (@janzill) — Outer Loop
• David Hensle (@dhensle) — RSG
• Sumit Bindra — TransLink

Duration

Approximately 30 minutes

---

Executive Summary

The meeting focused on profiling results comparing three sampling approaches for ActivitySim's destination choice model: Monte Carlo sampling, Poisson sampling, and explicit error term (EET) sampling. Jan Zill shared profiling data from a 100,000-household single-process run of the MTC model, showing that Poisson sampling is approximately 10–30% slower than Monte Carlo, while explicit ET sampling is roughly 3× slower due to the volume of random number draws. The group reached consensus that improving the random number generator provides modest but real gains across all approaches, and that Poisson sampling combined with RNG improvements is the most pragmatic path to production. Next steps include finishing the Poisson sampling implementation and having David Hensle conduct additional profiling on the SANDAG model.

---

Meeting Notes

Profiling Results: Sampling Approaches Compared

Jan Zill shared profiling results comparing three destination choice sampling methods on a 100,000-household single-process run (MTC model, ~1,450 zones), representing approximately one process in a 25-process multiprocessing run.

The three steps profiled were: sampling, log sum computation, and final choice.

Key findings:

• Log sum computation takes the same time across all methods — no difference there.
• Monte Carlo sampling: baseline; 9 seconds for the destination sample step. The Monte Carlo path uses Numba-compiled (JIT) code and is already highly optimized.
• Poisson sampling: ~11.8 seconds for the destination sample step, approximately 30% slower than Monte Carlo. Overall, across all three steps, Poisson sampling was about 10% slower. Random number drawing accounts for roughly 5% of total Poisson sampling runtime — a small but non-trivial share.
• Explicit ET sampling: ~30 seconds for the destination sample step, nearly 3× slower than Poisson and ~3× slower than Monte Carlo. The dominant cost is in the make_sample_choices_utility_based step (~21 seconds), driven almost entirely by random number generation (Gumbel max draws taking ~6 seconds alone). Jan noted that he had already implemented memory optimizations for ET to make the comparison fair (iterating over draws rather than realizing a large array in memory), but the runtime penalty remains severe.

The profiling showed that the random number rand method call for Poisson sampling takes roughly 10× longer per draw than the equivalent call in the Monte Carlo path.

Implications for Random Number Generator Improvements

The group discussed whether faster RNG is a "magic bullet":

• For Monte Carlo, RNG improvements offer modest gains — not a runtime halver, but real. Most of the implementation work is already done.
• For Poisson sampling, faster RNG could bring it much closer to Monte Carlo parity, since ~5% of its sampling runtime is in RNG. Full parity isn't achievable because Poisson requires computing probabilities and drawing a variable number of alternative random numbers, whereas Monte Carlo does not.
• For explicit ET sampling, faster RNG would be nearly mandatory to make it viable — but the group consensus was that ET sampling is not the preferred path regardless.
• A "top-N" approach was briefly discussed as theoretically interesting and potentially able to make all methods faster, but it would require significant theoretical work, risk of introducing bias, and is less generic/robust than Poisson sampling.

David Hensle noted the situation may be even more pronounced for the SANDAG model, which has two zone systems and many MAZs, potentially amplifying the share of runtime spent on random draws.

Consensus and Path Forward

The group reached the following conclusions:

1. RNG improvements: worth doing unconditionally — modest but real gains across all approaches. Most implementation work is already complete between Jeff Newman and David Hensle.
2. Poisson sampling: worth implementing unconditionally for ET use cases — it is the pragmatic, robust choice. Combined with RNG improvements, it offers a practical path to getting explicit error terms into production.
3. Explicit ET sampling (naive approach): not recommended; 3× slower and requires RNG to be near-instantaneous to become viable.
4. Top-N sampling: intellectually interesting but requires substantial theoretical investigation; deferred.

Sumit Bindra suggested a standardized profiling table format: report total and maximum runtimes per major step (e.g., destination choice) relative to Monte Carlo, omitting fine-grained sub-second breakdowns for easier cross-team comparison.

---

Action Items

• Jan Zill: Finalize implementation of Poisson sampling (Joe Castiglione to coordinate resources).
• Jan Zill: Produce a simplified profiling summary table (total and maximum per major step, with Monte Carlo multiplier) based on existing data, per Sumit Bindra's suggestion.
• David Hensle: Run profiling on the SANDAG model to assess RNG share of runtime in a two-zone-system context.
• Jeff Newman / David Hensle: Finish and finalize the RNG improvement implementation, deciding on the remaining implementation details (fast hash vs. other approach).
• All: Share profiling results in the standardized table format as they become available.