Repository navigation
Epic: 2020 GDP/MINLP roadmap revisited — status audit and open ideas (meeting_20200527) #2
Description
Epic: Revisiting the 2020 GDP/MINLP roadmap (IDAES meeting slides, 2020-05-27)
Source: meeting_20200527.pptx — IDAES Subtask 3.1: Conceptual Design of Advanced Energy Systems (D. Bernal & I. Grossmann, CMU, May 27, 2020).
Audit: August 16, 2026, against Pyomo/pyomo@ce205a5fe (main).
This epic audits every improvement idea in that deck against today's Pyomo (code + literature), records what already landed, and tracks what is still open and interesting. Every open item now has its own issue — defects and broadly-wanted features upstream on Pyomo/pyomo, research-agenda items here on this fork. This issue is the index. Slide references as (sN).
Part 1 — Landed since 2020 (context, no action)
| 2020 idea | Where it lives today |
|---|---|
| Logical modeling layer | BooleanVar/LogicalConstraint (2020), core.logical_to_linear, contrib.logical_to_disjunctive (2022); GDP→MIP transformations handle logic internally |
| Hull perspective options (s3–4) | gdp.hull with FurmanSawayaGrossmann (default) / LeeGrossmann / GrossmannLee ε-approximations (Furman et al., COAP 2020) |
| Exact hull via cones (s4) | Partial: exact_hull_quadratic option in gdp.hull (landed 2026-03; quadratic-only, rotated-SOC form, per Gusev & Bernal Neira, arXiv:2508.16093) — see A4 |
| Multiple big-M (s17) | gdp.mbigm (2022; [Trespalacios & Grossmann 2015]) — see A5 |
| Between big-M and hull (s17-adjacent) | gdp.partition_disjuncts / gdp.between_steps (Kronqvist, Misener & Tsay, CPAIOR 2021) — restricted case only, see A7 |
| Nested disjunctions (s10) | bigm/hull/binary_multiplication nested rework (2024, e.g. Pyomo#3213) |
| MindtPy single-tree LP/NLP B&B (s16) | mindtpy/single_tree.py lazy callbacks for cplex_persistent/gurobi_persistent — see C1 |
| MindtPy FP + level/regularized OA (s12) | mindtpy.fp; add_regularization ∈ {level_L1/L2/L∞, grad/hess/sqp Lagrangian via PyNumero} (Kronqvist et al., MathProg 2020; Bernal et al., JOGO 2022) |
| Community detection merge (s15) | contrib.community_detection (Louvain, generate_structured_model()) — analysis-only, see C5 |
| New GDPopt algorithms | gdpopt.ric, gdpopt.enumerate, gdpopt.ldsda (Ovalle et al., C&ChE 2025) — see B7 |
| GDP model library (s11) | gdplib — active development at SECQUOIA/gdplib, see D1 |
| Pyomo.GDP ecosystem paper (s18, s23) | Chen et al., OptEng 2022 |
In flight right now: non-rigorous crossed-bound detection (Pyomo#3947 → PR Pyomo#4013), LBB time-limit results (PR Pyomo#4010), GLOA convex cut slope (PR Pyomo#4011), LBB relaxed-NLP node solver (PR Pyomo#4012).
Part 2 — Open ideas, each now tracked in its own issue
Filed upstream on Pyomo/pyomo (defects & broadly-wanted features)
| Item | Upstream issue |
|---|---|
| B1 — disjunctive FBBT/OBBT options silently ignored (regression) | Pyomo#4014 |
| B7a — LDSDA missing from meta-solver dispatch (+ vestigial code) | Pyomo#4015 |
B9a — relative_gap() ZeroDivisionError / misplaced epsilon |
Pyomo#4016 |
| B9b — docstring advertises unimplemented algorithms | Pyomo#4017 |
(audit find) — gdpopt.enumerate TypeError on unbounded/float-bounded integers |
Pyomo#4018 |
| A8 — solver-native indicator-constraint transformation | Pyomo#4019 |
C1 — MindtPy single tree on APPSI/contrib.solver |
Pyomo#4020 |
| B2 — LBB solver failures misclassified as node infeasibility | Pyomo#4023 |
A. GDP reformulations (pyomo.gdp)
- A1. First-class basic steps (s6) → A1: Register basic steps as a first-class gdp.basic_step transformation #3. Register
gdp.basic_step+ selection heuristics; wire intogdp.cuttingplane. Unimplemented in any OSS package. - A2. Pseudo basic steps — Lagrangian dual bounds (s7) → A2: Pseudo basic steps — Lagrangian dual bounds for GDP #4. Papageorgiou & Trespalacios 2018; parallelizable GDP dual bounds; mpi-sppy analogy. Unimplemented anywhere.
- A3.
gdp.mpec— GDP→complementarity (s5) → A3: gdp.mpec — GDP-to-complementarity reformulation and NLP heuristics #5. Forward direction absent everywhere; NLP heuristics/initialization angle (Schewe & Schmidt 2019). - A4. Exact hull beyond quadratics (s4) → A4: Generalize exact hull beyond quadratics (cones, conic emission, reaggregation) #6. Exp/power cones (Bernal Neira & Grossmann, COAP 2024), kernel conic emission, reaggregated hull; lift mutable-Param freezing.
- A5.
gdp.mbigmnested-GDP support → A5: gdp.mbigm nested-GDP support #7. CurrentlyGDP_Error; sync with upstream maintainers on design before building. - A6.
gdp.cuttingplanemodernization (s3) → A6: gdp.cuttingplane modernization #8. Nonlinear separation, shipped basic-step tightening, structural soundness, cut management. - A7. General P-split formulations → A7: General P-split formulations #9. Kronqvist, Misener & Tsay, MathProg 2025 — general case implemented nowhere; automatic partition selection.
- A8. Indicator-constraint transformation (s3) → filed upstream: Support solver-native indicator constraints as a GDP transformation target Pyomo/pyomo#4019 (needs representation + interface support, not just a transformation).
B. GDPopt
- B1. Reinstate disjunctive FBBT/OBBT range reduction (s8) → filed upstream: GDPopt: calc_disjunctive_bounds and obbt_disjunctive_bounds are accepted but silently ignored Pyomo/pyomo#4014. Regression: options dead since the 2022 rewrite, GLOA cuts silently weakened, no-op test.
- B2. LBB rehabilitation (s9) → B2: LBB rehabilitation #10. Branching rules, non-XOR support, sane failure bounds, retire the 2022 "known issues" warning. (PRs Fix GDPopt LBB time-limit results Pyomo/pyomo#4010/Add LBB relaxed NLP node solver Pyomo/pyomo#4012 in flight; solver failures misclassified as node infeasibility filed upstream as GDPopt LBB treats solver RuntimeError as node infeasibility Pyomo/pyomo#4023.)
- B3. Logic-based branch-and-cut — "GDPopt-LBC" (s9) → B3: Logic-based branch-and-cut (GDPopt-LBC) #23. Node cuts + tree-wide cut pool; depends on B2/B1/B6.
- B4. Deeper Z3/SMT integration (s9) → B4: Deeper Z3/SMT integration in GDPopt #11. Complete term encoding, discrete presolve, unsat-core no-good strengthening, LD-SDA neighbor pruning.
- B5. Smarter set-covering initialization (s20–22) → B5: Smarter set-covering initialization for GDPopt #12. Max-coverage selection (deck's 8-process example: 2 vs 3 iterations); no newer literature variant exists — publishable.
- B6. Disjunctive cut strengthening (s17) → B6: Disjunctive cut strengthening and tightened-M cut placement #13. Kronqvist & Misener, OptEng 2021 — no OSS implementation anywhere; plus tightened-M cut placement (needs B1).
- B7. LD-SDA research extensions +
gdpopt.ldbd→ B7: LD-SDA research extensions and gdpopt.ldbd #14 (research); dispatch/vestigial bugs filed upstream: GDPopt meta-solver cannot dispatch to LDSDA (missing from _supported_algorithms) Pyomo/pyomo#4015. LD-BD per Liñán & Ricardez-Sandoval, AIChE J 2023. - B8. Convexify-then-LOA ("Ruiz way") (s10) → B8: Convexify-then-LOA for nonconvex disjuncts (the "Ruiz way") #15. Per-disjunct convexification feeding LOA; middle ground to GLOA.
- B9. Meta-solver honesty → filed upstream: GDPopt relative_gap() raises ZeroDivisionError near zero primal bounds (misplaced epsilon guard) Pyomo/pyomo#4016 (
relative_gapbug) and GDPopt docstring advertises unimplemented algorithms and omits implemented ones Pyomo/pyomo#4017 (phantom docstring features). A third audit find, thegdpopt.enumeratebounds crash, is gdpopt.enumerate raises bare TypeError for unbounded or float-bounded integer variables Pyomo/pyomo#4018.
C. MindtPy
- C1. Single tree on modern interfaces (s16) → filed upstream: MindtPy: support single-tree LP/NLP B&B with APPSI / contrib.solver persistent interfaces Pyomo/pyomo#4020 (APPSI callback exists; threads=1; solution-pool rewrite).
- C2. AD-based cut generation via PyNumero (s16) → C2: MindtPy AD-based cut generation via PyNumero #16. Numeric Jacobian rows → cut expressions; unify with the existing grey-box AD path.
- C3. Convexification/preprocessing layer (s16) → C3: Convexification/preprocessing layer for MindtPy and GDPopt #17. Coramin integration vs MC++ refresh vs in-tree FBBT/OBBT; home for B8 machinery.
- C4. Solver-interface QoL → C4: MindtPy solver-interface quality-of-life fixes #18. HiGHS QP routing, grey-box single tree, cyipopt duals, solution-pool surprise, fragile bound patching.
- C5. Community-detection decomposition — DeCODe-OA (s14–15) → C5: Community-detection-driven decomposition (DeCODe-OA) #19. Connect
contrib.community_detectionto OA/Benders; DECOA/DeCODe never landed as maintained software.
D. Ecosystem & papers
- D1. GDPlib refresh + archival paper + benchmark harness (s11) → D1: GDPlib refresh, archival citation paper, benchmark harness #20.
- D2. Archival MindtPy 2.0 software paper → D2: Archival MindtPy 2.0 software paper #21. Docs still cite PSE 2018; no consolidated citation for the FP/ROA/single-tree generation.
- D3. Documentation gaps → D3: Documentation gaps (GDP solving guidance) #22. LD-SDA vs enumerate guidance, disjunctive bounds,
exact_hull_quadraticcaveats.
Suggested first wave
Small & well-scoped (all upstream, PR-ready): Pyomo#4014 (revive disjunctive bounds), Pyomo#4015 (LDSDA dispatch), Pyomo#4016 (one-line gap fix), Pyomo#4017 (docstring), Pyomo#4018 (enumerate errors). Then fork-side #3 (A1) as the first transformation project.
Paper-sized projects: #4 (A2), #5 (A3), #23 (B3), #12 (B5), #13 (B6), #19 (C5) — each pairs an implementation with a publishable contribution; A2/A3/B6 are unimplemented in any OSS package per the August 2026 literature check.
Key references
- Kronqvist, Bernal Neira & Grossmann, 50 years of mixed-integer nonlinear and disjunctive programming, EJOR 2025 — current state-of-the-art survey.
- Chen et al., Pyomo.GDP: an ecosystem for logic based modeling and optimization development, OptEng 2022.
- Perez, Joshi & Grossmann, DisjunctiveProgramming.jl, JuliaCon Proc. 2023 — feature-parity checklist for A8.
- Full per-item citations in the linked issues.
Activity