fix(sqlalchemy-spanner): resolve compliance and migration failures with Alembic 1.20 and SQLAlchemy 2.1

[chalmerlowe]

Fixes #18580

Problem

Recent upstream releases of Alembic (1.20.0) and SQLAlchemy (2.1.3) caused failures in the sqlalchemy-spanner compliance and migration test suites:

1. Alembic 1.20+ dropped support for SQLAlchemy 1.4: alembic 1.20.0 requires SQLAlchemy>=2.0. In the compliance_test_14 Nox session, .[tracing] was installed first (pulling in SQLAlchemy 2.1 and Alembic 1.20) before force-reinstalling sqlalchemy>=1.4,<2.0. This left alembic 1.20.0 paired with sqlalchemy 1.4.54, causing an ImportError (cannot import name '_NoneName' from 'sqlalchemy.sql.base') when importing the Spanner dialect. Additionally, migration_test previously installed sqlalchemy>=1.4,<2.0 before calling _migration_test, which subsequently ran session.install("pytest", "alembic") and upgraded SQLAlchemy back to 2.x.
2. Missing optional dependency guard: Dialect failed on import if Alembic was missing or incompatible, even for customers only using SQLAlchemy Core or ORM.
3. SQLAlchemy 2.1 removed sqlalchemy.testing.suite.test_deprecations: tests/test_suite_20.py imported sqlalchemy.testing.suite.test_deprecations unconditionally at the top of the file, raising a ModuleNotFoundError when running compliance_test_20 against SQLAlchemy 2.1+.
4. SQLAlchemy 2.1 types.Float MRO change: Upstream decoupled types.Float from types.Numeric (it now inherits from NumericCommon), causing ComponentReflectionTest.test_get_columns intersection checks to fail.
5. SQLAlchemy 2.1 schema & JSON test additions: Upstream added HasTableTest.test_has_multi_table_schema and JSONTest.test_index_cross_casts. Cloud Spanner GoogleSQL does not support user-defined schema namespaces (TABLE_SCHEMA = '') or non-string return types from JSON_VALUE, requiring skips matching existing tests in those classes.

Solution

• Guard the Alembic import in sqlalchemy_spanner.py: Wrap the from alembic.ddl.base import ... block in a try ... except ImportError: statement and gate the @compiles hooks behind HAS_ALEMBIC_INSTALLED. This ensures customers who only use SQLAlchemy Core or ORM (without running Alembic database migrations) can still import the Spanner dialect even if Alembic is missing or mismatched in their environment.
• Pin alembic<1.20.0 for SQLAlchemy 1.4 test sessions and clean up migration_test in noxfile.py:
  • Add alembic<1.20.0 to SQLALCHEMY_14_DEPENDENCIES and install .[tracing] and *SQLALCHEMY_14_DEPENDENCIES in a single session.install step in compliance_test_14.
  • Parametrize migration_test with paired ("python", "extra_dependencies") values for SQLAlchemy 1.4 and 2.0+, and extract the shared migration workflow into a plain _run_migration_test helper called by both migration_test and system.
• Update compliance suite in tests/test_suite_20.py for SQLAlchemy 2.1:
  • Wrap from sqlalchemy.testing.suite.test_deprecations import * in a try ... except ModuleNotFoundError: block.
  • Include types.Float in ComponentReflectionTest.test_get_columns generic type intersection check.
  • Mark HasTableTest.test_has_multi_table_schema and JSONTest.test_index_cross_casts as skipped on Spanner with explanatory reasons matching sibling tests.
• Add unit test coverage in tests/unit/test_alembic.py: Verify that sqlalchemy_spanner imports cleanly and sets HAS_ALEMBIC_INSTALLED = False when alembic.ddl.base is unavailable.

Notes for Reviewers

• In tests/unit/test_alembic.py, test_dialect_import_without_alembic executes the module spec into an isolated module object (importlib.util.module_from_spec) inside mock.patch.dict(sys.modules, {"alembic.ddl.base": None}) rather than calling importlib.reload(sqlalchemy_spanner). Reloading the module in-place would recreate SpannerIdentifierPreparer with a new class identity in sqlalchemy_spanner.__dict__, breaking other tests that imported SpannerDialect prior to the reload.

[gemini-code-assist]

Code Review

This pull request makes the Spanner dialect for SQLAlchemy more robust by guarding Alembic imports, allowing the dialect to be imported cleanly even if Alembic is not installed or if there is a version mismatch. It wraps Alembic-specific DDL compilation overrides under a conditional check, updates the Nox test suite to run migration tests across both SQLAlchemy 1.4 and 2.0 environments, guards deprecated test imports for SQLAlchemy 2.1+ compatibility, and adds a unit test to verify import behavior when Alembic is missing. There are no review comments, so no additional feedback is provided.

[daniel-sanche]

Do you think we need to add any version guards to setup.py? Right now, there are no upper-bound limits, and alembic doesn't have any version pin at all. It seems like that could lead to more versioning issues in the future, if a major update comes out and breaks things overnight

[chalmerlowe]

RESOLVED
We added version bounds.

[daniel-sanche]

This feels misleading, since alembic is a required dependency, so we should be able to trust that it was installed. It seems like the ImportError is actually thrown here when the environment has a version compatibility mismatch?

I'm a little confused about how that would happen in practice, and how we should best guard against it. Because we should be able to trust the package managers to sort this out for us

In the compliance_test_14 Nox session, .[tracing] was installed first (pulling in SQLAlchemy 2.1 and Alembic 1.20) before force-reinstalling sqlalchemy>=1.4,<2.0. This left alembic 1.20.0 paired with sqlalchemy 1.4.54, causing an ImportError (cannot import name '_NoneName' from 'sqlalchemy.sql.base') when importing the Spanner dialect.

Is this really a problem with the package, or do we just have a broken test environment? Maybe we just need to solve the contradictions in our nox installations?

Or, should we change alembic to an optional dependency, and keep this fall-back code?

[chalmerlowe]

@daniel-sanche

You are correct that package managers prevent this mismatch in normal customer installs and our nox script had some long standing inefficiencies that broke due to updates in upstream dependencies. However, I would like to keep the guard as defensive decoupling:

• so the dialect doesn't hard-crash if Alembic is ever omitted (e.g. via --no-deps)
• AND more importantly, in anticipation of making Alembic an optional extra in the future, much as it is in sqlalchemy-bigquery.

See: sqlalchemy-spanner Make alembic an optional extra dependency

In the short term we will add version upper bounds (sqlalchemy<3.0.0, alembic<2.0.0) in setup.py.

Note

For context, it appears that alembic was originally installed as a hard dependency because someone put alembic into dependencies purely because sqlalchemy_spanner.py had unconditional @compiles decorators that imported from alembic.ddl.base at the top level. That code forced the packaging requirement, rather than the packaging requirement dictating the code.