Architecture

Overview

jdbc4py embeds a JVM inside the Python process using JPype. When you call connect(), jdbc4py starts the JVM (if it hasn't been started already), loads your JDBC driver JAR, and opens a java.sql.Connection. Everything from there goes through JDBC.

There's no socket, no subprocess, no protocol reimplementation. The Python objects you work with are thin wrappers around live Java objects accessed via JNI.

Python application
       │
       ▼
jdbc4py API  (connect, ConnectionPool, read_sql, …)
       │
       ├── DBAPI layer     cursor.py, connection.py
       ├── Type layer      types.py
       ├── DataFrame layer dataframe.py
       │
       ▼
JVMManager  (jvm.py — process-global singleton)
       │
       ├── JPype bridge
       ├── jdbc4py-helper.jar   (batch reader, Arrow IPC exporter)
       └── JDBC driver JARs
              │
              ▼
          Database

Package Layout

src/jdbc4py/
  __init__.py        public API surface
  connection.py      Connection, connect()
  cursor.py          Cursor (DBAPI)
  types.py           parameter binding, result conversion
  jvm.py             JVMManager, Java runtime detection
  drivers.py         driver discovery and JAR resolution
  exceptions.py      DBAPI exception hierarchy, credential redaction
  pool.py            ConnectionPool, PooledConnection
  async_query.py     run_query_async()
  dataframe.py       Arrow → pandas / polars / duckdb adapters
  write.py           Arrow / DataFrame insert path
  values.py          YearMonthInterval and other value types
  cloud.py           connect_snowflake/databricks/spanner/bigquery()
  aio.py             asyncio facade (AsyncConnection/AsyncCursor)
  sqlalchemy_dialect.py  +jdbc4py SQLAlchemy dialects (entry points)

java/jdbc4py-helper/
  src/main/java/io/jdbc4py/helper/
    JdbcBatchReader.java     column-major batch extraction
    JdbcArrowReader.java     ResultSet → Arrow IPC export
    JdbcArrowStreamer.java   async prefetch pipeline

drivers/
  postgres/ mysql/ mariadb/ mssql/ oracle/ clickhouse/ trino/
  snowflake/ databricks/ spanner/ bigquery/ sqlite/ sybase/ hana/

JVM Lifecycle

The JVM starts once per process and never restarts. jpype.startJVM is a one-shot operation — once the classpath is frozen you can't add JARs to a running JVM. This is a fundamental JNI constraint.

JVMManager in jvm.py manages this as a process-global singleton:

• Protected by threading.RLock so multiple threads can call ensure_started() concurrently without double-starting the JVM
• Detects Java via JAVA_HOME first, then falls back to PATH; validates that the runtime is Java 11 or newer
• Starts with convertStrings=False — JString → str conversions are always explicit, preventing silent type-coercion bugs
• Raises JVMError on failure with a description of what went wrong and what Java version was found

If you call connect() after the JVM is running and pass new JARs that aren't already on the classpath, you'll get a JVMError. The fix is to call JVMManager.ensure_started(jars=[...]) once at startup with all your JARs before opening any connections.

Connection

connect() in connection.py does five things in order:

1. Resolves the driver JARs via drivers.py
2. Calls JVMManager.ensure_started(jars) — starts the JVM or validates classpath compatibility
3. Builds a JDBC URL from engine/host/port/database/schema, or uses a raw jdbc_url= if provided
4. Opens the connection via java.sql.DriverManager.getConnection(url, properties)
5. Sets autoCommit=False (the DBAPI default)

Concurrency Model

Component	Thread safety
JVMManager	Thread-safe — protected by RLock
ConnectionPool	Thread-safe — share freely across threads
Connection	Not thread-safe — one per thread
Cursor	Not thread-safe — one per thread
cancel()	Safe to call from any thread

The rule: one connection per thread, borrowed from the pool and returned when done. cancel() is the one deliberate exception — it's designed to be called from a different thread to interrupt a running query.

Each Python thread that makes JNI calls must be attached to the JVM. Recent JPype versions handle this automatically. The explicit JVMManager._attach_thread() calls throughout the codebase are compatibility shims for older installs.

Exception Hierarchy

Error
  InterfaceError
    JVMError
    DriverNotFoundError
  DatabaseError
    DataError
    OperationalError
      QueryCancelledError
    IntegrityError
    InternalError
    ProgrammingError
    NotSupportedError

translate_exception() maps Java SQLException to the appropriate DBAPI class using SQL state codes:

SQL state prefix	Python exception
08xx	OperationalError (connection failure)
22xx	DataError (bad value)
23xx	IntegrityError (constraint violation)
42xx	ProgrammingError (bad SQL)
57014, HY008	QueryCancelledError

Security

A few things jdbc4py does by default:

• All parameterized queries go through PreparedStatement — no string interpolation into SQL
• Passwords and tokens are redacted from all exception messages and logs (see operator-guide.md for the full list of redacted keys)
• Statement timeouts are configurable via query_timeout on connect() or per-execute
• Long-running queries can be cancelled from any thread without closing the connection