jdbc4py — Operator Guide¶

This guide covers production deployment, security hardening, and operational configuration for jdbc4py. It is intended for infrastructure engineers and platform operators rather than application developers.

1. JDBC4PY_DRIVER_PATHS Security Model¶

What the variable does¶

JDBC4PY_DRIVER_PATHS is an operator-controlled environment variable that extends the JDBC driver search path. It accepts a colon-separated list (on Unix) or semicolon-separated list (on Windows) of filesystem locations. Each entry is treated as either:

A directory — jdbc4py scans it for *.jar files matching the target engine's glob patterns (e.g. postgresql-*.jar).
A single .jar file — jdbc4py adds it to the JVM classpath directly.

# Unix
export JDBC4PY_DRIVER_PATHS=/opt/jdbc-drivers:/opt/custom/myengine-2.0.jar

# Windows
set JDBC4PY_DRIVER_PATHS=C:\jdbc-drivers;C:\custom\myengine-2.0.jar

Why this variable is privileged¶

Every path in JDBC4PY_DRIVER_PATHS is loaded unconditionally onto the JVM classpath at startup. This is semantically equivalent to granting the process java -cp <your-jar> permissions. An attacker who can write to this variable can execute arbitrary code inside the JVM process by supplying a crafted JAR.

Treat JDBC4PY_DRIVER_PATHS as a root-equivalent configuration value.

Validation rules¶

jdbc4py validates every entry before adding it to the search path. Entries that fail validation are silently skipped with a WARNING-level log message; they never raise an exception (to preserve availability). Log the startup output and alert on unexpected JDBC4PY_DRIVER_PATHS warnings.

Rule	Reason
Entry must be an absolute path	Relative paths are resolved against the working directory, which may be attacker-controlled in some deployment models.
Entry must exist on disk at startup	A path that does not exist may be a sign of misconfiguration or a placeholder left by a failed deployment.
If entry is a file, it must end in `.jar`	Only Java Archive files are valid classpath entries. Rejecting other file types prevents accidental or malicious classpath poisoning with non-JAR payloads.
Entry must not contain glob metacharacters (`*`, `?`, `[`, `]`)	Glob expansion is never applied to this variable — metacharacters indicate a misconfiguration or injection attempt.

Example log messages emitted for rejected entries:

WARNING jdbc4py.drivers: JDBC4PY_DRIVER_PATHS entry '/opt/*.jar' contains glob
    metacharacters and was rejected for security reasons.

WARNING jdbc4py.drivers: JDBC4PY_DRIVER_PATHS entry 'relative/path' is not an
    absolute path and was rejected.  Use an absolute path.

WARNING jdbc4py.drivers: JDBC4PY_DRIVER_PATHS entry '/missing/driver.jar' does
    not exist and was skipped.

WARNING jdbc4py.drivers: JDBC4PY_DRIVER_PATHS entry '/scripts/setup.sh' is a
    file but does not end in .jar and was rejected.

Deployment guidance¶

Container / Kubernetes¶

Set the variable at the infrastructure layer, not in application code:

# Kubernetes Deployment
env:
  - name: JDBC4PY_DRIVER_PATHS
    valueFrom:
      secretKeyRef:
        name: jdbc-driver-config
        key: driver_paths

Avoid allowing application pods to write to the environment of sibling pods. In multi-tenant clusters, ensure that no tenant workload can set or override JDBC4PY_DRIVER_PATHS for another tenant's process.

Systemd service¶

[Service]
Environment=JDBC4PY_DRIVER_PATHS=/opt/jdbc-drivers

Do not use EnvironmentFile= pointing to a user-writable path.

CI / test environments¶

If you supply custom JARs in tests via JDBC4PY_DRIVER_PATHS, use absolute paths in your CI matrix configuration and never derive them from user input.

Monitoring¶

Configure your log aggregation to alert on:

jdbc4py.drivers WARN.*JDBC4PY_DRIVER_PATHS

Any warning here at startup indicates either a misconfiguration that should be corrected before the process handles traffic, or an attempted injection that should trigger a security incident.

2. CLOB / Large LOB Configuration¶

The cap¶

The Java helper (JdbcBatchReader/JdbcArrowReader) imposes an upper bound on the number of characters read from a CLOB or large text column per row. The default is 64 million characters (≈128 MB UTF-16 on Java heap).

If a CLOB value exceeds the cap, it is silently truncated and a one-time WARNING is emitted to the JVM log:

jdbc4py CLOB truncation: value exceeded <N> chars; truncating. Set
-Djdbc4py.clob.max_chars=<N> to adjust.  (further truncations will not be
logged)

Adjusting the cap¶

Pass the JVM system property at JVM startup. The JVM can only be started once per process, so this must be set before the first jdbc4py.connect() call.

import jpype
import jpype.imports

# Start JVM with a higher cap (256 M chars ≈ 512 MB heap)
jpype.startJVM(
    "-Djdbc4py.clob.max_chars=268435456",
)

import jdbc4py
conn = jdbc4py.connect(engine="oracle", ...)

Alternatively, export the property via _JAVA_OPTIONS (affects all JVM processes — use carefully in production):

export _JAVA_OPTIONS="-Djdbc4py.clob.max_chars=268435456"

Capacity planning¶

Cap (chars)	Approximate Java heap per CLOB column per row
64 M (default)	~128 MB
16 M	~32 MB
256 M	~512 MB

For batch reads (read_sql_arrow), the cap applies per cell, not per batch. With batch_size=1000 rows and a 64 M-char cap, the worst-case allocation is 1000 × 128 MB = 128 GB — this would OOM. If you are reading wide CLOB tables, lower the cap or reduce the batch size.

3. Connection Pool Tuning¶

Key parameters¶

Parameter	Default	Description
`min_size`	1	Connections opened at startup and kept warm
`max_connections`	—	Hard cap on total connections; takes precedence over `max_size`
`max_size`	10	Alias for `max_connections`
`max_idle_seconds`	300	Evict a connection idle longer than this
`max_age_seconds`	3600	Recycle a connection older than this
`checkout_timeout`	30 s	How long a thread waits for a connection before raising `OperationalError`
`validate_on_checkout`	`True`	Run a ping query before handing out a connection
`validate_on_return`	`False`	Run a ping query when a connection is returned to the pool

Long-lived processes¶

For services that run for days or weeks, use max_age_seconds to recycle connections before they go stale (validate_on_checkout is already on by default):

pool = jdbc4py.ConnectionPool(
    engine="postgres",
    host="db.prod",
    database="analytics",
    max_connections=10,
    max_age_seconds=1800,       # recycle after 30 minutes
    validate_on_checkout=True,  # ping before use (default)
)

Circuit breaker¶

The pool maintains a circuit breaker that opens after repeated connection failures. While open, checkout attempts fail fast with OperationalError and only one recovery probe is let through per cooldown window (the cooldown doubles on each consecutive failure, up to 30 s). The breaker resets once a connection opens successfully. Monitor for circuit-breaker log messages.

4. JVM Thread Management¶

Thread attachment¶

Every Python thread that makes a JDBC call must be attached to the JVM. jdbc4py handles this automatically for:

Threads created by AsyncQuery (via ThreadPoolExecutor)
The deferred-close daemon thread spawned by AsyncQuery.close() when cancellation times out
Connection.__del__ (on GC finalizer threads)

If you spawn raw Python threads that call jdbc4py APIs directly, the JPype library will attempt auto-attachment. If it does not (older JPype versions), you will see a RuntimeError: Failed to attach thread to JVM. Upgrade to JPype ≥ 1.5 or attach manually:

import jpype
from jdbc4py.jvm import JVMManager

JVMManager._attach_thread(jpype)
try:
    with conn.cursor() as cur:
        cur.execute("select 1")
finally:
    JVMManager._detach_thread(jpype)

GC finalizer safety¶

Connection.__del__ and AsyncQuery.__del__ are designed to be safe on GC finalizer threads: they check jpype.isJVMStarted(), attach to the JVM if needed, and silently skip cleanup if attachment fails rather than risking a SIGSEGV from an unattached JNI call.

Do not rely on __del__ for deterministic cleanup. Always use conn.close() or the context-manager (with jdbc4py.connect(...) as conn:).

5. Credential Handling¶

Automatic redaction¶

All exception messages and log entries pass through redact_jdbc_url(), which replaces credential values with *** before they leave the jdbc4py boundary. The following key names are redacted:

password, passwd, pwd, secret, token, credential, apikey, api_key, authtoken, auth_token, bearer_token, oauth_token, access_token, refresh_token, client_secret, app_secret, private_key, access_key, ssl_key, ssl_cert, aws_secret_access_key, aws_access_key_id, db_password, db_passwd, connection_password

JDBC URL credentials¶

Never embed credentials directly in a JDBC URL passed to a log aggregator. jdbc4py redacts them from exception messages, but the raw URL is stored in ResolvedDriver.jdbc_url and may appear in tracebacks or repr() output from third-party exception handlers.

Prefer credential-free URLs with a JDBC Properties object or your database's IAM/Kerberos integration.

6. Observability Checklist¶

Enabling logging¶

jdbc4py logs through the standard library logging module under the jdbc4py namespace (jdbc4py.connection, jdbc4py.pool, jdbc4py.jvm, …). Following library convention, it attaches a NullHandler to the top-level jdbc4py logger, so it emits nothing until your application configures logging:

import logging

logging.basicConfig(level=logging.INFO)            # your app's handlers
logging.getLogger("jdbc4py").setLevel(logging.DEBUG)   # turn jdbc4py up/down

Levels: DEBUG covers connection lifecycle, query timing, pool checkout/return, and JVM thread attach/detach; WARNING covers operational problems worth alerting on (below). jdbc4py never calls basicConfig() or attaches stream handlers itself — handler and level policy stay entirely with your application.

Alert on these log patterns in production:

Logger	Level	Pattern	Meaning
`jdbc4py.drivers`	`WARNING`	`JDBC4PY_DRIVER_PATHS`	Invalid driver path entry — misconfiguration or injection attempt
`jdbc4py.pool`	`WARNING`	`eviction thread did not stop`	Pool shutdown hung — investigate connection leak
`jdbc4py.async_query`	`WARNING`	`deferred cursor close failed`	Background cleanup error after query cancellation
`jdbc4py.async_query`	`WARNING`	`query still running after`	Query did not respond to cancel within 300 s
`jdbc4py.connection`	`WARNING`	`setQueryTimeout`	Driver rejected timeout hint — queries may run unbounded
Java `jdbc4py` logger	`WARNING`	`CLOB truncation`	CLOB data was silently truncated — adjust `jdbc4py.clob.max_chars`