Skip to content

Guarantees and Concurrency

Relevant source files

Tenant Switching Guarantees

A tenant switch is transactional:

validate -> prepare -> snapshot -> apply context -> connect -> verify -> commit

Two invariants follow from that:

  1. At every switch boundary, ConsoleKit is either still on the previous tenant or completely on the new one. The new tenant only becomes current at the final commit, so once switch_tenant returns - successfully or not - you are never left with the context on tenant B, SQL on B and Redis still on A.

This is a guarantee about boundaries, not about instants. A switch applies the context, then each backend in turn, then verifies. While it is running there is genuinely a window in which some backends have moved and others have not, and for the process-global backends below that window is visible to other threads. See Concurrency and Isolation. 2. A successful connection is not enough. After connecting, each backend is read back and must report the tenant that was asked for. A handler that silently ignored the switch, or a foreign writer that moved the backend between connect and verify, fails the switch instead of quietly serving another tenant's data.

Be precise about what this buys you: verification is a local read of the handle ConsoleKit just wrote, not a round trip that interrogates the server. It catches a write that did not take. It cannot catch a database.yml entry that points shard_acme at another tenant's server. It is an attestation, not a proof of provenance.

If anything fails, every touched component is restored - the context attributes and all four backends - and the switch raises. Each component is attempted even when an earlier restore fails, so one broken backend cannot strand the rest.

Errors

ConsoleKit.switch_tenant(:globex)
# ConsoleKit::TenantSwitchError:
#   Failed to switch tenant from :acme to :globex (MongoDB):
#   MongoDB verification failed. Expected "globex_db", got "acme_db"
#
#   Previous tenant state was restored successfully.

A rollback can itself fail, and that is reported separately rather than replacing the root cause:

# ConsoleKit::TenantSwitchError:
#   Failed to switch tenant from :acme to :globex (MongoDB):
#   MongoDB verification failed. Expected "globex_db", got "acme_db"
#
#   WARNING: rollback did not fully succeed:
#     - SQL: shard registry offline

TenantSwitchError exposes #original_error, #rollback_failures and #rollback_succeeded?. The exception hierarchy, all under ConsoleKit::Error:

Error Raised when
ConfigurationError configuration is missing, malformed or unusable
TenantNotFoundError the tenant key is not in the configured tenant map
ConnectionError a backend could not be connected or inspected
ConnectionVerificationError a backend connected but points at the wrong tenant
UnsupportedBackendError the installed client cannot support the requested operation
TenantSwitchError a switch failed; carries the root cause and any rollback failures
RollbackError restoring previous state failed

Credentials are scrubbed from error messages, diagnostic rows and console output: connection URIs, key=value and key => value fragments, Authorization: Bearer <token> style auth headers, bare password <value> phrases and for user <name> principals. Hostnames and ports are deliberately kept - they are not secrets, and removing them would gut the diagnostic value of a connection error.

Scrubbing is shape-matching over strings ConsoleKit did not produce, so treat it as defence in depth rather than a boundary: it errs toward redacting, and a client version emitting a shape it has not seen could still get through. Do not put secrets anywhere they could be logged in the first place.

Concurrency and Isolation

Read this before using ConsoleKit anywhere other than a console.

ConsoleKit invents no isolation of its own. It writes through whatever handle your client library gives it, so isolation is exactly as good as that handle:

Component Isolated per thread?
ConsoleKit's own tenant state Yes
Mongoid overrides (Mongoid::Threaded) Yes
ActiveRecord native shard path (connecting_to) Yes - fiber-local
Your context object Only if it stores per-thread (e.g. ActiveSupport::CurrentAttributes)
ActiveRecord establish_connection fallback No - replaces a process-wide pool
Redis via Redis.current (redis-rb 4) No - process-global singleton
Elasticsearch index_name_prefix No - one process-wide attribute

Where isolation does not exist, the last writer wins for the whole process. The two backends whose isolation depends on the installed client report it at runtime:

ConsoleKit::Connections::RedisConnectionHandler.new(ctx).isolation_model
# => :scoped, :process_global, :none, or :unknown

ConsoleKit::Connections::ElasticsearchConnectionHandler.new(ctx).isolation_model
# => :process_global (always)

Both also answer thread_isolated?. The SQL and Mongoid handlers do not define these - their isolation depends on which code path is taken, as the table above shows.

Redis on redis-rb 4 prints a one-time warning that DB selection is process-wide. Elasticsearch warns, naming both prefixes, when live threads disagree about the prefix.

switch_tenant is a console tool, not a per-request multi-tenancy mechanism. Because it mutates process-global backend state, calling it from a request or a background job will change the tenant for every other thread in that process. For real isolation, run one tenant per process.

ConsoleKit.with_tenant and ConsoleKit.current_tenant are safe to read anywhere; it is the switching itself that is process-affecting.

Observability

ConsoleKit emits timing and counts through a small internal hook. There is no external dependency - wire it to whatever you already use:

ConsoleKit::Instrumentation.subscribe do |name, duration_ms, payload|
  Rails.logger.info("#{name} #{duration_ms}ms #{payload.inspect}")
end

ConsoleKit::Instrumentation.counts
# => { "console_kit.tenant_switch" => 3, "console_kit.rollback" => 1, ... }

Subscribers receive timed events for tenant switches, per-backend connect and verify, and diagnostics. Rollbacks, verification failures, dropped handlers and similar conditions are counted in counts only. Payloads carry tenant keys and backend names - never credentials. The full list is in the Public API reference.