oslodb(1)

OSLODB(1)

oslo.db

OSLODB(1)

NAME

oslodb - oslo.db 12.3.1

The oslo.db (database) handling library, provides database connectivity to different database backends and various other helper utils.

INSTALLATION

At the command line:

$ pip install oslo.db

You will also need to install at least one SQL backend:

$ pip install PyMySQL

Or:

$ pip install psycopg2

Or:

$ pip install pysqlite

Using with PostgreSQL

If you are using PostgreSQL make sure to install the PostgreSQL client development package for your distro. On Ubuntu this is done as follows:

$ sudo apt-get install libpq-dev
$ pip install psycopg2

The installation of psycopg2 will fail if libpq-dev is not installed first. Note that even in a virtual environment the libpq-dev will be installed system wide. If you would like to contribute to the development of oslo's libraries, first you must take a look to this page:

https://specs.openstack.org/openstack/oslo-specs/specs/policy/contributing.html

HOW TO CONTRIBUTE

If you would like to contribute to the development of OpenStack, you must follow the steps in this page:

https://docs.openstack.org/infra/manual/developers.html

Once those steps have been completed, changes to OpenStack should be submitted for review via the Gerrit tool, following the workflow documented at:

https://docs.openstack.org/infra/manual/developers.html#development-workflow

Pull requests submitted through GitHub will be ignored.

Bugs should be filed on Launchpad, not GitHub:

https://bugs.launchpad.net/oslo.db

How to run unit tests

oslo.db (as all OpenStack projects) uses tox to run unit tests. You can find general information about OpenStack unit tests and testing with tox in wiki.

oslo.db tests use PyMySQL as the default MySQL DB API driver (which is true for OpenStack), and psycopg2 for PostgreSQL. pip will build these libs in your venv, so you must ensure that you have the required system packages installed for psycopg2 (PyMySQL is a pure-Python implementation and so needs no additional system packages). For Ubuntu/Debian they are python-dev, and libpq-dev. For Fedora/CentOS - gcc, python-devel and postgresql-devel.

The oslo.db unit tests system allows to run unittests on real databases. At the moment it supports MySQL, PostgreSQL and SQLite. For testing on a real database backend you need to set up a user openstack_citest with password openstack_citest on localhost (some OpenStack projects require a database named 'openstack_citest' too). Please note, that this user must have permissions to create and drop databases. If the testing system is not able to connect to the backend, tests on it will be skipped.

For PostgreSQL on Ubuntu you can create a user in the following way:

sudo -u postgres psql
postgres=# create user openstack_citest with createdb login password


           'openstack_citest';

For MySQL you can use the following commands:

mysql -u root
mysql> CREATE USER 'openstack_citest'@'localhost' IDENTIFIED BY


       'openstack_citest';
mysql> GRANT ALL PRIVILEGES ON * . * TO 'openstack_citest'@'localhost';
mysql> FLUSH PRIVILEGES;

See the script tools/test-setup.sh on how the databases are set up excactly in the OpenStack CI infrastructure and use that for your set up.

Alternatively, you can use pifpaf to run the unit tests directly without setting up the database yourself. You still need to have the database software installed on your system. The following tox environments can be used:

tox -e py27-mysql
tox -e py27-postgresql
tox -e py34-mysql
tox -e py34-postgresql
tox -e py27-all
tox -e py34-all

The database will be set up for you locally and temporarily on each run.

Another way is to start pifpaf manually and use it to run the tests as you wish:

$ eval `pifpaf -g OS_TEST_DBAPI_ADMIN_CONNECTION run postgresql`
$ echo $OS_TEST_DBAPI_ADMIN_CONNECTION
postgresql://localhost/postgres?host=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpMGqN8C&port=9824
$ tox -e py27
[…]
$ tox -e py34
[…]
# Kill pifpaf once you're done
$ kill $PIFPAF_PID

USING OSLO.DB

Usage

To use oslo.db in a project:

Session Handling

Session handling is achieved using the oslo_db.sqlalchemy.enginefacade system. This module presents a function decorator as well as a context manager approach to delivering session.Session as well as Connection objects to a function or block.

Both calling styles require the use of a context object. This object may be of any class, though when used with the decorator form, requires special instrumentation.

The context manager form is as follows:

from oslo_db.sqlalchemy import enginefacade
class MyContext(object):


    "User-defined context class."
def some_reader_api_function(context):


    with enginefacade.reader.using(context) as session:


        return session.query(SomeClass).all()
def some_writer_api_function(context, x, y):


    with enginefacade.writer.using(context) as session:


        session.add(SomeClass(x, y))
def run_some_database_calls():


    context = MyContext()


    results = some_reader_api_function(context)


    some_writer_api_function(context, 5, 10)

The decorator form accesses attributes off the user-defined context directly; the context must be decorated with the oslo_db.sqlalchemy.enginefacade.transaction_context_provider() decorator. Each function must receive the context argument:

from oslo_db.sqlalchemy import enginefacade
@enginefacade.transaction_context_provider
class MyContext(object):


    "User-defined context class."
@enginefacade.reader
def some_reader_api_function(context):


    return context.session.query(SomeClass).all()
@enginefacade.writer
def some_writer_api_function(context, x, y):


    context.session.add(SomeClass(x, y))
def run_some_database_calls():


    context = MyContext()


    results = some_reader_api_function(context)


    some_writer_api_function(context, 5, 10)

connection modifier can be used when a session.Session object is not needed, e.g. when SQLAlchemy Core is preferred:

@enginefacade.reader.connection
def _refresh_from_db(context, cache):


    sel = sa.select(table.c.id, table.c.name)


    res = context.connection.execute(sel).fetchall()


    cache.id_cache = {r[1]: r[0] for r in res}


    cache.str_cache = {r[0]: r[1] for r in res}

NOTE:

The context.session and context.connection attributes must be accessed within the scope of an appropriate writer/reader block (either the decorator or contextmanager approach). An AttributeError is raised otherwise.

The decorator form can also be used with class and instance methods which implicitly receive the first positional argument:

class DatabaseAccessLayer(object):


    @classmethod


    @enginefacade.reader


    def some_reader_api_function(cls, context):


        return context.session.query(SomeClass).all()


    @enginefacade.writer


    def some_writer_api_function(self, context, x, y):


        context.session.add(SomeClass(x, y))

NOTE:

Note that enginefacade decorators must be applied before classmethod, otherwise you will get a TypeError at import time (as enginefacade will try to use inspect.getargspec() on a descriptor, not on a bound method, please refer to the Data Model section of the Python Language Reference for details).

The scope of transaction and connectivity for both approaches is managed transparently. The configuration for the connection comes from the standard oslo_config.cfg.CONF collection. Additional configurations can be established for the enginefacade using the oslo_db.sqlalchemy.enginefacade.configure() function, before any use of the database begins:

from oslo_db.sqlalchemy import enginefacade
enginefacade.configure(


    sqlite_fk=True,


    max_retries=5,


    mysql_sql_mode='ANSI'
)

Base class for models usage

from oslo_db.sqlalchemy import models
class ProjectSomething(models.TimestampMixin,


                       models.ModelBase):


    id = Column(Integer, primary_key=True)


    ...

DB API backend support

from oslo_config import cfg
from oslo_db import api as db_api
_BACKEND_MAPPING = {'sqlalchemy': 'project.db.sqlalchemy.api'}
IMPL = db_api.DBAPI.from_config(cfg.CONF, backend_mapping=_BACKEND_MAPPING)
def get_engine():


    return IMPL.get_engine()
def get_session():


    return IMPL.get_session()
# DB-API method
def do_something(somethind_id):


    return IMPL.do_something(somethind_id)

DB migration extensions

Available extensions for oslo_db.migration.

CHANGES

12.3.1

•: Fix default value for wsrep_sync_wait option

12.3.0

Add option for wsrep_sync_wait
skip bandit on oslo_db/tests

12.2.0

Add Python3 antelope unit tests
Imported Translations from Zanata
Fix misuse of assert_has_calls
tests: Define a primary key
tests: Fix compatibility with PostgreSQL 14+
Update master for stable/zed
types: Set 'cache_ok' (redux)

12.1.0

Replace abc.abstractproperty with property and abc.abstractmethod
Deprecate MySQL NDB Cluster Support
trivial: Formatting changes for oslo_db.options

12.0.0

Imported Translations from Zanata
Drop python3.6/3.7 support in testing runtime

11.3.0

Add Python3 zed unit tests
Update master for stable/yoga
tox: Silence output
trivial: Don't emit warnings for our own deprecations
tests: Enable SAWarning warnings
Remove the 'Session.autocommit' parameter
Add missing 'connect' wrapper
Don't call 'begin()' on existing transaction

11.2.0

utils: Remove troublesome utility methods
Update python testing classifier
tests: Restore - don't reset - warning filters

11.1.0

Configure driver for postgres
Add Python3 yoga unit tests
Update master for stable/xena

11.0.0

requirements: Bump sqlalchemy lower constraint
Remove use of Session.begin.subtransactions flag
Don't rely on implicit autocommit
Replace use of 'Engine.execute()'
Don't call mapper() outside of declarative registry
Don't pass kwargs to connection.execute()
Replace use of Executable.execute method
Remove unnecessary warning filter
Replace use of Engine.scalar()
Don't use the 'Row.keys()' method
Don't use dict-style attribute accesses
Don't use plain string SQL statements
Update import of declarative_base()
Replace use of Table.autoload parameter
Replace use of update.values parameter
Replace use of update.whereclause parameter
Replace use of insert.values parameter
Add missing bind argument to calls
Don't pass strings to Connection.execute()
Remove use of MetaData.bind argument
Remove legacy calling style of select()
tests: Enable SQLAlchemy 2.0 deprecation warnings
utils: Deprecate sqlalchemy-migrate-related functions
tests: Enable SADeprecationWarning warnings
tests: Use common base class
tests: Enfeeble 'oslo_db.tests.utils.BaseTestCase'
tests: Clean up base test
Drop checks for IBM DB2
tox: Simplify test running
tests: Remove 'ModelsMigrationsSync.check_foreign_keys'
concurrency: Deprecate 'TpoolDbapiWrapper'
sqlalchemy: Remove checks for older deps

10.0.0

Remove the idle_timeout option
Remove the useless else
types: Set 'cache_ok'
Changed minversion in tox to 3.18.0

9.1.0

Followup of "Added handler for mysql 8.0.19 duplicate key error update"
Added handler for mysql 8.0.19 duplicate key error update
update the pre-commit-hooks version
Replace getargspec with getfullargspec

9.0.0

•: setup.cfg: Replace dashes with underscores

8.6.0

Don't use private API to get query criteria
Remove the sql_max_pool_size option
Fix formatting of release list
move flake8 as a pre-commit local target
Add Python3 xena unit tests
Update master for stable/wallaby
Fix the conflict status with hacking
Remove lower-constraints remnants
remove unicode from code
Use TOX_CONSTRAINTS_FILE
Dropping lower constraints testing
Accommodate immutable URL api
Use TOX_CONSTRAINTS_FILE
Use py3 as the default runtime for tox

8.5.0

Deprecate the 'oslo_db.sqlalchemy.migration_cli' module
Deprecate 'oslo_db.sqlalchemy.migration' module
Imported Translations from Zanata
Adding pre-commit
Add Python3 wallaby unit tests
Update master for stable/victoria

8.4.0

[goal] Migrate testing to ubuntu focal
Bump bandit version

8.3.0

requirements: Drop os-testr
Make test-setup.sh compatible with mysql8

8.2.1

Fix pygments style
Set create_constraint=True for boolean constraint test

8.2.0

Fix hacking min version to 3.0.1
Switch to newer openstackdocstheme and reno versions
Remove the unused coding style modules
Remove translation sections from setup.cfg
Align contributing doc with oslo's policy
Bump default tox env from py37 to py38
Add py38 package metadata
Imported Translations from Zanata
Add release notes links to doc index
Remove use of deprecated LOG.warn
Add Python3 victoria unit tests
Update master for stable/ussuri
Modernize use of table.count() with func.count()

8.1.0

Use unittest.mock instead of third party mock
Update hacking for Python3

8.0.0

Remove 'handle_connect_error'
Drop use of six
Remove final references to mox
Raise minimum SQLAlchemy version to 1.2.0
remove outdated header
reword releasenote for py27 support dropping
Ignore releasenote artifacts files

7.0.0

[ussuri][goal] Drop python 2.7 support and testing
Drop unittest2 usage
tox: Trivial cleanup

6.0.0

Imported Translations from Zanata
gitignore: Add reno artefacts
Convert remaining use of mox
Remove deprecated class DbMigrationError since we already have DBMigrationError

5.1.1

Use regex to compare SQL strings with IN
Bump the openstackdocstheme extension to 1.20
Reduce severity of wrapped exceptions logs to warning
tox: Keeping going with docs
Switch to official Ussuri jobs

5.1.0

•: Update master for stable/train

5.0.2

Add Python 3 Train unit tests
Add libpq-dev to bindep.txt
Use connect, not contextual_connect
Add local bindep.txt

5.0.1

Rollback existing nested transacvtion before restarting
Dropping the py35 testing

5.0.0

docs: Use sphinxcontrib.apidoc for building API docs
Cap Bandit below 1.6.0 and update Sphinx requirement
Replace git.openstack.org URLs with opendev.org URLs

4.46.0

OpenDev Migration Patch
Fix deprecation warnings under py36
Support context function argument as keyword
Removing deprecated min_pool_size
Bump psycopg lower-constraint to 2.7
Update master for stable/stein

4.44.0

exc_filters: fix deadlock detection for MariaDB/Galera cluster
Resolve SAWarning in Query.soft_delete()
Update hacking version

4.43.0

Remove convert_unicode flag
Use template for lower-constraints
Update mailinglist from dev to discuss

4.42.0

Add "is_started" flag to enginefacade
Move warnings to their own module
Clean up .gitignore references to personal tools
Always build universal wheels
Don't quote {posargs} in tox.ini

4.41.1

Fix FOREIGN KEY messages for MariaDB 10.2, 10.3
Imported Translations from Zanata
add lib-forward-testing-python3 test job
add python 3.6 unit test job
import zuul job settings from project-config
Update reno for stable/rocky
Switch to stestr

4.40.0

Rename enginefacade.async to enginefacade.async_
Add release notes to README.rst

4.39.0

Remove most server_default comparison logic
remove sqla_09 test environment
fix tox python3 overrides

4.38.0

Remove stale pip-missing-reqs tox test
Deprecate min_pool_size
List PyMySQL first in installation docs

4.37.0

•: Trivial: Update pypi url to new url

4.36.0

set default python to python3
Improve exponential backoff for wrap_db_retry
uncap eventlet
add lower-constraints job

4.35.0

Add testresources / testscenarios to requirements.txt
Updated from global requirements

4.34.0

Ignore 'use_tpool' option
Remove tox_install.sh and align constraints consumption
Update links in README
Ensure all test fixtures in oslo_db.tests are private
Imported Translations from Zanata
Conditionally adjust for quoting in comparing MySQL defaults
Imported Translations from Zanata
Allow connection query string to be passed separately
Reverse role of synchronous_reader
Imported Translations from Zanata
Update reno for stable/queens
Updated from global requirements
Fix a typo of "transaction" in comment
Updated from global requirements
Updated from global requirements
Updated from global requirements

4.33.0

Updated from global requirements
add bandit to pep8 job
Drop tox-mysql-python job
Use the new PTI for document build
Updated from global requirements

4.32.0

Imported Translations from Zanata
Updated from global requirements
Add requirements.txt for docs builds

4.31.0

Updated from global requirements
Handle deprecation of inspect.getargspec

4.30.0

Remove setting of version/release from releasenotes
Updated from global requirements
Updated from global requirements
Updated from global requirements
Use assertRegex instead of assertRegexpMatches
Zuul: add file extension to playbook path
Remove kwarg retry_on_request in wrap_db_retry
Migrate to zuulv3 - move legacy jobs to project

4.29.0

Use skipTest() method instead of deprecated skip()
Drop MySQL-python dependency from oslo.db
Updated from global requirements
Remove method get_connect_string and is_backend_avail
Test illegal "boolean" values without Boolean datatype
Imported Translations from Zanata

4.28.0

Add new foreign key utility function to utils
Updated from global requirements

4.27.0

Fix pagination when marker value is None
Updated from global requirements
Remove property message for DBInvalidUnicodeParameter and InvalidSortKey
Throw DBMigrationError instead of DbMigrationError
Remove function optimize_db_test_loader in test_base.py
Remove provisioned_engine in class BackendImpl
Remove AutoString* in ndb.py
Remove class InsertFromSelect
Remove class TransactionResource
Remove method provisioned_engine in class Backend
Update the documentation link

4.26.0

Updated from global requirements
Rename idle_timeout to connection_recycle_time
Workaround non-compatible type.adapt() for SQLAlchemy < 1.1
Let others listen to SQLAlchemy errors
Update reno for stable/pike
Replace ndb "auto" types with unified String
Updated from global requirements
Remove deprecation warning when loading tests/sqlalchemy
Replace six.iteritems() with .items()

4.25.0

Log an exception when reconnect-to-disconnected occurs
Don't access connection.info if connection is invalidated
Update URLs according to document migration

4.24.1

update the docs url in the readme
turn on warning-is-error in doc build
switch from oslosphinx to openstackdocstheme

4.24.0

rearrange content to fit the new standard layout
Updated from global requirements
Fix compatibility with SQLAlchemy < 1.1.0
Enable MySQL Storage Engine selection
Updated from global requirements
Updated from global requirements
Using assertIsNone(xxx) instead of assertEqual(None, xxx)

4.23.0

Updated from global requirements
Updated from global requirements

4.22.0

Updated from global requirements
add release note for new warning about missing driver
Raise only DbMigrationError from migrate 'upgrade' method
Warn on URL without a drivername
Updated from global requirements

4.21.1

Updated from global requirements
Add 'save_and_reraise_exception' method when call 'session.rollback()'
Move oslo.context to test-requirements
Attach context being used to session/connection info
Updated from global requirements

4.21.0

Updated from global requirements
Updated from global requirements

4.20.0

•: Remove log translations

4.19.0

Updated from global requirements
Remove deprecated config option sqlite_db
Imported Translations from Zanata
Updated from global requirements

4.18.0

Updated from global requirements
Update test requirement
Establish flush() for "sub" facade contexts
Remove unused logging import
Support facade arguments, unstarted facade for patch_engine()
Repair unused rollback_reader_sessions parameter
Updated from global requirements
Prepare for using standard python tests
Explain paginate_query doesn't provide parameter offset
Updated from global requirements
Update reno for stable/ocata
Coerce booleans to integer values in paginate_query
Remove references to Python 3.4

4.17.0

Modify word "whetever" to "whether"
Updated from global requirements
Add Constraints support
Support packet sequence wrong error
docs: mention that it's possible to use Connection directly
exc_filters: fix deadlock detection for percona xtradb cluster
Replaces uuid.uuid4 with uuidutils.generate_uuid()
Fix marker checking when value is None
Strip prefix `migrate_` in parameter `migrate_engine`

4.16.0

migration: don't assume the mixin use provision
Check if an index on given columns exists

4.15.0

Ensure provision_new_database is True by default
Don't run LegacyBaseClass provision test for unavailable database
SoftDeleteMixin: allow for None values
SoftDeleteMixin: coerce deleted param to be an integer
Show team and repo badges on README
Support MariaDB error 1927
Break optimize_db_test_loader into package and module level
Adjust SAVEPOINT cause test for SQLA 1.1
Updated from global requirements
Restore provisioning of DBs in legacy test base
Updated from global requirements
Updated from global requirements
Enhanced fixtures for enginefacade-based provisioning
utils: deprecate InsertFromSelect properly
Using new style assertions instead of old style assertions
Updated from global requirements
Fix exc_filters for mysql-python
Updated from global requirements
Change assertTrue(isinstance()) by optimal assert
OpenStack typo
Changed the home-page link

4.14.0

standardize release note page ordering
Add a specific exception for 'unknown database' errors
Enable release notes translation
Updated from global requirements
Add DBDataError for "Data too long"
Updated from global requirements
Updated from global requirements
Add additional caution looking for table, info
utils: fix get_unique_keys() when model has no table attached to it
Update reno for stable/newton
Updated from global requirements
Correctly detect incomplete sort_keys passed to paginate_query
Fix DBReferenceError and DBNonExistentTable with new PyMySQL version

4.13.0

•: Updated from global requirements

4.12.0

Updated from global requirements
Link enginefacade to test database provisioning
Display full reason for backend not available
Updated from global requirements
Deprecate argument sqlite_db in method set_defaults

4.11.0

Updated from global requirements
Add test helpers to enginefacade
Add logging_name to enginefacade config

4.10.0

Capture DatabaseError for deadlock check
Add a hook to process newly created engines

4.9.0

Updated from global requirements
Memoize sys.exc_info() before attempting a savepoint rollback
Updated from global requirements
Fix parameters of assertEqual are misplaced
Consolidate pifpaf commands into variables
Updated from global requirements
Updated from global requirements
Fixed unit tests running on Windows
Remove discover from setup.cfg
Add dispose_pool() method to enginefacade context, factory

4.8.0

Updated from global requirements
Set a min and max on the connection_debug option
Set max pool size default to 5
Add support for LONGTEXT, MEDIUMTEXT to JsonEncodedType
tox: add py35 envs for convenience
Deprecate config option sqlite_db for removal
Catch empty value DBDuplicate errors
release notes: mention changes in wrap_db_retry()
Updated from global requirements
api: use sane default in wrap_db_retry()
Imported Translations from Zanata
exc_filters: catch and translate non existent table on drop
exception: make message mandatory in DbMigrationError and deprecates it
Make it possible to use enginefacade decorators with class methods
Updated from global requirements
tests: fix order of assertEqual in exc_filter
exc_filters: catch and translate non existent constraint on drop
Replace tempest-lib dependency with os-testr
Imported Translations from Zanata
Fix typos in comments and docstring
Updated from global requirements
Updated from global requirements
Fix typo: 'olso' to 'oslo'
Repair boolean CHECK constraint detection
api: do not log a traceback if error is not expected
Fix imports in doc
Allow testing of MySQL and PostgreSQL scenario locally
Add support for custom JSON serializer
api: always enable retry_on_request
Remove oslo-incubator related stuff
Updated from global requirements
Updated from global requirements
Remove direct dependency on babel
Imported Translations from Zanata
Add debtcollector to requirements
Fix unit tests failures, when psycopg2 is not installed
Fix server_default comparison for BigInteger
Remove unused sqlite_fk in _init_connection_args call
Updated from global requirements
Fix db_version checking for sqlalchemy-migrate
Correct docstring
Updated from global requirements
Updated from global requirements
Updated from global requirements
Updated from global requirements
Updated from global requirements
Raise DbMigrationError for invalid version
Add new filter for DBDataError exception
Fix spelling mistake
Let enginefacade._TransactionContextManager look for context
Remove sqlalchemy < 1.0.0 compatible layer
Update reno for stable/mitaka
Updated from global requirements
Add tests for float interval values in wrap_db_retry()

4.6.0

Increase the default max_overflow value
Updated from global requirements
add reno for release notes management
Updated from global requirements
Updated from global requirements
Clarify the types for retry_interval args of wrap_db_retry

4.5.0

Updated from global requirements
stop making a copy of options discovered by config generator

4.4.0

exceptions: provide .message attribute for Py3K compatibility
Updated from global requirements
InvalidSortKey constructor change breaks Heat unittests
exception: fix DBInvalidUnicodeParameter error message
exceptions: enhance InvalidSortKey to carry the invalid key
exception: fix InvalidSortKey message
Update translation setup
Updated from global requirements
Add exc_filter for invalid Unicode string
Updated from global requirements
Updated from global requirements
Updated from global requirements

4.3.1

Imported Translations from Zanata
Updated from global requirements
Fix tests to work under both pymsysql 0.6.2 and 0.7.x
Don't log non-db error in retry wrapper

4.3.0

Updated from global requirements
Put py34 first in the env order of tox
Updated from global requirements

4.2.0

Fix comparison of Variant and other type in compare_type
Updated from global requirements
Updated from global requirements
Don't trace DB errors when we're retrying
Updated from global requirements
Remove iso8601 in requirements.txt
Trival: Remove 'MANIFEST.in'

4.1.0

Refactor deps to use extras and env markers
Added allow_async property

4.0.0

Updated from global requirements
Remove python 2.6 classifier
Remove python 2.6 and cleanup tox.ini

3.2.0

Detect not-started _TransactionFactory in legacy
Added method get_legacy_facade() to the _TransactionContextManager
Removed Unused variable 'drivertype'
Updated from global requirements
Imported Translations from Zanata
Updated from global requirements

3.1.0

Updated from global requirements
Updated from global requirements
Add debug logging for DB retry attempt

3.0.0

Fix coverage configuration and execution
Add universal wheel tag to setup.cfg
No need for Oslo Incubator Sync
Updated from global requirements
Correct invalid reference
Imported Translations from Zanata
Use stevedore directive to document plugins
Make readme and documentation titles consistent
Docstring fixes for enginefacade
Fix warnings in docstrings
Autogenerate the module docs
Add config options to the documentation
Add support for pickling enginefacade context objects
Change ignore-errors to ignore_errors
Fix the home-page value with Oslo wiki page
Updated from global requirements
Imported Translations from Zanata

2.6.0

Imported Translations from Transifex
Handle case where oslo_db.tests has not been imported
Updated from global requirements

2.5.0

Updated from global requirements
Imported Translations from Transifex
Updated from global requirements
Move runtime test resources into setup.cfg [extras]
Updated from global requirements

2.4.1

Assume relative revisions belong to alembic
Use correct config key in alembic extension
Fix exception message about unavailable backend

2.4.0

Imported Translations from Transifex
Updated from global requirements
Updated from global requirements
Fix hacking rules and docs job
Imported Translations from Transifex
pagination: enhance sorting of null values
Upgrade and downgrade based on revision existence
Imported Translations from Transifex
Updated from global requirements
Add JSON-encoded types for sqlalchemy

2.3.0

Imported Translations from Transifex
Python 3: Use use_unicode=1 under Python 3
Imported Translations from Transifex
Updated from global requirements
Fix test_migrations on Python 3
Improve failure mode handling in enginefacade

2.2.0

Imported Translations from Transifex
Updated from global requirements
Updated from global requirements
Add mock to test-requirements.txt
Test that concurrent sqlalchemy transactions don't block
Updated from global requirements
Added catching of errors 1047 (Galera) for MySQL oslo db reconnect
Remove outdated tox environments for SQLAlchemy 0.8
Imported Translations from Transifex

2.1.0

Allow projects that use test_models_sync to filter some changes
Updated from global requirements
Add legacy get_sessionmaker() method

2.0.0

Fix sqlalchemy.ModelBase.__contains__() behaviour
Add tox target to find missing requirements
Allow additional exceptions in wrap_db_retry
Remove implicit RequestContext decoration
Add a new ModelBase.items() method
Updated from global requirements
Add oslo.context to requirements.txt
Imported Translations from Transifex

1.12.0

Updated from global requirements
Remove oslo namespace package
Drop use of 'oslo' namespace package
Switch from MySQL-python to PyMySQL
Updated from global requirements
Switch badges from 'pypip.in' to 'shields.io'
Updated from global requirements

1.11.0

Replace utils method with oslo.utils reflection provided one
Implement new oslo.db.sqlalchemy.enginefacade module
Allow to fail instead of skip in DbFixture

1.10.0

Updated from global requirements
Imported Translations from Transifex
Add a keys() method to SQLAlchemy ModelBase
Remove support for Python 3.3
Updated from global requirements
Remove run_cross_tests.sh
Sort model fields using getattr(), not inspect()
Imported Translations from Transifex
Updated from global requirements
Remove pre-SQLAlchemy-0.9.7 compat utilities
Add Python 3 classifiers to setup.cfg

1.9.0

•: Uncap library requirements for liberty

1.8.0

Sanity check after migration
Add filters for DBDataError exception
Add pypi download + version badges
exc_filters: support for ForeignKey error on delete
Standardize setup.cfg summary for oslo libs
Update to latest hacking
Handle CHECK constraint integrity in PostgreSQL
Catch DBDuplicateError in MySQL if primary key is binary
Imported Translations from Transifex
Updated from global requirements
Imported Translations from Transifex
Provide working SQLA_VERSION attribute
Avoid excessing logging of RetryRequest exception
Fixed bug in InsertFromSelect columns order
Add process guards + invalidate to the connection pool

1.7.0

Switch to non-namespaced module import - oslo_i18n
Fix documented env variable for test connection
Updated from global requirements
Implement generic update-on-match feature

1.6.0

•: Updated from global requirements

1.5.0

Make DBAPI class work with mocks correctly
Updated from global requirements
Imported Translations from Transifex
Fix PyMySQL reference error detection
Use PyMySQL as DB driver in py3 environment
Updated from global requirements
Organize provisioning to use testresources
Add retry decorator allowing to retry DB operations on request
Imported Translations from Transifex
Implement backend-specific drop_all_objects for provisioning
Ensure that create_engine() close test connection
Refactor database migration manager to use given engine
Fix 0 version handling in migration_cli manager
Updated from global requirements
Fix PatchStacktraceTest for pypy
Update Oslo imports to remove namespace package
Retry query if db deadlock error is received

1.4.1

Restore the check_foreign_keys() method
Ensure DBConnectionError is raised on failed revalidate

1.4.0

Fix slowest test output after test run
Updated from global requirements
Make sure sort_key_attr is QueryableAttribute when query
Ensure mysql_sql_mode is set for MySQLOpportunisticTests
Add pretty_tox wrapper script
Fix PatchStacktraceTest test
Ensure PostgreSQL connection errors are wrapped
Remove check_foreign_keys from ModelsMigrationsSync
Move files out of the namespace package
Updated from global requirements
Fix the link to the bug reporting site

1.3.0

Repair string-based disconnect filters for MySQL, DB2
Fix python3.x scoping issues with removed 'uee' variable
Updated from global requirements
Fix test_migrate_cli for py3
Fix TestConnectionUtils to py3x compatibility
Updated from global requirements
Upgrade exc_filters for 'engine' argument and connect behavior
Workflow documentation is now in infra-manual

1.2.0

Imported Translations from Transifex
Fix nested() for py3
Make test_models pass on py3
Repair include_object to accommodate new objects
Add table name to foreign keys diff
Updated from global requirements
Handle Galera deadlock on SELECT FOR UPDATE
Add exception filter for _sqlite_dupe_key_error
Add info on how to run unit tests
Ensure is_backend_avail() doesn't leave open connections
Updated from global requirements

1.1.0

Imported Translations from Transifex
Add pbr to installation requirements
Updated from global requirements
Activate pep8 check that _ is imported
Assert exceptions based on API, not string messages
Fix python3.x scoping issues with removed 'de' variable
Updated from global requirements
Updated from global requirements
Reorganize DbTestCase to use provisioning completely
Set utf8 encoding for mysql and postgresql
ModelsMigrationsSync: Add check for foreign keys
Updated from global requirements
Remove extraneous vim editor configuration comments
Remove utils.drop_unique_constraint()
Improve error reporting for backend import failures
Ensure create_engine() retries the initial connection test
Imported Translations from Transifex
Use fixture from oslo.config instead of oslo-incubator
Move begin ping listener to a connect listener
Create a nested helper function that will work on py3.x
Imported Translations from Transifex
Start adding a environment for py34/py33
Explicitly depend on six in requirements file
Unwrap DialectFunctionDispatcher from itself
Updated from global requirements
Use six.wraps instead of functools.wraps
Update help string to use database

1.0.1

Use __qualname__ if we can
Fixup Fixtures Use in db test classes
Add description for test_models_sync function
Use the six provided iterator mix-in
ModelsMigrationsSync:add correct server_default check for Enum

1.0.0

Updated from global requirements
Imported Translations from Transifex
Add history/changelog to docs
Add a check for SQLite transactional state
Add run_cross_tests.sh script
Let oslotest manage the six.move setting for mox
Fix DBReferenceError on MySQL and SQLite
Renaming in WalkVersionsMixin
Clean up documentation
Use single quotes for db schema sanity check
warn against sorting requirements
ModelsMigrationsSync:Override compare_server_default
Updated from global requirements
Imported Translations from Transifex
Add doc8 to tox environment docs
Use oslo.i18n
Repair pysqlite transaction support
Extract logging setup into a separate function
Updated from global requirements
Remove reliance on create_engine() from TestsExceptionFilter
Consolidate sqlite and mysql event listeners
Use dialect dispatch for engine initiailization
Add get_non_innodb_tables() to utils
Added check to see whether oslotest is installed

0.4.0

Implement a dialect-level function dispatch system
Move to oslo.utils
Restore correct source file encodings
Handle DB2 SmallInteger type for change_deleted_column_type_to_boolean
Imported Translations from Transifex
Fixes comments to pass E265 check
Fixes indentations to pass E128 check
Uses keyword params for i18n string to pass H703
Adds empty line to multilines docs to pass H405
Updates one line docstring with dot to pass H402
Changes import orders to pass H305 check
Fixed DeprecationWarning in exc_filters
Imported Translations from Transifex
oslo.db.exceptions module documentation
Updated from global requirements
Extension of DBDuplicateEntry exception
oslo.db.options module documentation
oslo.db.api module documentation
Imported Translations from Transifex
Use SQLAlchemy cursor execute events for tracing
Remove sqla_07 from tox.ini
Updated from global requirements
Specify raise_on_warnings=False for mysqlconnector
Make MySQL regexes generic across MySQL drivers
Allow tox tests with complex OS_TEST_DBAPI_CONNECTION URLs
Raise DBReferenceError on foreign key violation
Add host argument to get_connect_string()
Imported Translations from Transifex
Don't drop pre-existing database before tests
Port _is_db_connection_error check to exception filters
Integrate the ping listener into the filter system
Add disconnect modification support to exception handling
Implement new exception interception and filtering layer
Implement the SQLAlchemy ``handle_error()`` event
Remove moxstubout.py from oslo.db
Added check for DB2 deadlock error
Bump hacking to version 0.9.2
Opportunistic migration tests
Move all db exception to exception.py
Enable skipped tests from test_models.py
Use explicit loops instead of list comprehensions
Imported Translations from Transifex
Allow usage of several iterators on ModelBase
Add DBDuplicateEntry detection for mysqlconnector driver
Check for mysql_sql_mode is not None in create_engine()

0.3.0

Add a base test case for DB schema comparison
Test for distinct SQLAlchemy major releases
Updated from global requirements
Add __contains__ to ModelBase to fully behave like a dict
Fix test to not assume eventlet isn't present
Avoid usage of mutables as default args
Updated from global requirements

0.2.0

•: Fix kwarg passed twice error in EngineFacade.from_config()

0.1.0

Add psycopg2 to test-requirements.txt
Adding dependency documentation for MySQL
Prevent races in opportunistic db test cases
Fix Sphinx directive name
Bump hacking to 0.9.x series
Add _wrap_db_error support for postgresql
Handle slave database connection in EngineFacade
Add eventlet.tpool.Proxy for DB API calls
Added ``docs`` environment to tox.ini
Setup for translation
Remove common context from oslo.db
Remove common context usage from db model_query()
replace string format arguments with function parameters
Make get_session() pass kwargs to a sessionmaker
Allow for skipping thread_yielding
Add index modifying methods
Log a cause of db backend connection failure
Do not always adjust sqlalchemy.engine logging
Fix the test using in-file SQLite database
Updated from global requirements
cleaning up index.rst file
Fix usage of oslo.config
Add import_exceptions to tox.ini
Fix changing the type of column deleted
Remove redundant default=None for config options
remove definitions of Python Source Code Encoding
Improve help strings
Ignore oslo.db.egg-info
Allow cover tests to work
Fix wrong method name with assert_called_once_with
Fix call to mock.assert_not_called()
Remove obsolete entries from .gitignore
Remove patch_migrate()
Fix typos: Remove extra ")" in message
Fix .gitreview for oslo.db
Fix dhellmann's notes from April 18
Make the tests passing
Fix the graduate.sh script result
Prevent races in opportunistic db test cases
Drop dependency oslo.db from common.log
Use oslotest instead of common test module
Start ping listener also for postgresql
Add a warning to not use get_table for working with ForeignKeys
Ignore migrate versioning tables in utf8 sanity check
Fix sqlalchemy utils test cases for SA 0.9.x
Fix Keystone doc build errors with SQLAlchemy 0.9
Make table utf-8 charset checking be optional for DB migration
Dispose db connections pool on disconnect
Python3: pass bytes as 'contents' to create_tempfiles()
Do not use the 'extend' method on a dict_items object
Set sql_mode callback on connect instead of checkout
Fix excessive logging from db.sqlalchemy.session
Add lockutils fixture to OpportunisticTestCase
Move test_insert_from_select unit test from nova to oslo
Adapt DB provisioning code for CI requirements
Make db utils importable without migrate
Remove requirements.txt from .gitignore
Get mysql_sql_mode parameter from config
Prevent incorrect usage of _wrap_db_error()
Python3: define a __next__() method for ModelBase
Add from_config() method to EngineFacade
db: move all options into database group
Drop special case for MySQL traditional mode, update unit tests
Make TRADITIONAL the default SQL mode
Introduce mysql_sql_mode option, remove old warning
Introduce a method to set any MySQL session SQL mode
Handle ibm_db_sa DBDuplicateEntry integrity errors
Fix doc build errors in db.sqlalchemy
Fix migration.db_version when no tables
Update log translation domains
Add model_query() to db.sqlalchemy.utils module
Fix a small typo in api.py
migration.db_sync requires an engine now
Remove CONF.database.connection default value
Remove None for dict.get()
Fix duplicating of SQL queries in logs
Update oslo log messages with translation domains
Restore the ability to load the DB backend lazily
Don't use cfg.CONF in oslo.db
Don't store engine instances in oslo.db
Add etc/openstack.conf.sample to .gitignore
py3kcompat: remove
Don't raise MySQL 2013 'Lost connection' errors
Format sql in db.sqlalchemy.session docstring
Handle exception messages with six.text_type
Drop dependency on log from oslo db code
Automatic retry db.api query if db connection lost
Clean up docstring in db.sqlalchemy.session
Only enable MySQL TRADITIONAL mode if we're running against MySQL
Move db tests base.py to common code
Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
Use dialect rather than a particular DB API driver
Move helper DB functions to db.sqlalchemy.utils
Small edits on help strings
Transition from migrate to alembic
Fix mocking of utcnow() for model datetime cols
Add a db check for CHARSET=utf8
Remove unused variables
Remove "vim: tabstop=4 shiftwidth=4 softtabstop=4" from headers
Fix database connection string is secret
Cleanup unused log related code
Removed copyright from empty files
Fix the obsolete exception message
Fix filter() usage due to python 3 compability
Use hacking import_exceptions for gettextutils._
Add docstring for exception handlers of session
Removal of _REPOSITORY global variable
Remove string.lowercase usage
Remove eventlet tpool from common db.api
Database hook enabling traditional mode at MySQL
Replace xrange in for loop with range
SQLAlchemy error patterns improved
Remove unused import
Correct invalid docstrings
Remove start index 0 in range()
Make _extra_keys a property of ModelBase
Fix mis-spellings
Fix violations of H302:import only modules
Enables db2 server disconnects to be handled pessimistically
db.sqlalchemy.session add [sql].idle_timeout
Use six.iteritems to make dict work on Python2/3
Trivial: Make vertical white space after license header consistent
Drop dependency on processutils from oslo db code
Fix locking in migration tests
Incorporating MIT licensed code
Typos fix in db and periodic_task module
Use six.moves.configparser instead of ConfigParser
Drop dependency on fileutils from oslo db tests
fix typo in db session docstring
Added opportunistic DB test cases
The ability to run tests at various backend
Use log.warning() instead of log.warn() in oslo.db
Replace removed items in Python3
Remove vim header
Use py3kcompat urlutils functions instead of urlparse
Don't use deprecated module commands
Remove sqlalchemy-migrate 0.7.3 patching
SQLite behavior independent DB test cases
Drop dependency on lockutils from oslo db code
Remove lazy loading of database backend
Do not name variables as builtins
Add db2 communication error code when check the db connection
Replace using tests.utils part3
Add [sql].connection as deprecated opt for db
Modify SQLA session due to dispose of eventlet
Use monkey_patch() in TestMigrationUtils setUp()
Clean up db.sqla.Models.extra_keys interface
Use functools.wrap() instead of custom implementation
Move base migration test classes to common code
Bump hacking to 0.7.0
exception: remove
Replace using tests.utils with openstack.common.test
Use single meta when change column type
Helper function to sanitize db url credentials
BaseException.message is deprecated since Python 2.6
Add function drop_unique_constraint()
Change sqlalchemy/utils.py mode back to 644
Move sqlalchemy migration from Nova
Allow use of hacking 0.6.0 and enable new checks
Add eclipse project files to .gitignore
Raise ValueError if sort_dir is unknown
Add tests for cinder/common/sqlalchemyutils.py
python3: Add python3 compatibility support
Add .testrepository to .gitignore
Move `test_migrations` from Nova
Migrate sqlalchemy utils from Nova
Enable H302 hacking check
Add a monkey-patching util for sqlalchemy-migrate
Don't use mixture of cfg.Opt() deprecated args
Allow BaseTestCase use a different conf object
Ensure that DB configuration is backward compatible
Add a fixture for using of SQLite in-memory DB
Enable hacking H404 test
Enable user to configure pool_timeout
Changed processing unique constraint name
Enable H306 hacking check
Add a slave db handle for the SQLAlchemy backend
Enable hacking H403 test
Changed processing unique constraint name
Ignore backup files in .gitignore
Specify database group instead of DEFAULT
Fixes import order nits
Line wrapper becomes to long when expanded
Convert unicode for python3 portability
Add test coverage for sqlite regexp function
Use range rather than xrange
Add support to clear DB
Add enforcement for foreign key contraints with sqlite
Improve Python 3.x compatibility
Removes metadata from ModelBase
Removes created_at, updated_at from ModelBase
Fixes private functions private
Mark sql_connection with secret flag
Fix Copyright Headers - Rename LLC to Foundation
Fixes import order nits
Clean up sqlalchemy exception code
Move DB thread pooling to DB API loader
Use oslo-config-2013.1b3
Add join_consumer_pool() to RPC connections
Use importutils.try_import() for MySQLdb
Minor tweak to make update.py happy
Remove pointless use of OpenStackException
Remove unused context from test_sqlalchemy
Remove openstack.common.db.common
Provide creating real unique constraints for columns
Fix missing wrap_db_error for Session.execute() method
Fix eventlet/mysql db pooling code
Add missing DBDuplicateEntry
Be explicit about set_default() parameters
Remove duplicate DB options
Eliminate gratuitous DB difference vs Nova
Import sqlalchemy session/models/utils
updating sphinx documentation
Correcting openstack-common mv to oslo-incubator
Update .gitreview for oslo
.gitignore updates for generated files
Updated tox config for multi-python testing
Added .gitreview file
ignore cover's html directory
Rajaram/Vinkesh|increased tests for Request and Response serializers
Rajaram/Vinkesh|Added nova's serializaiton classes into common
Initial skeleton project

REFERENCE

Configuration Options

oslo.db uses oslo.config to define and manage configuration options to allow the deployer to control how an application uses the underlying database.

database

sqlite_synchronous

Type: boolean
Default: True

If True, SQLite uses synchronous mode.

Deprecated Variations

Group	Name
DEFAULT	sqlite_synchronous

backend

Type: string
Default: sqlalchemy

The back end to use for the database.

Deprecated Variations

Group	Name
DEFAULT	db_backend

connection

Type: string
Default: <None>

The SQLAlchemy connection string to use to connect to the database.

Deprecated Variations

Group	Name
DEFAULT	sql_connection
DATABASE	sql_connection
sql	connection

slave_connection

Type: string
Default: <None>

The SQLAlchemy connection string to use to connect to the slave database.

mysql_sql_mode

Type: string
Default: TRADITIONAL

The SQL mode to be used for MySQL sessions. This option, including the default, overrides any server-set SQL mode. To use whatever SQL mode is set by the server configuration, set this to no value. Example: mysql_sql_mode=

mysql_wsrep_sync_wait

Type: integer
Default: <None>

For Galera only, configure wsrep_sync_wait causality checks on new connections. Default is None, meaning don't configure any setting.

mysql_enable_ndb

Type: boolean
Default: False

If True, transparently enables support for handling MySQL Cluster (NDB).

WARNING:

This option is deprecated for removal since 12.1.0. Its value may be silently ignored in the future.

Reason: Support for the MySQL NDB Cluster storage engine has been deprecated and will be removed in a future release.

connection_recycle_time

Type: integer
Default: 3600

Connections which have been present in the connection pool longer than this number of seconds will be replaced with a new one the next time they are checked out from the pool.

max_pool_size

Type: integer
Default: 5

Maximum number of SQL connections to keep open in a pool. Setting a value of 0 indicates no limit.

max_retries

Type: integer
Default: 10

Maximum number of database connection retries during startup. Set to -1 to specify an infinite retry count.

Deprecated Variations

Group	Name
DEFAULT	sql_max_retries
DATABASE	sql_max_retries

retry_interval

Type: integer
Default: 10

Interval between retries of opening a SQL connection.

Deprecated Variations

Group	Name
DEFAULT	sql_retry_interval
DATABASE	reconnect_interval

max_overflow

Type: integer
Default: 50

If set, use this value for max_overflow with SQLAlchemy.

Deprecated Variations

Group	Name
DEFAULT	sql_max_overflow
DATABASE	sqlalchemy_max_overflow

connection_debug

Type: integer
Default: 0
Minimum Value: 0
Maximum Value: 100

Verbosity of SQL debugging information: 0=None, 100=Everything.

Deprecated Variations

Group	Name
DEFAULT	sql_connection_debug

connection_trace

Type: boolean
Default: False

Add Python stack traces to SQL as comment strings.

Deprecated Variations

Group	Name
DEFAULT	sql_connection_trace

pool_timeout

Type: integer
Default: <None>

If set, use this value for pool_timeout with SQLAlchemy.

Deprecated Variations

Group	Name
DATABASE	sqlalchemy_pool_timeout

use_db_reconnect

Type: boolean
Default: False

Enable the experimental use of database reconnect on connection lost.

db_retry_interval

Type: integer
Default: 1

Seconds between retries of a database transaction.

db_inc_retry_interval

Type: boolean
Default: True

If True, increases the interval between retries of a database operation up to db_max_retry_interval.

db_max_retry_interval

Type: integer
Default: 10

If db_inc_retry_interval is set, the maximum seconds between retries of a database operation.

db_max_retries

Type: integer
Default: 20

Maximum retries in case of connection error or deadlock error before error is raised. Set to -1 to specify an infinite retry count.

connection_parameters

Type: string
Default: ''

Optional URL parameters to append onto the connection URL at connect time; specify as param1=value1&param2=value2&...

API

oslo_db

oslo_db package

Subpackages

oslo_db.sqlalchemy package

Subpackages

oslo_db.sqlalchemy.migration_cli package

Submodules

oslo_db.sqlalchemy.migration_cli.ext_alembic module

class oslo_db.sqlalchemy.migration_cli.ext_alembic.AlembicExtension(engine, migration_config): Bases: oslo_db.sqlalchemy.migration_cli.ext_base.MigrationExtensionBase
Extension to provide alembic features.

Parameters

engine (sqlalchemy.engine.Engine) -- SQLAlchemy engine instance for a given database
migration_config (dict) -- Stores specific configuration for migrations

downgrade(version): Used for downgrading database.

Parameters: version (string) -- Desired database version

property enabled: Used for availability verification of a plugin.

Return type: bool

has_revision(rev_id): Checks whether the repo contains a revision

Parameters: rev_id -- Revision to check
Returns: Whether the revision is in the repo
Return type: bool

order = 2

revision(message='', autogenerate=False): Creates template for migration.

Parameters

message (string) -- Text that will be used for migration title
autogenerate (bool) -- If True - generates diff based on current database state

stamp(revision): Stamps database with provided revision.

Parameters: revision (string) -- Should match one from repository or head - to stamp database with most recent revision

upgrade(version): Used for upgrading database.

Parameters: version (string) -- Desired database version

version(): Current database version.

Returns: Databse version
Return type: string

oslo_db.sqlalchemy.migration_cli.ext_base module

class oslo_db.sqlalchemy.migration_cli.ext_base.MigrationExtensionBase: Bases: object

abstract downgrade(version): Used for downgrading database.

Parameters: version (string) -- Desired database version

property enabled: Used for availability verification of a plugin.

Return type: bool

has_revision(rev_id): Checks whether the repo contains a revision

Parameters: rev_id -- Revision to check
Returns: Whether the revision is in the repo
Return type: bool

order = 0

revision(*args, **kwargs)

Used to generate migration script.

In migration engines that support this feature, it should generate new migration script.

Accept arbitrary set of arguments.

stamp(*args, **kwargs): Stamps database based on plugin features.
Accept arbitrary set of arguments.

abstract upgrade(version): Used for upgrading database.

Parameters: version (string) -- Desired database version

abstract version(): Current database version.

Returns: Databse version
Return type: string

oslo_db.sqlalchemy.migration_cli.ext_migrate module

class oslo_db.sqlalchemy.migration_cli.ext_migrate.MigrateExtension(engine, migration_config): Bases: oslo_db.sqlalchemy.migration_cli.ext_base.MigrationExtensionBase
Extension to provide sqlalchemy-migrate features.

Parameters: migration_config (dict) -- Stores specific configuration for migrations

downgrade(version): Used for downgrading database.

Parameters: version (string) -- Desired database version

property enabled: Used for availability verification of a plugin.

Return type: bool

has_revision(rev_id): Checks whether the repo contains a revision

Parameters: rev_id -- Revision to check
Returns: Whether the revision is in the repo
Return type: bool

order = 1

upgrade(version): Used for upgrading database.

Parameters: version (string) -- Desired database version

version(): Current database version.

Returns: Databse version
Return type: string

oslo_db.sqlalchemy.migration_cli.manager module

class oslo_db.sqlalchemy.migration_cli.manager.MigrationManager(migration_config, engine=None): Bases: object

downgrade(revision): Downgrade database with available backends.

revision(message, autogenerate): Generate template or autogenerated revision.

stamp(revision): Create stamp for a given revision.

upgrade(revision): Upgrade database with all available backends.

version(): Return last version of db.

oslo_db.sqlalchemy.migration_cli.manager.check_plugin_enabled(ext): Used for EnabledExtensionManager.

Module contents

Submodules

oslo_db.sqlalchemy.enginefacade module

exception oslo_db.sqlalchemy.enginefacade.AlreadyStartedError

Bases: TypeError

Raises when a factory is being asked to initialize a second time.

Subclasses TypeError for legacy support.

class oslo_db.sqlalchemy.enginefacade.LegacyEngineFacade(sql_connection, slave_connection=None, sqlite_fk=False, autocommit=False, expire_on_commit=False, _conf=None, _factory=None, **kwargs)

Bases: object

A helper class for removing of global engine instances from oslo.db.

Deprecated since version 1.12.0: Please use oslo_db.sqlalchemy.enginefacade for new development.

As a library, oslo.db can't decide where to store/when to create engine and sessionmaker instances, so this must be left for a target application.

On the other hand, in order to simplify the adoption of oslo.db changes, we'll provide a helper class, which creates engine and sessionmaker on its instantiation and provides get_engine()/get_session() methods that are compatible with corresponding utility functions that currently exist in target projects, e.g. in Nova.

engine/sessionmaker instances will still be global (and they are meant to be global), but they will be stored in the app context, rather that in the oslo.db context.

Two important things to remember:

1.: An Engine instance is effectively a pool of DB connections, so it's meant to be shared (and it's thread-safe).
2.: A Session instance is not meant to be shared and represents a DB transactional context (i.e. it's not thread-safe). sessionmaker is a factory of sessions.

Parameters

sql_connection (string) -- the connection string for the database to use
slave_connection (string) -- the connection string for the 'slave' database to use. If not provided, the master database will be used for all operations. Note: this is meant to be used for offloading of read operations to asynchronously replicated slaves to reduce the load on the master database.
sqlite_fk (bool) -- enable foreign keys in SQLite
autocommit (bool) -- use autocommit mode for created Session instances
expire_on_commit (bool) -- expire session objects on commit

Keyword arguments:

Parameters

mysql_sql_mode -- the SQL mode to be used for MySQL sessions. (defaults to TRADITIONAL)
mysql_wsrep_sync_wait -- value of wsrep_sync_wait for Galera (defaults to None, which indicates no setting will be passed)
mysql_enable_ndb -- If True, transparently enables support for handling MySQL Cluster (NDB). (defaults to False) DEPRECATED
connection_recycle_time -- Time period for connections to be recycled upon checkout (defaults to 3600)
connection_debug -- verbosity of SQL debugging information. -1=Off, 0=None, 100=Everything (defaults to 0)
max_pool_size -- maximum number of SQL connections to keep open in a pool (defaults to SQLAlchemy settings)
max_overflow -- if set, use this value for max_overflow with sqlalchemy (defaults to SQLAlchemy settings)
pool_timeout -- if set, use this value for pool_timeout with sqlalchemy (defaults to SQLAlchemy settings)
sqlite_synchronous -- if True, SQLite uses synchronous mode (defaults to True)
connection_trace -- add python stack traces to SQL as comment strings (defaults to False)
max_retries -- maximum db connection retries during startup. (setting -1 implies an infinite retry count) (defaults to 10)
retry_interval -- interval between retries of opening a sql connection (defaults to 10)
thread_checkin -- boolean that indicates that between each engine checkin event a sleep(0) will occur to allow other greenthreads to run (defaults to True)

classmethod from_config(conf, sqlite_fk=False, autocommit=False, expire_on_commit=False): Initialize EngineFacade using oslo.config config instance options.

Parameters

conf (oslo_config.cfg.ConfigOpts) -- oslo.config config instance
sqlite_fk (bool) -- enable foreign keys in SQLite
autocommit (bool) -- use autocommit mode for created Session instances
expire_on_commit (bool) -- expire session objects on commit

get_engine(use_slave=False): Get the engine instance (note, that it's shared).

Parameters: use_slave (bool) -- if possible, use 'slave' database for this engine. If the connection string for the slave database wasn't provided, 'master' engine will be returned. (defaults to False)

get_session(use_slave=False, **kwargs): Get a Session instance.

Parameters: use_slave (bool) -- if possible, use 'slave' database connection for this session. If the connection string for the slave database wasn't provided, a session bound to the 'master' engine will be returned. (defaults to False)

Keyword arguments will be passed to a sessionmaker instance as is (if passed, they will override the ones used when the sessionmaker instance was created). See SQLAlchemy Session docs for details.

get_sessionmaker(use_slave=False): Get the sessionmaker instance used to create a Session.
This can be called for those cases where the sessionmaker() is to be temporarily injected with some state such as a specific connection.

oslo_db.sqlalchemy.enginefacade.async_compat

oslo_db.sqlalchemy.enginefacade.configure(**kw)

Apply configurational options to the global factory.

This method can only be called before any specific transaction-beginning methods have been called.

SEE ALSO:

_TransactionFactory.configure()

oslo_db.sqlalchemy.enginefacade.get_legacy_facade(): Return a LegacyEngineFacade for the global factory.
This facade will make use of the same engine and sessionmaker as this factory, however will not share the same transaction context; the legacy facade continues to work the old way of returning a new Session each time get_session() is called.

oslo_db.sqlalchemy.enginefacade.reader = <oslo_db.sqlalchemy.enginefacade._TransactionContextManager object>: The global 'reader' starting point.

oslo_db.sqlalchemy.enginefacade.transaction_context(): Construct a local transaction context.

oslo_db.sqlalchemy.enginefacade.transaction_context_provider(klass): Decorate a class with session and connection attributes.

oslo_db.sqlalchemy.enginefacade.writer = <oslo_db.sqlalchemy.enginefacade._TransactionContextManager object>: The global 'writer' starting point.

oslo_db.sqlalchemy.engines module

Core SQLAlchemy connectivity routines.

oslo_db.sqlalchemy.engines.create_engine(sql_connection, sqlite_fk=False, mysql_sql_mode=None, mysql_wsrep_sync_wait=None, mysql_enable_ndb=False, connection_recycle_time=3600, connection_debug=0, max_pool_size=None, max_overflow=None, pool_timeout=None, sqlite_synchronous=True, connection_trace=False, max_retries=10, retry_interval=10, thread_checkin=True, logging_name=None, json_serializer=None, json_deserializer=None, connection_parameters=None): Return a new SQLAlchemy engine.

oslo_db.sqlalchemy.exc_filters module

Define exception redefinitions for SQLAlchemy DBAPI exceptions.

oslo_db.sqlalchemy.exc_filters.filters(dbname, exception_type, regex): Mark a function as receiving a filtered exception.

Parameters

dbname -- string database name, e.g. 'mysql'
exception_type -- a SQLAlchemy database exception class, which extends from sqlalchemy.exc.DBAPIError.
regex -- a string, or a tuple of strings, that will be processed as matching regular expressions.

oslo_db.sqlalchemy.exc_filters.handler(context): Iterate through available filters and invoke those which match.
The first one which raises wins. The order in which the filters are attempted is sorted by specificity - dialect name or "*", exception class per method resolution order (__mro__). Method resolution order is used so that filter rules indicating a more specific exception class are attempted first.

oslo_db.sqlalchemy.exc_filters.register_engine(engine)

oslo_db.sqlalchemy.migration module

oslo_db.sqlalchemy.migration.db_sync(engine, abs_path, version=None, init_version=0, sanity_check=True): Upgrade or downgrade a database.
Function runs the upgrade() or downgrade() functions in change scripts.

Parameters

engine -- SQLAlchemy engine instance for a given database
abs_path -- Absolute path to migrate repository.
version -- Database will upgrade/downgrade until this version. If None - database will update to the latest available version.
init_version -- Initial database version
sanity_check -- Require schema sanity checking for all tables

oslo_db.sqlalchemy.migration.db_version(engine, abs_path, init_version): Show the current version of the repository.

Parameters

engine -- SQLAlchemy engine instance for a given database
abs_path -- Absolute path to migrate repository
init_version -- Initial database version

oslo_db.sqlalchemy.migration.db_version_control(engine, abs_path, version=None): Mark a database as under this repository's version control.
Once a database is under version control, schema changes should only be done via change scripts in this repository.

Parameters

engine -- SQLAlchemy engine instance for a given database
abs_path -- Absolute path to migrate repository
version -- Initial database version

oslo_db.sqlalchemy.models module

SQLAlchemy models.

class oslo_db.sqlalchemy.models.ModelBase: Bases: object
Base class for models.

get(key, default=None)

items(): Make the model object behave like a dict.

iteritems(): Make the model object behave like a dict.

keys(): Make the model object behave like a dict.

save(session): Save this object.

update(values): Make the model object behave like a dict.

class oslo_db.sqlalchemy.models.ModelIterator(model, columns): Bases: object

class oslo_db.sqlalchemy.models.SoftDeleteMixin: Bases: object

deleted = Column(None, SoftDeleteInteger(), table=None, default=ColumnDefault(0))

deleted_at = Column(None, DateTime(), table=None)

soft_delete(session): Mark this object as deleted.

class oslo_db.sqlalchemy.models.TimestampMixin: Bases: object

created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<function TimestampMixin.<lambda>>))

updated_at = Column(None, DateTime(), table=None, onupdate=ColumnDefault(<function TimestampMixin.<lambda>>))

oslo_db.sqlalchemy.ndb module

Core functions for MySQL Cluster (NDB) Support.

oslo_db.sqlalchemy.ndb.enable_ndb_support(engine): Enable NDB Support.
Function to flag the MySQL engine dialect to support features specific to MySQL Cluster (NDB).

oslo_db.sqlalchemy.ndb.init_ndb_events(engine): Initialize NDB Events.
Function starts NDB specific events.

oslo_db.sqlalchemy.ndb.ndb_status(engine_or_compiler): Test if NDB Support is enabled.
Function to test if NDB support is enabled or not.

oslo_db.sqlalchemy.ndb.prefix_inserts(create_table, compiler, **kw): Replace InnoDB with NDBCLUSTER automatically.
Function will intercept CreateTable() calls and automatically convert InnoDB to NDBCLUSTER. Targets compiler events.

oslo_db.sqlalchemy.orm module

SQLAlchemy ORM connectivity and query structures.

class oslo_db.sqlalchemy.orm.Query(entities, session=None): Bases: sqlalchemy.orm.query.Query
Subclass of sqlalchemy.query with soft_delete() method.

soft_delete(synchronize_session='evaluate')

update_on_match(specimen, surrogate_key, values, **kw): Emit an UPDATE statement matching the given specimen.
This is a method-version of oslo_db.sqlalchemy.update_match.update_on_match(); see that function for usage details.

update_returning_pk(values, surrogate_key): Perform an UPDATE, returning the primary key of the matched row.
This is a method-version of oslo_db.sqlalchemy.update_match.update_returning_pk(); see that function for usage details.

class oslo_db.sqlalchemy.orm.Session(bind=None, autoflush=True, future=False, expire_on_commit=True, autocommit=False, twophase=False, binds=None, enable_baked_queries=True, info=None, query_cls=None): Bases: sqlalchemy.orm.session.Session
oslo.db-specific Session subclass.

oslo_db.sqlalchemy.orm.get_maker(engine, autocommit=False, expire_on_commit=False): Return a SQLAlchemy sessionmaker using the given engine.

oslo_db.sqlalchemy.provision module

Provision test environment for specific DB backends

class oslo_db.sqlalchemy.provision.Backend(database_type, url)

Bases: object

Represent a particular database backend that may be provisionable.

The Backend object maintains a database type (e.g. database without specific driver type, such as "sqlite", "postgresql", etc.), a target URL, a base Engine for that URL object that can be used to provision databases and a BackendImpl which knows how to perform operations against this type of Engine.

classmethod all_viable_backends(): Return an iterator of all Backend objects that are present
and provisionable.

classmethod backend_for_database_type(database_type): Return the Backend for the given database type.

backends_by_database_type = {'mysql': <oslo_db.sqlalchemy.provision.Backend object>, 'postgresql': <oslo_db.sqlalchemy.provision.Backend object>, 'sqlite': <oslo_db.sqlalchemy.provision.Backend object>}

create_named_database(ident, conditional=False): Create a database with the given name.

database_exists(ident): Return True if a database of the given name exists.

drop_all_objects(engine): Drop all database objects.
Drops all database objects remaining on the default schema of the given engine.

drop_named_database(ident, conditional=False): Drop a database with the given name.

provisioned_database_url(ident)

Given the identifier of an anoymous database, return a URL.

For hostname-based URLs, this typically involves switching just the 'database' portion of the URL with the given name and creating a URL.

For SQLite URLs, the identifier may be used to create a filename or may be ignored in the case of a memory database.

class oslo_db.sqlalchemy.provision.BackendImpl(drivername)

Bases: object

Provide database-specific implementations of key provisioning

functions.

BackendImpl is owned by a Backend instance which delegates to it for all database-specific features.

classmethod all_impls(): Return an iterator of all possible BackendImpl objects.
These are BackendImpls that are implemented, but not necessarily provisionable.

abstract create_named_database(engine, ident, conditional=False): Create a database with the given name.

abstract create_opportunistic_driver_url(): Produce a string url known as the 'opportunistic' URL.
This URL is one that corresponds to an established OpenStack convention for a pre-established database login, which, when detected as available in the local environment, is automatically used as a test platform for a specific type of driver.

default_engine_kwargs = {}

dispose(engine)

drop_additional_objects(conn)

drop_all_objects(engine)

Drop all database objects.

Drops all database objects remaining on the default schema of the given engine.

Per-db implementations will also need to drop items specific to those systems, such as sequences, custom types (e.g. pg ENUM), etc.

abstract drop_named_database(engine, ident, conditional=False): Drop a database with the given name.

impl = <oslo_db.sqlalchemy.utils.DialectSingleFunctionDispatcher object>

provisioned_database_url(base_url, ident)

Return a provisioned database URL.

Given the URL of a particular database backend and the string name of a particular 'database' within that backend, return an URL which refers directly to the named database.

For hostname-based URLs, this typically involves switching just the 'database' portion of the URL with the given name and creating an engine.

For URLs that instead deal with DSNs, the rules may be more custom; for example, the engine may need to connect to the root URL and then emit a command to switch to the named database.

supports_drop_fk = True

class oslo_db.sqlalchemy.provision.BackendResource(database_type, ad_hoc_url=None): Bases: testresources.TestResourceManager

clean(resource): Override this to class method to hook into resource removal.

isDirty(): Return True if this managers cached resource is dirty.
Calling when the resource is not currently held has undefined behaviour.

make(dependency_resources): Override this to construct resources.

Parameters: dependency_resources -- A dict mapping name -> resource instance for the resources specified as dependencies.
Returns: The made resource.

class oslo_db.sqlalchemy.provision.DatabaseResource(database_type, _enginefacade=None, provision_new_database=True, ad_hoc_url=None)

Bases: testresources.TestResourceManager

Database resource which connects and disconnects to a URL.

For SQLite, this means the database is created implicitly, as a result of SQLite's usual behavior. If the database is a file-based URL, it will remain after the resource has been torn down.

For all other kinds of databases, the resource indicates to connect and disconnect from that database.

clean(resource): Override this to class method to hook into resource removal.

isDirty(): Return True if this managers cached resource is dirty.
Calling when the resource is not currently held has undefined behaviour.

make(dependency_resources): Override this to construct resources.

Parameters: dependency_resources -- A dict mapping name -> resource instance for the resources specified as dependencies.
Returns: The made resource.

class oslo_db.sqlalchemy.provision.ProvisionedDatabase(backend, enginefacade, engine, db_token)

Bases: object

Represents a database engine pointing to a DB ready to run tests.

backend: an instance of Backend

enginefacade: an instance of _TransactionFactory

engine: a SQLAlchemy Engine

db_token: if provision_new_database were used, this is the randomly: generated name of the database. Note that with SQLite memory connections, this token is ignored. For a database that wasn't actually created, will be None.

backend

db_token

engine

enginefacade

class oslo_db.sqlalchemy.provision.Schema

Bases: object

"Represents a database schema that has or will be populated.

This is a marker object as required by testresources but otherwise serves no purpose.

database

class oslo_db.sqlalchemy.provision.SchemaResource(database_resource, generate_schema, teardown=False): Bases: testresources.TestResourceManager

clean(resource): Override this to class method to hook into resource removal.

isDirty(): Return True if this managers cached resource is dirty.
Calling when the resource is not currently held has undefined behaviour.

make(dependency_resources): Override this to construct resources.

Parameters: dependency_resources -- A dict mapping name -> resource instance for the resources specified as dependencies.
Returns: The made resource.

oslo_db.sqlalchemy.session module

Session Handling for SQLAlchemy backend.

Recommended ways to use sessions within this framework:

•: Use the enginefacade system for connectivity, session and transaction management:

from oslo_db.sqlalchemy import enginefacade
@enginefacade.reader
def get_foo(context, foo):


    return (model_query(models.Foo, context.session).


            filter_by(foo=foo).


            first())
@enginefacade.writer
def update_foo(context, id, newfoo):


    (model_query(models.Foo, context.session).


            filter_by(id=id).


            update({'foo': newfoo}))
@enginefacade.writer
def create_foo(context, values):


    foo_ref = models.Foo()


    foo_ref.update(values)


    foo_ref.save(context.session)


    return foo_ref

In the above system, transactions are committed automatically, and are shared among all dependent database methods. Ensure that methods which "write" data are enclosed within @writer blocks.

NOTE:

Statements in the session scope will not be automatically retried.

•: If you create models within the session, they need to be added, but you do not need to call model.save():

@enginefacade.writer
def create_many_foo(context, foos):


    for foo in foos:


        foo_ref = models.Foo()


        foo_ref.update(foo)


        context.session.add(foo_ref)
@enginefacade.writer
def update_bar(context, foo_id, newbar):


    foo_ref = (model_query(models.Foo, context.session).


                filter_by(id=foo_id).


                first())


    (model_query(models.Bar, context.session).


                filter_by(id=foo_ref['bar_id']).


                update({'bar': newbar}))

The two queries in update_bar can alternatively be expressed using a single query, which may be more efficient depending on scenario:

@enginefacade.writer
def update_bar(context, foo_id, newbar):


    subq = (model_query(models.Foo.id, context.session).


            filter_by(id=foo_id).


            limit(1).


            subquery())


    (model_query(models.Bar, context.session).


            filter_by(id=subq.as_scalar()).


            update({'bar': newbar}))

For reference, this emits approximately the following SQL statement:

UPDATE bar SET bar = '${newbar}'


    WHERE id=(SELECT bar_id FROM foo WHERE id = '${foo_id}' LIMIT 1);

NOTE:

create_duplicate_foo is a trivially simple example of catching an exception while using a savepoint. Here we create two duplicate instances with same primary key, must catch the exception out of context managed by a single session:

@enginefacade.writer
def create_duplicate_foo(context):


    foo1 = models.Foo()


    foo2 = models.Foo()


    foo1.id = foo2.id = 1


    try:


        with context.session.begin_nested():


            session.add(foo1)


            session.add(foo2)


    except exception.DBDuplicateEntry as e:


        handle_error(e)

•: The enginefacade system eliminates the need to decide when sessions need to be passed between methods. All methods should instead share a common context object; the enginefacade system will maintain the transaction across method calls.

@enginefacade.writer
def myfunc(context, foo):


    # do some database things


    bar = _private_func(context, foo)


    return bar
def _private_func(context, foo):


    with enginefacade.using_writer(context) as session:


        # do some other database things


        session.add(SomeObject())


    return bar

•: Avoid with_lockmode('UPDATE') when possible.
FOR UPDATE is not compatible with MySQL/Galera. Instead, an "opportunistic" approach should be used, such that if an UPDATE fails, the entire transaction should be retried. The @wrap_db_retry decorator is one such system that can be used to achieve this.

Enabling soft deletes:

•: To use/enable soft-deletes, SoftDeleteMixin may be used. For example:

class NovaBase(models.SoftDeleteMixin, models.ModelBase):


    pass

Efficient use of soft deletes:

•: While there is a model.soft_delete() method, prefer query.soft_delete(). Some examples:

@enginefacade.writer
def soft_delete_bar(context):


    # synchronize_session=False will prevent the ORM from attempting


    # to search the Session for instances matching the DELETE;


    # this is typically not necessary for small operations.


    count = model_query(BarModel, context.session).\


        find(some_condition).soft_delete(synchronize_session=False)


    if count == 0:


        raise Exception("0 entries were soft deleted")
@enginefacade.writer
def complex_soft_delete_with_synchronization_bar(context):


    # use synchronize_session='evaluate' when you'd like to attempt


    # to update the state of the Session to match that of the DELETE.


    # This is potentially helpful if the operation is complex and


    # continues to work with instances that were loaded, though


    # not usually needed.


    count = (model_query(BarModel, context.session).


                find(some_condition).


                soft_delete(synchronize_session='evaulate'))


    if count == 0:


        raise Exception("0 entries were soft deleted")

oslo_db.sqlalchemy.session.EngineFacade: alias of oslo_db.sqlalchemy.enginefacade.LegacyEngineFacade

class oslo_db.sqlalchemy.session.Query(entities, session=None): Bases: sqlalchemy.orm.query.Query
Subclass of sqlalchemy.query with soft_delete() method.

soft_delete(synchronize_session='evaluate')

update_on_match(specimen, surrogate_key, values, **kw): Emit an UPDATE statement matching the given specimen.
This is a method-version of oslo_db.sqlalchemy.update_match.update_on_match(); see that function for usage details.

update_returning_pk(values, surrogate_key): Perform an UPDATE, returning the primary key of the matched row.
This is a method-version of oslo_db.sqlalchemy.update_match.update_returning_pk(); see that function for usage details.

class oslo_db.sqlalchemy.session.Session(bind=None, autoflush=True, future=False, expire_on_commit=True, autocommit=False, twophase=False, binds=None, enable_baked_queries=True, info=None, query_cls=None): Bases: sqlalchemy.orm.session.Session
oslo.db-specific Session subclass.

oslo_db.sqlalchemy.session.create_engine(sql_connection, sqlite_fk=False, mysql_sql_mode=None, mysql_wsrep_sync_wait=None, mysql_enable_ndb=False, connection_recycle_time=3600, connection_debug=0, max_pool_size=None, max_overflow=None, pool_timeout=None, sqlite_synchronous=True, connection_trace=False, max_retries=10, retry_interval=10, thread_checkin=True, logging_name=None, json_serializer=None, json_deserializer=None, connection_parameters=None): Return a new SQLAlchemy engine.

oslo_db.sqlalchemy.session.get_maker(engine, autocommit=False, expire_on_commit=False): Return a SQLAlchemy sessionmaker using the given engine.

oslo_db.sqlalchemy.test_base module

class oslo_db.sqlalchemy.test_base.DbFixture(test, skip_on_unavailable_db=True)

Bases: fixtures.fixture.Fixture

Basic database fixture.

Allows to run tests on various db backends, such as SQLite, MySQL and PostgreSQL. By default use sqlite backend. To override default backend uri set env variable OS_TEST_DBAPI_ADMIN_CONNECTION with database admin credentials for specific backend.

DBNAME = 'openstack_citest'

DRIVER = 'sqlite'

PASSWORD = 'openstack_citest'

USERNAME = 'openstack_citest'

setUp()

Prepare the Fixture for use.

This should not be overridden. Concrete fixtures should implement _setUp. Overriding of setUp is still supported, just not recommended.

After setUp has completed, the fixture will have one or more attributes which can be used (these depend totally on the concrete subclass).

Raises: MultipleExceptions if _setUp fails. The last exception captured within the MultipleExceptions will be a SetupError exception.
Returns: None.
Changed in 1.3: The recommendation to override setUp has been reversed - before 1.3, setUp() should be overridden, now it should not be.
Changed in 1.3.1: BaseException is now caught, and only subclasses of Exception are wrapped in MultipleExceptions.

class oslo_db.sqlalchemy.test_base.DbTestCase(*args, **kwds): Bases: oslotest.base.BaseTestCase
Base class for testing of DB code.

FIXTURE: alias of oslo_db.sqlalchemy.test_base.DbFixture

SCHEMA_SCOPE = None

SKIP_ON_UNAVAILABLE_DB = True

generate_schema(engine): Generate schema objects to be used within a test.
The function is separate from the setUp() case as the scope of this method is controlled by the provisioning system. A test that specifies SCHEMA_SCOPE may not call this method for each test, as the schema may be maintained from a previous run.

property resources

setUp(): Hook method for setting up the test fixture before exercising it.

class oslo_db.sqlalchemy.test_base.MySQLOpportunisticFixture(test, skip_on_unavailable_db=True): Bases: oslo_db.sqlalchemy.test_base.DbFixture

DRIVER = 'mysql'

class oslo_db.sqlalchemy.test_base.MySQLOpportunisticTestCase(*args, **kwds): Bases: oslo_db.sqlalchemy.test_base.OpportunisticTestCase

FIXTURE: alias of oslo_db.sqlalchemy.test_base.MySQLOpportunisticFixture

class oslo_db.sqlalchemy.test_base.OpportunisticTestCase(*args, **kwds): Bases: oslo_db.sqlalchemy.test_base.DbTestCase
Placeholder for backwards compatibility.

class oslo_db.sqlalchemy.test_base.PostgreSQLOpportunisticFixture(test, skip_on_unavailable_db=True): Bases: oslo_db.sqlalchemy.test_base.DbFixture

DRIVER = 'postgresql'

class oslo_db.sqlalchemy.test_base.PostgreSQLOpportunisticTestCase(*args, **kwds): Bases: oslo_db.sqlalchemy.test_base.OpportunisticTestCase

FIXTURE: alias of oslo_db.sqlalchemy.test_base.PostgreSQLOpportunisticFixture

oslo_db.sqlalchemy.test_base.backend_specific(*dialects): Decorator to skip backend specific tests on inappropriate engines.
::dialects: list of dialects names under which the test will be launched.

oslo_db.sqlalchemy.test_fixtures module

class oslo_db.sqlalchemy.test_fixtures.AdHocDbFixture(url=None)

Bases: oslo_db.sqlalchemy.test_fixtures.SimpleDbFixture

"Fixture which creates and disposes a database engine per test.

Also allows a specific URL to be passed, meaning the fixture can be hardcoded to a specific SQLite file.

For a SQLite, this fixture will create the named database upon setup and tear it down upon teardown. For other databases, the database is assumed to exist already and will remain after teardown.

class oslo_db.sqlalchemy.test_fixtures.BaseDbFixture(driver=None, ident=None)

Bases: fixtures.fixture.Fixture

Base database provisioning fixture.

This serves as the base class for the other fixtures, but by itself does not implement _setUp(). It provides the basis for the flags implemented by the various capability mixins (GenerateSchema, DeletesFromSchema, etc.) as well as providing an abstraction over the provisioning objects, which are specific to testresources. Overall, consumers of this fixture just need to use the right classes and the testresources mechanics are taken care of.

DRIVER = 'sqlite'

get_enginefacade()

Return an enginefacade._TransactionContextManager.

This is typically a global variable like "context_manager" declared in the db/api.py module and is the object returned by enginefacade.transaction_context().

If left not implemented, the global enginefacade manager is used.

For the case where a project uses per-object or per-test enginefacades like Gnocchi, the get_per_test_enginefacade() method should also be implemented.

get_per_test_enginefacade(): Return an enginefacade._TransactionContextManager per test.
This facade should be the one that the test expects the code to use. Usually this is the same one returned by get_engineafacade() which is the default. For special applications like Gnocchi, this can be overridden to provide an instance-level facade.

class oslo_db.sqlalchemy.test_fixtures.DeletesFromSchema

Bases: oslo_db.sqlalchemy.test_fixtures.ResetsData

Mixin defining a fixture that can delete from all tables in place.

When DeletesFromSchema is present in a fixture, _DROP_SCHEMA_PER_TEST is now False; this means that the "teardown" flag of provision.SchemaResource will be False, which prevents SchemaResource from dropping all objects within the schema after each test.

This is a "capability" mixin that works in conjunction with classes that include BaseDbFixture as a base.

delete_from_schema(engine): A hook which should delete all data from an existing schema.
Should not drop any objects, just remove data from tables that needs to be reset between tests.

reset_schema_data(engine, facade): Reset the data in the schema.

class oslo_db.sqlalchemy.test_fixtures.GeneratesSchema

Bases: object

Mixin defining a fixture as generating a schema using create_all().

This is a "capability" mixin that works in conjunction with classes that include BaseDbFixture as a base.

generate_schema_create_all(engine): A hook which should generate the model schema using create_all().
This hook is called within the scope of creating the database assuming BUILD_WITH_MIGRATIONS is False.

class oslo_db.sqlalchemy.test_fixtures.GeneratesSchemaFromMigrations

Bases: oslo_db.sqlalchemy.test_fixtures.GeneratesSchema

Mixin defining a fixture as generating a schema using migrations.

This is a "capability" mixin that works in conjunction with classes that include BaseDbFixture as a base.

generate_schema_migrations(engine): A hook which should generate the model schema using migrations.
This hook is called within the scope of creating the database assuming BUILD_WITH_MIGRATIONS is True.

class oslo_db.sqlalchemy.test_fixtures.MySQLOpportunisticFixture(test, driver=None, ident=None): Bases: oslo_db.sqlalchemy.test_fixtures.OpportunisticDbFixture

DRIVER = 'mysql'

class oslo_db.sqlalchemy.test_fixtures.OpportunisticDBTestMixin

Bases: object

Test mixin that integrates the test suite with testresources.

There are three goals to this system:

1.: Allow creation of "stub" test suites that will run all the tests in a parent suite against a specific kind of database (e.g. Mysql, Postgresql), where the entire suite will be skipped if that target kind of database is not available to the suite.
2.: provide a test with a process-local, anonymously named schema within a target database, so that the test can run concurrently with other tests without conflicting data
3.: provide compatibility with the testresources.OptimisingTestSuite, which organizes TestCase instances ahead of time into groups that all make use of the same type of database, setting up and tearing down a database schema once for the scope of any number of tests within. This technique is essential when testing against a non-SQLite database because building of a schema is expensive, and also is most ideally accomplished using the applications schema migration which are even more vastly slow than a straight create_all().

This mixin provides the .resources attribute required by testresources when using the OptimisingTestSuite.The .resources attribute then provides a collection of testresources.TestResourceManager objects, which are defined here in oslo_db.sqlalchemy.provision. These objects know how to find available database backends, build up temporary databases, and invoke schema generation and teardown instructions. The actual "build the schema objects" part of the equation, and optionally a "delete from all the tables" step, is provided by the implementing application itself.

FIXTURE: alias of oslo_db.sqlalchemy.test_fixtures.OpportunisticDbFixture

SKIP_ON_UNAVAILABLE_DB = True

generate_fixtures()

property resources: Provide a collection of TestResourceManager objects.
The collection here is memoized, both at the level of the test case itself, as well as in the fixture object(s) which provide those resources.

setUp()

class oslo_db.sqlalchemy.test_fixtures.OpportunisticDbFixture(test, driver=None, ident=None)

Bases: oslo_db.sqlalchemy.test_fixtures.BaseDbFixture

Fixture which uses testresources fully for optimised runs.

This fixture relies upon the use of the OpportunisticDBTestMixin to supply a test.resources attribute, and also works much more effectively when combined the testresources.OptimisingTestSuite. The optimize_package_test_loader() function should be used at the module and package levels to optimize database provisioning across many tests.

class oslo_db.sqlalchemy.test_fixtures.PostgresqlOpportunisticFixture(test, driver=None, ident=None): Bases: oslo_db.sqlalchemy.test_fixtures.OpportunisticDbFixture

DRIVER = 'postgresql'

class oslo_db.sqlalchemy.test_fixtures.ReplaceEngineFacadeFixture(enginefacade, replace_with_enginefacade)

Bases: fixtures.fixture.Fixture

A fixture that will plug the engine of one enginefacade into another.

This fixture can be used by test suites that already have their own non- oslo_db database setup / teardown schemes, to plug any URL or test-oriented enginefacade as-is into an enginefacade-oriented API.

For applications that use oslo.db's testing fixtures, the ReplaceEngineFacade fixture is used internally.

E.g.:

class MyDBTest(TestCase):


    def setUp(self):


        from myapplication.api import main_enginefacade


        my_test_enginefacade = enginefacade.transaction_context()


        my_test_enginefacade.configure(connection=my_test_url)


        self.useFixture(


            ReplaceEngineFacadeFixture(


                main_enginefacade, my_test_enginefacade))

Above, the main_enginefacade object is the normal application level one, and my_test_enginefacade is a local one that we've created to refer to some testing database. Throughout the fixture's setup, the application level enginefacade will use the engine factory and engines of the testing enginefacade, and at fixture teardown will be replaced back.

class oslo_db.sqlalchemy.test_fixtures.ResetsData: Bases: object
Mixin defining a fixture that resets schema data without dropping.

reset_schema_data(engine, enginefacade): Reset the data in the schema.

setup_for_reset(engine, enginefacade): "Perform setup that may be needed before the test runs.

class oslo_db.sqlalchemy.test_fixtures.SimpleDbFixture(driver=None, ident=None)

Bases: oslo_db.sqlalchemy.test_fixtures.BaseDbFixture

Fixture which provides an engine from a fixed URL.

The SimpleDbFixture is generally appropriate only for a SQLite memory database, as this database is naturally isolated from other processes and does not require management of schemas. For tests that need to run specifically against MySQL or Postgresql, the OpportunisticDbFixture is more appropriate.

The database connection information itself comes from the provisoning system, matching the desired driver (typically sqlite) to the default URL that provisioning provides for this driver (in the case of sqlite, it's the SQLite memory URL, e.g. sqlite://. For MySQL and Postgresql, it's the familiar "openstack_citest" URL on localhost).

There are a variety of create/drop schemes that can take place:

The default is to procure a database connection on setup, and at teardown, an instruction is issued to "drop" all objects in the schema (e.g. tables, indexes). The SQLAlchemy engine itself remains referenced at the class level for subsequent re-use.
When the GeneratesSchema or GeneratesSchemaFromMigrations mixins are implemented, the appropriate generate_schema method is also called when the fixture is set up, by default this is per test.
When the DeletesFromSchema mixin is implemented, the generate_schema method is now only called once, and the "drop all objects" system is replaced with the delete_from_schema method. This allows the same database to remain set up with all schema objects intact, so that expensive migrations need not be run on every test.
The fixture does not dispose the engine at the end of a test. It is assumed the same engine will be re-used many times across many tests. The AdHocDbFixture extends this one to provide engine.dispose() at the end of a test.

This fixture is intended to work without needing a reference to the test itself, and therefore cannot take advantage of the OptimisingTestSuite.

oslo_db.sqlalchemy.test_fixtures.optimize_module_test_loader()

Organize module-level tests into a testresources.OptimizingTestSuite.

This function provides a unittest-compatible load_tests hook for a given module; for per-package, use the optimize_package_test_loader() function.

When a unitest or subunit style test runner is used, the function will be called in order to return a TestSuite containing the tests to run; this function ensures that this suite is an OptimisingTestSuite, which will organize the production of test resources across groups of tests at once.

The function is invoked as:

from oslo_db.sqlalchemy import test_base
load_tests = test_base.optimize_module_test_loader()

The loader must be present in an individual module, and not the package level __init__.py.

The function also applies testscenarios expansion to all test collections. This so that an existing test suite that already needs to build TestScenarios from a load_tests call can still have this take place when replaced with this function.

oslo_db.sqlalchemy.test_fixtures.optimize_package_test_loader(file_)

Organize package-level tests into a testresources.OptimizingTestSuite.

This function provides a unittest-compatible load_tests hook for a given package; for per-module, use the optimize_module_test_loader() function.

The function is invoked as:

from oslo_db.sqlalchemy import test_base
load_tests = test_base.optimize_package_test_loader(__file__)

The loader must be present in the package level __init__.py.

oslo_db.sqlalchemy.test_migrations module

class oslo_db.sqlalchemy.test_migrations.ModelsMigrationsSync

Bases: object

A helper class for comparison of DB migration scripts and models.

It's intended to be inherited by test cases in target projects. They have to provide implementations for methods used internally in the test (as we have no way to implement them here).

test_model_sync() will run migration scripts for the engine provided and then compare the given metadata to the one reflected from the database. The difference between MODELS and MIGRATION scripts will be printed and the test will fail, if the difference is not empty. The return value is really a list of actions, that should be performed in order to make the current database schema state (i.e. migration scripts) consistent with models definitions. It's left up to developers to analyze the output and decide whether the models definitions or the migration scripts should be modified to make them consistent.

Output:

[(


    'add_table',


    description of the table from models
),
(


    'remove_table',


    description of the table from database
),
(


    'add_column',


    schema,


    table name,


    column description from models
),
(


    'remove_column',


    schema,


    table name,


    column description from database
),
(


    'add_index',


    description of the index from models
),
(


    'remove_index',


    description of the index from database
),
(


    'add_constraint',


    description of constraint from models
),
(


    'remove_constraint,


    description of constraint from database
),
(


    'modify_nullable',


    schema,


    table name,


    column name,


    {


        'existing_type': type of the column from database,


        'existing_server_default': default value from database


    },


    nullable from database,


    nullable from models
),
(


    'modify_type',


    schema,


    table name,


    column name,


    {


        'existing_nullable': database nullable,


        'existing_server_default': default value from database


    },


    database column type,


    type of the column from models
),
(


    'modify_default',


    schema,


    table name,


    column name,


    {


        'existing_nullable': database nullable,


        'existing_type': type of the column from database


    },


    connection column default value,


    default from models
)]

Method include_object() can be overridden to exclude some tables from comparison (e.g. migrate_repo).

compare_server_default(ctxt, ins_col, meta_col, insp_def, meta_def, rendered_meta_def): Compare default values between model and db table.
Return True if the defaults are different, False if not, or None to allow the default implementation to compare these defaults.

Parameters

ctxt -- alembic MigrationContext instance
insp_col -- reflected column
meta_col -- column from model
insp_def -- reflected column default value
meta_def -- column default value from model
rendered_meta_def -- rendered column default value (from model)

compare_type(ctxt, insp_col, meta_col, insp_type, meta_type): Return True if types are different, False if not.
Return None to allow the default implementation to compare these types.

Parameters

ctxt -- alembic MigrationContext instance
insp_col -- reflected column
meta_col -- column from model
insp_type -- reflected column type
meta_type -- column type from model

abstract db_sync(engine): Run migration scripts with the given engine instance.
This method must be implemented in subclasses and run migration scripts for a DB the given engine is connected to.

filter_metadata_diff(diff): Filter changes before assert in test_models_sync().
Allow subclasses to whitelist/blacklist changes. By default, no filtering is performed, changes are returned as is.

Parameters: diff -- a list of differences (see compare_metadata() docs for details on format)
Returns: a list of differences

abstract get_engine(): Return the engine instance to be used when running tests.
This method must be implemented in subclasses and return an engine instance to be used when running tests.

abstract get_metadata(): Return the metadata instance to be used for schema comparison.
This method must be implemented in subclasses and return the metadata instance attached to the BASE model.

include_object(object_, name, type_, reflected, compare_to): Return True for objects that should be compared.

Parameters

object -- a SchemaItem object such as a Table or Column object
name -- the name of the object
type -- a string describing the type of object (e.g. "table")
reflected -- True if the given object was produced based on table reflection, False if it's from a local MetaData object
compare_to -- the object being compared against, if available, else None

test_models_sync()

class oslo_db.sqlalchemy.test_migrations.WalkVersionsMixin

Bases: object

Test mixin to check upgrade and downgrade ability of migration.

This is only suitable for testing of migrate migration scripts. An abstract class mixin. INIT_VERSION, REPOSITORY and migration_api attributes must be implemented in subclasses.

Auxiliary Methods:

migrate_up and migrate_down instance methods of the class can be used with auxiliary methods named _pre_upgrade_<revision_id>, _check_<revision_id>, _post_downgrade_<revision_id>. The methods intended to check applied changes for correctness of data operations. This methods should be implemented for every particular revision which you want to check with data. Implementation recommendations for _pre_upgrade_<revision_id>, _check_<revision_id>, _post_downgrade_<revision_id> implementation:

•

_pre_upgrade_<revision_id>: provide a data appropriate to: a next revision. Should be used an id of revision which going to be applied.

•

_check_<revision_id>: Insert, select, delete operations: with newly applied changes. The data provided by _pre_upgrade_<revision_id> will be used.

•: _post_downgrade_<revision_id>: check for absence (inability to use) changes provided by reverted revision.

Execution order of auxiliary methods when revision is upgrading:

_pre_upgrade_### => upgrade => _check_###

Execution order of auxiliary methods when revision is downgrading:

downgrade => _post_downgrade_###

abstract property INIT_VERSION: Initial version of a migration repository.
Can be different from 0, if a migrations were squashed.

Return type: int

abstract property REPOSITORY: Allows basic manipulation with migration repository.

Returns: migrate.versioning.repository.Repository subclass.

migrate_down(version, with_data=False): Migrate down to a previous version of the db.

Parameters

version (str) -- id of revision to downgrade.
with_data (Bool) -- Whether to verify the absence of changes from migration(s) being downgraded, see Auxiliary Methods.

abstract property migrate_engine

Provides engine instance.

Should be the same instance as used when migrations are applied. In most cases, the engine attribute provided by the test class in a setUp method will work.

Example of implementation:

def migrate_engine(self):: return self.engine

Returns: sqlalchemy engine instance

migrate_up(version, with_data=False): Migrate up to a new version of the db.

Parameters

version (str) -- id of revision to upgrade.
with_data (Bool) -- Whether to verify the applied changes with data, see Auxiliary Methods.

abstract property migration_api: Provides API for upgrading, downgrading and version manipulations.

Returns: migrate.api or overloaded analog.

walk_versions(snake_walk=False, downgrade=True)

Check if migration upgrades and downgrades successfully.

Determine the latest version script from the repo, then upgrade from 1 through to the latest, with no data in the databases. This just checks that the schema itself upgrades successfully.

walk_versions calls migrate_up and migrate_down with with_data argument to check changes with data, but these methods can be called without any extra check outside of walk_versions method.

Parameters

•

snake_walk (bool) --

enables checking that each individual migration can be upgraded/downgraded by itself.

If we have ordered migrations 123abc, 456def, 789ghi and we run upgrading with the snake_walk argument set to True, the migrations will be applied in the following order:

`123abc => 456def => 123abc =>


 456def => 789ghi => 456def => 789ghi`

•: downgrade (bool) -- Check downgrade behavior if True.

oslo_db.sqlalchemy.types module

class oslo_db.sqlalchemy.types.JsonEncodedDict(mysql_as_long=False, mysql_as_medium=False)

Bases: oslo_db.sqlalchemy.types.JsonEncodedType

Represents dict serialized as json-encoded string in db.

Note that this type does NOT track mutations. If you want to update it, you have to assign existing value to a temporary variable, update, then assign back. See this page for more robust work around: http://docs.sqlalchemy.org/en/rel_1_0/orm/extensions/mutable.html

cache_ok = True: This type is safe to cache.

type: alias of dict

class oslo_db.sqlalchemy.types.JsonEncodedList(mysql_as_long=False, mysql_as_medium=False)

Bases: oslo_db.sqlalchemy.types.JsonEncodedType

Represents list serialized as json-encoded string in db.

cache_ok = True: This type is safe to cache.

type: alias of list

class oslo_db.sqlalchemy.types.JsonEncodedType(mysql_as_long=False, mysql_as_medium=False): Bases: sqlalchemy.sql.type_api.TypeDecorator
Base column type for data serialized as JSON-encoded string in db.

cache_ok = True: This type is safe to cache.

impl: alias of sqlalchemy.sql.sqltypes.Text

process_bind_param(value, dialect): Bind parameters to the process.

process_result_value(value, dialect): Process result value.

type = None

class oslo_db.sqlalchemy.types.SoftDeleteInteger(*args, **kwargs)

Bases: sqlalchemy.sql.type_api.TypeDecorator

Coerce a bound param to be a proper integer before passing it to DBAPI.

Some backends like PostgreSQL are very strict about types and do not perform automatic type casts, e.g. when trying to INSERT a boolean value like false into an integer column. Coercing of the bound param in DB layer by the means of a custom SQLAlchemy type decorator makes sure we always pass a proper integer value to a DBAPI implementation.

This is not a general purpose boolean integer type as it specifically allows for arbitrary positive integers outside of the boolean int range (0, 1, False, True), so that it's possible to have compound unique constraints over multiple columns including deleted (e.g. to soft-delete flavors with the same name in Nova without triggering a constraint violation): deleted is set to be equal to a PK int value on deletion, 0 denotes a non-deleted row.

cache_ok = True: This type is safe to cache.

impl: alias of sqlalchemy.sql.sqltypes.Integer

process_bind_param(value, dialect): Return the binding parameter.

class oslo_db.sqlalchemy.types.String(length, mysql_ndb_length=None, mysql_ndb_type=None, **kw)

Bases: sqlalchemy.sql.sqltypes.String

String subclass that implements oslo_db specific options.

Initial goal is to support ndb-specific flags.

mysql_ndb_type is used to override the String with another data type. mysql_ndb_size is used to adjust the length of the String.

cache_ok = True: This type is safe to cache.

oslo_db.sqlalchemy.update_match module

exception oslo_db.sqlalchemy.update_match.CantUpdateException: Bases: Exception

exception oslo_db.sqlalchemy.update_match.MultiRowsMatched: Bases: oslo_db.sqlalchemy.update_match.CantUpdateException

exception oslo_db.sqlalchemy.update_match.NoRowsMatched: Bases: oslo_db.sqlalchemy.update_match.CantUpdateException

oslo_db.sqlalchemy.update_match.manufacture_criteria(mapped, values)

Given a mapper/class and a namespace of values, produce a WHERE clause.

The class should be a mapped class and the entries in the dictionary correspond to mapped attribute names on the class.

A value may also be a tuple in which case that particular attribute will be compared to a tuple using IN. The scalar value or tuple can also contain None which translates to an IS NULL, that is properly joined with OR against an IN expression if appropriate.

Parameters

cls -- a mapped class, or actual Mapper object.
values -- dictionary of values.

oslo_db.sqlalchemy.update_match.manufacture_entity_criteria(entity, include_only=None, exclude=None)

Given a mapped instance, produce a WHERE clause.

The attributes set upon the instance will be combined to produce a SQL expression using the mapped SQL expressions as the base of comparison.

Values on the instance may be set as tuples in which case the criteria will produce an IN clause. None is also acceptable as a scalar or tuple entry, which will produce IS NULL that is properly joined with an OR against an IN expression if appropriate.

Parameters

entity -- a mapped entity.
include_only -- optional sequence of keys to limit which keys are included.
exclude -- sequence of keys to exclude

oslo_db.sqlalchemy.update_match.manufacture_persistent_object(session, specimen, values=None, primary_key=None)

Make an ORM-mapped object persistent in a Session without SQL.

The persistent object is returned.

If a matching object is already present in the given session, the specimen is merged into it and the persistent object returned. Otherwise, the specimen itself is made persistent and is returned.

The object must contain a full primary key, or provide it via the values or primary_key parameters. The object is peristed to the Session in a "clean" state with no pending changes.

Parameters

session -- A Session object.
specimen -- a mapped object which is typically transient.
values -- a dictionary of values to be applied to the specimen, in addition to the state that's already on it. The attributes will be set such that no history is created; the object remains clean.
primary_key -- optional tuple-based primary key. This will also be applied to the instance if present.

oslo_db.sqlalchemy.update_match.update_on_match(query, specimen, surrogate_key, values=None, attempts=3, include_only=None, process_query=None, handle_failure=None): Emit an UPDATE statement matching the given specimen.
E.g.:

with enginefacade.writer() as session:


    specimen = MyInstance(


        uuid='ccea54f',


        interface_id='ad33fea',


        vm_state='SOME_VM_STATE',


    )


    values = {


        'vm_state': 'SOME_NEW_VM_STATE'


    }


    base_query = model_query(


        context, models.Instance,


        project_only=True, session=session)


    hostname_query = model_query(


            context, models.Instance, session=session,


            read_deleted='no').


        filter(func.lower(models.Instance.hostname) == 'SOMEHOSTNAME')


    surrogate_key = ('uuid', )


    def process_query(query):


        return query.where(~exists(hostname_query))


    def handle_failure(query):


        try:


            instance = base_query.one()


        except NoResultFound:


            raise exception.InstanceNotFound(instance_id=instance_uuid)


        if session.query(hostname_query.exists()).scalar():


            raise exception.InstanceExists(


                name=values['hostname'].lower())


        # try again


        return False


    persistent_instance = base_query.update_on_match(


        specimen,


        surrogate_key,


        values=values,


        process_query=process_query,


        handle_failure=handle_failure


    )

The UPDATE statement is constructed against the given specimen using those values which are present to construct a WHERE clause. If the specimen contains additional values to be ignored, the include_only parameter may be passed which indicates a sequence of attributes to use when constructing the WHERE.

The UPDATE is performed against an ORM Query, which is created from the given Session, or alternatively by passing the `query parameter referring to an existing query.

Before the query is invoked, it is also passed through the callable sent as process_query, if present. This hook allows additional criteria to be added to the query after it is created but before invocation.

The function will then invoke the UPDATE statement and check for "success" one or more times, up to a maximum of that passed as attempts.

The initial check for "success" from the UPDATE statement is that the number of rows returned matches 1. If zero rows are matched, then the UPDATE statement is assumed to have "failed", and the failure handling phase begins.

The failure handling phase involves invoking the given handle_failure function, if any. This handler can perform additional queries to attempt to figure out why the UPDATE didn't match any rows. The handler, upon detection of the exact failure condition, should throw an exception to exit; if it doesn't, it has the option of returning True or False, where False means the error was not handled, and True means that there was not in fact an error, and the function should return successfully.

If the failure handler is not present, or returns False after attempts number of attempts, then the function overall raises CantUpdateException. If the handler returns True, then the function returns with no error.

The return value of the function is a persistent version of the given specimen; this may be the specimen itself, if no matching object were already present in the session; otherwise, the existing object is returned, with the state of the specimen merged into it. The returned persistent object will have the given values populated into the object.

The object is is returned as "persistent", meaning that it is associated with the given Session and has an identity key (that is, a real primary key value).

In order to produce this identity key, a strategy must be used to determine it as efficiently and safely as possible:

1.: If the given specimen already contained its primary key attributes fully populated, then these attributes were used as criteria in the UPDATE, so we have the primary key value; it is populated directly.
2.: If the target backend supports RETURNING, then when the update() query is performed with a RETURNING clause so that the matching primary key is returned atomically. This currently includes Postgresql, Oracle and others (notably not MySQL or SQLite).
3.: If the target backend is MySQL, and the given model uses a single-column, AUTO_INCREMENT integer primary key value (as is the case for Nova), MySQL's recommended approach of making use of LAST_INSERT_ID(expr) is used to atomically acquire the matching primary key value within the scope of the UPDATE statement, then it fetched immediately following by using SELECT LAST_INSERT_ID(). http://dev.mysql.com/doc/refman/5.0/en/information- functions.html#function_last-insert-id
4.: Otherwise, for composite keys on MySQL or other backends such as SQLite, the row as UPDATED must be re-fetched in order to acquire the primary key value. The surrogate_key parameter is used for this in order to re-fetch the row; this is a column name with a known, unique value where the object can be fetched.

oslo_db.sqlalchemy.update_match.update_returning_pk(query, values, surrogate_key): Perform an UPDATE, returning the primary key of the matched row.
The primary key is returned using a selection of strategies:

if the database supports RETURNING, RETURNING is used to retrieve the primary key values inline.
If the database is MySQL and the entity is mapped to a single integer primary key column, MySQL's last_insert_id() function is used inline within the UPDATE and then upon a second SELECT to get the value.
Otherwise, a "refetch" strategy is used, where a given "surrogate" key value (typically a UUID column on the entity) is used to run a new SELECT against that UUID. This UUID is also placed into the UPDATE query to ensure the row matches.

Parameters

query -- a Query object with existing criterion, against a single entity.
values -- a dictionary of values to be updated on the row.
surrogate_key -- a tuple of (attrname, value), referring to a UNIQUE attribute that will also match the row. This attribute is used to retrieve the row via a SELECT when no optimized strategy exists.

Returns: the primary key, returned as a tuple. Is only returned if rows matched is one. Otherwise, CantUpdateException is raised.

oslo_db.sqlalchemy.utils module

class oslo_db.sqlalchemy.utils.DialectFunctionDispatcher: Bases: object

dispatch_for(expr)

classmethod dispatch_for_dialect(expr, multiple=False): Provide dialect-specific functionality within distinct functions.
e.g.:

@dispatch_for_dialect("*")
def set_special_option(engine):


    pass
@set_special_option.dispatch_for("sqlite")
def set_sqlite_special_option(engine):


    return engine.execute("sqlite thing")
@set_special_option.dispatch_for("mysql+mysqldb")
def set_mysqldb_special_option(engine):


    return engine.execute("mysqldb thing")

After the above registration, the set_special_option() function is now a dispatcher, given a SQLAlchemy Engine, Connection, URL string, or sqlalchemy.engine.URL object:

eng = create_engine('...')
result = set_special_option(eng)

The filter system supports two modes, "multiple" and "single". The default is "single", and requires that one and only one function match for a given backend. In this mode, the function may also have a return value, which will be returned by the top level call.

"multiple" mode, on the other hand, does not support return arguments, but allows for any number of matching functions, where each function will be called:

# the initial call sets this up as a "multiple" dispatcher
@dispatch_for_dialect("*", multiple=True)
def set_options(engine):


    # set options that apply to *all* engines
@set_options.dispatch_for("postgresql")
def set_postgresql_options(engine):


    # set options that apply to all Postgresql engines
@set_options.dispatch_for("postgresql+psycopg2")
def set_postgresql_psycopg2_options(engine):


    # set options that apply only to "postgresql+psycopg2"
@set_options.dispatch_for("*+pyodbc")
def set_pyodbc_options(engine):


    # set options that apply to all pyodbc backends

Note that in both modes, any number of additional arguments can be accepted by member functions. For example, to populate a dictionary of options, it may be passed in:

@dispatch_for_dialect("*", multiple=True)
def set_engine_options(url, opts):


    pass
@set_engine_options.dispatch_for("mysql+mysqldb")
def _mysql_set_default_charset_to_utf8(url, opts):


    opts.setdefault('charset', 'utf-8')
@set_engine_options.dispatch_for("sqlite")
def _set_sqlite_in_memory_check_same_thread(url, opts):


    if url.database in (None, 'memory'):


        opts['check_same_thread'] = False
opts = {}
set_engine_options(url, opts)

The driver specifiers are of the form: <database | *>[+<driver | *>]. That is, database name or "*", followed by an optional + sign with driver or "*". Omitting the driver name implies all drivers for that database.

dispatch_on_drivername(drivername): Return a sub-dispatcher for the given drivername.
This provides a means of calling a different function, such as the "*" function, for a given target object that normally refers to a sub-function.

class oslo_db.sqlalchemy.utils.DialectMultiFunctionDispatcher: Bases: oslo_db.sqlalchemy.utils.DialectFunctionDispatcher

class oslo_db.sqlalchemy.utils.DialectSingleFunctionDispatcher: Bases: oslo_db.sqlalchemy.utils.DialectFunctionDispatcher

oslo_db.sqlalchemy.utils.add_index(engine, table_name, index_name, idx_columns): Create an index for given columns.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table
index_name -- name of the index
idx_columns -- tuple with names of columns that will be indexed

oslo_db.sqlalchemy.utils.change_deleted_column_type_to_boolean(engine, table_name, **col_name_col_instance)

oslo_db.sqlalchemy.utils.change_deleted_column_type_to_id_type(engine, table_name, **col_name_col_instance)

oslo_db.sqlalchemy.utils.change_index_columns(engine, table_name, index_name, new_columns): Change set of columns that are indexed by given index.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table
index_name -- name of the index
new_columns -- tuple with names of columns that will be indexed

oslo_db.sqlalchemy.utils.column_exists(engine, table_name, column): Check if table has given column.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table
column -- name of the colmn

oslo_db.sqlalchemy.utils.dispatch_for_dialect(expr, multiple=False): Provide dialect-specific functionality within distinct functions.
e.g.:

@dispatch_for_dialect("*")
def set_special_option(engine):


    pass
@set_special_option.dispatch_for("sqlite")
def set_sqlite_special_option(engine):


    return engine.execute("sqlite thing")
@set_special_option.dispatch_for("mysql+mysqldb")
def set_mysqldb_special_option(engine):


    return engine.execute("mysqldb thing")

After the above registration, the set_special_option() function is now a dispatcher, given a SQLAlchemy Engine, Connection, URL string, or sqlalchemy.engine.URL object:

eng = create_engine('...')
result = set_special_option(eng)

"multiple" mode, on the other hand, does not support return arguments, but allows for any number of matching functions, where each function will be called:

# the initial call sets this up as a "multiple" dispatcher
@dispatch_for_dialect("*", multiple=True)
def set_options(engine):


    # set options that apply to *all* engines
@set_options.dispatch_for("postgresql")
def set_postgresql_options(engine):


    # set options that apply to all Postgresql engines
@set_options.dispatch_for("postgresql+psycopg2")
def set_postgresql_psycopg2_options(engine):


    # set options that apply only to "postgresql+psycopg2"
@set_options.dispatch_for("*+pyodbc")
def set_pyodbc_options(engine):


    # set options that apply to all pyodbc backends

Note that in both modes, any number of additional arguments can be accepted by member functions. For example, to populate a dictionary of options, it may be passed in:

@dispatch_for_dialect("*", multiple=True)
def set_engine_options(url, opts):


    pass
@set_engine_options.dispatch_for("mysql+mysqldb")
def _mysql_set_default_charset_to_utf8(url, opts):


    opts.setdefault('charset', 'utf-8')
@set_engine_options.dispatch_for("sqlite")
def _set_sqlite_in_memory_check_same_thread(url, opts):


    if url.database in (None, 'memory'):


        opts['check_same_thread'] = False
opts = {}
set_engine_options(url, opts)

oslo_db.sqlalchemy.utils.drop_index(engine, table_name, index_name): Drop index with given name.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table
index_name -- name of the index

oslo_db.sqlalchemy.utils.drop_old_duplicate_entries_from_table(engine, table_name, use_soft_delete, *uc_column_names): Drop all old rows having the same values for columns in uc_columns.
This method drop (or mark ad deleted if use_soft_delete is True) old duplicate rows form table with name table_name.

Parameters

engine -- Sqlalchemy engine
table_name -- Table with duplicates
use_soft_delete -- If True - values will be marked as deleted, if False - values will be removed from table
uc_column_names -- Unique constraint columns

oslo_db.sqlalchemy.utils.get_db_connection_info(conn_pieces)

oslo_db.sqlalchemy.utils.get_foreign_key_constraint_name(engine, table_name, column_name): Find the name of foreign key in a table, given constrained column name.

Parameters

engine -- a SQLAlchemy engine (or connection)
table_name -- name of table which contains the constraint
column_name -- name of column that is constrained by the foreign key.

Returns: the name of the first foreign key constraint which constrains the given column in the given table.

oslo_db.sqlalchemy.utils.get_indexes(engine, table_name): Get all index list from a given table.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table

oslo_db.sqlalchemy.utils.get_non_innodb_tables(connectable, skip_tables=('migrate_version', 'alembic_version')): Get a list of tables which don't use InnoDB storage engine.

Parameters

connectable -- a SQLAlchemy Engine or a Connection instance
skip_tables -- a list of tables which might have a different storage engine

oslo_db.sqlalchemy.utils.get_non_ndbcluster_tables(connectable, skip_tables=None): Get a list of tables which don't use MySQL Cluster (NDB) storage engine.

Parameters

connectable -- a SQLAlchemy Engine or Connection instance
skip_tables -- a list of tables which might have a different storage engine

oslo_db.sqlalchemy.utils.get_table(engine, name)

Returns an sqlalchemy table dynamically from db.

Needed because the models don't work for us in migrations as models will be far out of sync with the current data.

WARNING:

Do not use this method when creating ForeignKeys in database migrations because sqlalchemy needs the same MetaData object to hold information about the parent table and the reference table in the ForeignKey. This method uses a unique MetaData object per table object so it won't work with ForeignKey creation.

oslo_db.sqlalchemy.utils.get_unique_keys(model): Get a list of sets of unique model keys.

Parameters: model -- the ORM model class
Return type: list of sets of strings
Returns: unique model keys or None if unable to find them

oslo_db.sqlalchemy.utils.index_exists(engine, table_name, index_name): Check if given index exists.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table
index_name -- name of the index

oslo_db.sqlalchemy.utils.index_exists_on_columns(engine, table_name, columns): Check if an index on given columns exists.

Parameters

engine -- sqlalchemy engine
table_name -- name of the table
columns -- a list type of columns that will be checked

oslo_db.sqlalchemy.utils.model_query(model, session, args=None, **kwargs): Query helper for db.sqlalchemy api methods.
This accounts for deleted and project_id fields.

Parameters

model (models.ModelBase) -- Model to query. Must be a subclass of ModelBase.
session (sqlalchemy.orm.session.Session) -- The session to use.
args (tuple) -- Arguments to query. If None - model is used.

Keyword arguments:

Parameters

project_id (iterable, model.__table__.columns.project_id.type, None type) -- If present, allows filtering by project_id(s). Can be either a project_id value, or an iterable of project_id values, or None. If an iterable is passed, only rows whose project_id column value is on the project_id list will be returned. If None is passed, only rows which are not bound to any project, will be returned.
deleted (bool) -- If present, allows filtering by deleted field. If True is passed, only deleted entries will be returned, if False - only existing entries.

Usage:

from oslo_db.sqlalchemy import utils
def get_instance_by_uuid(uuid):


    session = get_session()


    with session.begin()


        return (utils.model_query(models.Instance, session=session)


                     .filter(models.Instance.uuid == uuid)


                     .first())
def get_nodes_stat():


    data = (Node.id, Node.cpu, Node.ram, Node.hdd)


    session = get_session()


    with session.begin()


        return utils.model_query(Node, session=session, args=data).all()

Also you can create your own helper, based on utils.model_query(). For example, it can be useful if you plan to use project_id and deleted parameters from project's context

from oslo_db.sqlalchemy import utils
def _model_query(context, model, session=None, args=None,


                 project_id=None, project_only=False,


                 read_deleted=None):


    # We suppose, that functions ``_get_project_id()`` and


    # ``_get_deleted()`` should handle passed parameters and


    # context object (for example, decide, if we need to restrict a user


    # to query his own entries by project_id or only allow admin to read


    # deleted entries). For return values, we expect to get


    # ``project_id`` and ``deleted``, which are suitable for the


    # ``model_query()`` signature.


    kwargs = {}


    if project_id is not None:


        kwargs['project_id'] = _get_project_id(context, project_id,


                                               project_only)


    if read_deleted is not None:


        kwargs['deleted'] = _get_deleted_dict(context, read_deleted)


    session = session or get_session()


    with session.begin():


        return utils.model_query(model, session=session,


                                 args=args, **kwargs)
def get_instance_by_uuid(context, uuid):


    return (_model_query(context, models.Instance, read_deleted='yes')


                  .filter(models.Instance.uuid == uuid)


                  .first())
def get_nodes_data(context, project_id, project_only='allow_none'):


    data = (Node.id, Node.cpu, Node.ram, Node.hdd)


    return (_model_query(context, Node, args=data, project_id=project_id,


                         project_only=project_only)


                  .all())

oslo_db.sqlalchemy.utils.paginate_query(query, model, limit, sort_keys, marker=None, sort_dir=None, sort_dirs=None)

Returns a query with sorting / pagination criteria added.

Pagination works by requiring a unique sort_key, specified by sort_keys. (If sort_keys is not unique, then we risk looping through values.) We use the last row in the previous page as the 'marker' for pagination. So we must return values that follow the passed marker in the order. With a single-valued sort_key, this would be easy: sort_key > X. With a compound-values sort_key, (k1, k2, k3) we must do this to repeat the lexicographical ordering: (k1 > X1) or (k1 == X1 && k2 > X2) or (k1 == X1 && k2 == X2 && k3 > X3)

We also have to cope with different sort_directions and cases where k2, k3, ... are nullable.

Typically, the id of the last row is used as the client-facing pagination marker, then the actual marker object must be fetched from the db and passed in to us as marker.

The "offset" parameter is intentionally avoided. As offset requires a full scan through the preceding results each time, criteria-based pagination is preferred. See http://use-the-index-luke.com/no-offset for further background.

Parameters

query -- the query object to which we should add paging/sorting
model -- the ORM model class
limit -- maximum number of items to return
sort_keys -- array of attributes by which results should be sorted
marker -- the last item of the previous page; we returns the next results after this value.
sort_dir -- direction in which results should be sorted (asc, desc) suffix -nullsfirst, -nullslast can be added to defined the ordering of null values
sort_dirs -- per-column array of sort_dirs, corresponding to sort_keys

Return type: sqlalchemy.orm.query.Query
Returns: The query with sorting/pagination added.

oslo_db.sqlalchemy.utils.sanitize_db_url(url)

oslo_db.sqlalchemy.utils.suspend_fk_constraints_for_col_alter(engine, table_name, column_name, referents=[])

Detect foreign key constraints, drop, and recreate.

This is used to guard against a column ALTER that on some backends cannot proceed unless foreign key constraints are not present.

e.g.:

from oslo_db.sqlalchemy.util import (


    suspend_fk_constraints_for_col_alter
)
with suspend_fk_constraints_for_col_alter(


    migrate_engine, "user_table",


    referents=[


        "local_user", "nonlocal_user", "project"


    ]):


    user_table.c.domain_id.alter(nullable=False)

Parameters

engine -- a SQLAlchemy engine (or connection)
table_name -- target table name. All foreign key constraints that refer to the table_name / column_name will be dropped and recreated.
column_name -- target column name. all foreign key constraints which refer to this column, either partially or fully, will be dropped and recreated.
referents -- sequence of string table names to search for foreign key constraints. A future version of this function may no longer require this argument, however for the moment it is required.

oslo_db.sqlalchemy.utils.to_list(x, default=None)

Module contents

Submodules

oslo_db.api module

Multiple DB API backend support.

A DB backend module should implement a method named 'get_backend' which takes no arguments. The method can return any object that implements DB API methods.

class oslo_db.api.DBAPI(backend_name, backend_mapping=None, lazy=False, **kwargs)

Bases: object

Initialize the chosen DB API backend.

After initialization API methods is available as normal attributes of DBAPI subclass. Database API methods are supposed to be called as DBAPI instance methods.

Parameters

backend_name (str) -- name of the backend to load
backend_mapping (dict) -- backend name -> module/class to load mapping
lazy (bool) -- load the DB backend lazily on the first DB API method call
use_db_reconnect (bool) -- retry DB transactions on disconnect or not
retry_interval (int) -- seconds between transaction retries
inc_retry_interval (bool) -- increase retry interval or not
max_retry_interval (int) -- max interval value between retries
max_retries (int) -- max number of retries before an error is raised

Default backend_mapping: None
Default lazy: False

classmethod from_config(conf, backend_mapping=None, lazy=False): Initialize DBAPI instance given a config instance.

Parameters

conf (oslo.config.cfg.ConfigOpts) -- oslo.config config instance
backend_mapping (dict) -- backend name -> module/class to load mapping
lazy (bool) -- load the DB backend lazily on the first DB API method call

oslo_db.api.retry_on_deadlock(f): Retry a DB API call if Deadlock was received.
wrap_db_entry will be applied to all db.api functions marked with this decorator.

oslo_db.api.retry_on_request(f): Retry a DB API call if RetryRequest exception was received.
wrap_db_entry will be applied to all db.api functions marked with this decorator.

oslo_db.api.safe_for_db_retry(f)

Indicate api method as safe for re-connection to database.

Database connection retries will be enabled for the decorated api method. Database connection failure can have many causes, which can be temporary. In such cases retry may increase the likelihood of connection.

Usage:

@safe_for_db_retry
def api_method(self):


    self.engine.connect()

Parameters: f (function.) -- database api method.

class oslo_db.api.wrap_db_retry(retry_interval=1, max_retries=20, inc_retry_interval=True, max_retry_interval=10, retry_on_disconnect=False, retry_on_deadlock=False, exception_checker=<function wrap_db_retry.<lambda>>, jitter=False)

Bases: object

Retry db.api methods, if db_error raised

Retry decorated db.api methods. This decorator catches db_error and retries function in a loop until it succeeds, or until maximum retries count will be reached.

Keyword arguments:

Parameters

retry_interval (int or float) -- seconds between transaction retries
max_retries (int) -- max number of retries before an error is raised
inc_retry_interval (bool) -- determine increase retry interval or not
max_retry_interval (int or float) -- max interval value between retries
exception_checker (callable) -- checks if an exception should trigger a retry
jitter (bool) -- determine increase retry interval use jitter or not, jitter is always interpreted as True for a DBDeadlockError

oslo_db.concurrency module

class oslo_db.concurrency.TpoolDbapiWrapper(conf, backend_mapping)

Bases: object

DB API wrapper class.

This wraps the oslo DB API with an option to be able to use eventlet's thread pooling. Since the CONF variable may not be loaded at the time this class is instantiated, we must look at it on the first DB API call.

oslo_db.concurrency.list_opts(): Returns a list of oslo.config options available in this module.

Returns: a list of (group_name, opts) tuples

oslo_db.exception module

DB related custom exceptions.

Custom exceptions intended to determine the causes of specific database errors. This module provides more generic exceptions than the database-specific driver libraries, and so users of oslo.db can catch these no matter which database the application is using. Most of the exceptions are wrappers. Wrapper exceptions take an original exception as positional argument and keep it for purposes of deeper debug.

Example:

try:


    statement(arg)
except sqlalchemy.exc.OperationalError as e:


    raise DBDuplicateEntry(e)

This is useful to determine more specific error cases further at execution, when you need to add some extra information to an error message. Wrapper exceptions takes care about original error message displaying to not to loose low level cause of an error. All the database api exceptions wrapped into the specific exceptions provided belove.

Please use only database related custom exceptions with database manipulations with try/except statement. This is required for consistent handling of database errors.

exception oslo_db.exception.BackendNotAvailable

Bases: Exception

Error raised when a particular database backend is not available

within a test suite.

exception oslo_db.exception.CantStartEngineError: Bases: Exception
Error raised when the enginefacade cannot start up correctly.

exception oslo_db.exception.ColumnError: Bases: Exception
Error raised when no column or an invalid column is found.

exception oslo_db.exception.ContextNotRequestedError

Bases: AttributeError

Error raised when requesting a not-setup enginefacade attribute.

This applies to the session and connection attributes of a user-defined context and/or RequestContext object, when they are accessed within the scope of an enginefacade decorator or context manager, but the context has not requested that attribute (e.g. like "with enginefacade.connection.using(context)" and "context.session" is requested).

exception oslo_db.exception.DBConnectionError(inner_exception=None, cause=None)

Bases: oslo_db.exception.DBError

Wrapped connection specific exception.

Raised when database connection is failed.

exception oslo_db.exception.DBConstraintError(table, check_name, inner_exception=None)

Bases: oslo_db.exception.DBError

Check constraint fails for column error.

Raised when made an attempt to write to a column a value that does not satisfy a CHECK constraint.

Parameters

table (str) -- the table name for which the check fails
check_name (str) -- the table of the check that failed to be satisfied

exception oslo_db.exception.DBDataError(inner_exception=None, cause=None)

Bases: oslo_db.exception.DBError

Raised for errors that are due to problems with the processed data.

E.g. division by zero, numeric value out of range, incorrect data type, etc

exception oslo_db.exception.DBDeadlock(inner_exception=None)

Bases: oslo_db.exception.DBError

Database dead lock error.

Deadlock is a situation that occurs when two or more different database sessions have some data locked, and each database session requests a lock on the data that another, different, session has already locked.

exception oslo_db.exception.DBDuplicateEntry(columns=None, inner_exception=None, value=None)

Bases: oslo_db.exception.DBError

Duplicate entry at unique column error.

Raised when made an attempt to write to a unique column the same entry as existing one. :attr: columns available on an instance of the exception and could be used at error handling:

try:


    instance_type_ref.save()
except DBDuplicateEntry as e:


    if 'colname' in e.columns:


        # Handle error.

Parameters

columns (list) -- a list of unique columns have been attempted to write a duplicate entry.
value -- a value which has been attempted to write. The value will be None, if we can't extract it for a particular database backend. Only MySQL and PostgreSQL 9.x are supported right now.

exception oslo_db.exception.DBError(inner_exception=None, cause=None): Bases: oslo_utils.excutils.CausedByException
Base exception for all custom database exceptions.

Parameters: inner_exception -- an original exception which was wrapped with DBError or its subclasses.

exception oslo_db.exception.DBInvalidUnicodeParameter

Bases: Exception

Database unicode error.

Raised when unicode parameter is passed to a database without encoding directive.

exception oslo_db.exception.DBMigrationError(message)

Bases: oslo_db.exception.DBError

Wrapped migration specific exception.

Raised when migrations couldn't be completed successfully.

exception oslo_db.exception.DBNonExistentConstraint(table, constraint, inner_exception=None): Bases: oslo_db.exception.DBError
Constraint does not exist.

Parameters

table (str) -- table name
constraint -- constraint name

exception oslo_db.exception.DBNonExistentDatabase(database, inner_exception=None): Bases: oslo_db.exception.DBError
Database does not exist.

Parameters: database (str) -- database name

exception oslo_db.exception.DBNonExistentTable(table, inner_exception=None): Bases: oslo_db.exception.DBError
Table does not exist.

Parameters: table (str) -- table name

exception oslo_db.exception.DBNotSupportedError(inner_exception=None, cause=None): Bases: oslo_db.exception.DBError
Raised when a database backend has raised sqla.exc.NotSupportedError

exception oslo_db.exception.DBReferenceError(table, constraint, key, key_table, inner_exception=None): Bases: oslo_db.exception.DBError
Foreign key violation error.

Parameters

table (str) -- a table name in which the reference is directed.
constraint (str) -- a problematic constraint name.
key (str) -- a broken reference key name.
key_table (str) -- a table name which contains the key.

exception oslo_db.exception.InvalidSortKey(key=None): Bases: Exception
A sort key destined for database query usage is invalid.

exception oslo_db.exception.NoEngineContextEstablished

Bases: AttributeError

Error raised for enginefacade attribute access with no context.

This applies to the session and connection attributes of a user-defined context and/or RequestContext object, when they are accessed outside of the scope of an enginefacade decorator or context manager.

The exception is a subclass of AttributeError so that normal Python missing attribute behaviors are maintained, such as support for getattr(context, 'session', None).

exception oslo_db.exception.RetryRequest(inner_exc)

Bases: Exception

Error raised when DB operation needs to be retried.

That could be intentionally raised by the code without any real DB errors.

oslo_db.options module

oslo_db.options.list_opts()

Returns a list of oslo.config options available in the library.

The returned list includes all oslo.config options which may be registered at runtime by the library.

Each element of the list is a tuple. The first element is the name of the group under which the list of elements in the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config files.

The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed to users by this library.

Returns: a list of (group_name, opts) tuples

oslo_db.options.set_defaults(conf, connection=None, max_pool_size=None, max_overflow=None, pool_timeout=None): Set defaults for configuration variables.
Overrides default options values.

Parameters

conf (oslo.config.cfg.ConfigOpts instance.) -- Config instance specified to set default options in it. Using of instances instead of a global config object prevents conflicts between options declaration.
connection (str) -- SQL connection string. Valid SQLite URL forms are: * sqlite:///:memory: (or, sqlite://) * sqlite:///relative/path/to/file.db * sqlite:////absolute/path/to/file.db
max_pool_size (int) -- maximum connections pool size. The size of the pool to be maintained, defaults to 5. This is the largest number of connections that will be kept persistently in the pool. Note that the pool begins with no connections; once this number of connections is requested, that number of connections will remain.
max_overflow (int) -- The maximum overflow size of the pool. When the number of checked-out connections reaches the size set in pool_size, additional connections will be returned up to this limit. When those additional connections are returned to the pool, they are disconnected and discarded. It follows then that the total number of simultaneous connections the pool will allow is pool_size + max_overflow, and the total number of "sleeping" connections the pool will allow is pool_size. max_overflow can be set to -1 to indicate no overflow limit; no limit will be placed on the total number of concurrent connections. Defaults to 10, will be used if value of the parameter in None.
pool_timeout (int) -- The number of seconds to wait before giving up on returning a connection. Defaults to 30, will be used if value of the parameter is None.

Default max_pool_size: 5
Default max_overflow: None
Default pool_timeout: None

oslo_db.warning module

Custom warnings.

exception oslo_db.warning.NotSupportedWarning

Bases: Warning

Warn that an argument or call that was passed is not supported.

This subclasses Warning so that it can be filtered as a distinct category.

SEE ALSO:

https://docs.python.org/2/library/warnings.html

exception oslo_db.warning.OsloDBDeprecationWarning

Bases: DeprecationWarning

Issued per usage of a deprecated API.

This subclasses DeprecationWarning so that it can be filtered as a distinct category.

SEE ALSO:

https://docs.python.org/2/library/warnings.html

Module contents

RELEASE NOTES

INDICES AND TABLES

genindex
modindex
search

AUTHOR

unknown

COPYRIGHT

2023, OpenStack Foundation

December 29, 2023

12.3.1