osloserialization(1)
| OSLOSERIALIZATION(1) | oslo.serialization | OSLOSERIALIZATION(1) |
NAME
osloserialization - oslo.serialization 5.7.0
The oslo serialization library provides support for representing objects in transmittable and storable formats, such as JSON and MessagePack.
CONTENTS
Installation
At the command line:
$ pip install oslo.serialization
Or, if you have virtualenvwrapper installed:
$ mkvirtualenv oslo.serialization $ pip install oslo.serialization
Using oslo.serialization
CHANGES
5.7.0
- Drop fallback for Python < 3.9
- Skip installation to speed up pep8
- Ignore iteritems() and use items() solely
- reno: Update master for unmaintained/2023.1
5.6.0
- Remove explicit pbr dependency
- Add note about requirements lower bounds
- Run pyupgrade to clean up Python 2 syntaxes
- Fix outdated tox minversion
- Declare Python 3.12 support
- Remove Python 3.8 support
- Update master for stable/2024.2
5.5.0
- [jsonutils] Add handling of datetime.date format
- reno: Update master for unmaintained/zed
- Remove old excludes
- Update master for stable/2024.1
- reno: Update master for unmaintained/xena
- reno: Update master for unmaintained/wallaby
- reno: Update master for unmaintained/victoria
5.4.0
- Display coverage report
- reno: Update master for unmaintained/yoga
- Update python classifier in setup.cfg
5.3.0
- pre-commit: Integrate bandit
- pre-commit: Bump versions
- bandit: Disable B411
- Require pytz only in Python < 3.9
- jsonutils: Raise ValueError in case the input can't be converted
- Update master for stable/2023.2
5.2.0
- Remove extra spaces in tox.ini
- Bump bandit
- Fix compatibility with Python 3.8
- Revert "Moves supported python runtimes from version 3.8 to 3.10"
- Moves supported python runtimes from version 3.8 to 3.10
- Implement zoneinfo support to drop dependency to pytz
- Use the new openstack-python3-jobs template
- Update master for stable/2023.1
5.1.1
- •
- Fix issues related to tox4
5.1.0
- Add Python3 antelope unit tests
- Update master for stable/zed
5.0.0
- Drop python3.6/3.7 support in testing runtime
- Remove unnecessary unicode prefixes
- Add Python3 zed unit tests
- Update master for stable/yoga
4.3.0
- Add Python3 yoga unit tests
- Update master for stable/xena
4.2.0
- setup.cfg: Replace dashes with underscores
- setup.cfg: Replace dashes with underscores
- Move flake8 as a pre-commit local target
- Add Python3 xena unit tests
- Update master for stable/wallaby
- Remove lower-constraints remnants
- Drop use of deprecated collections classes
4.1.0
- Fix json to_primitive when using IO OBjects
- Use TOX_CONSTRAINTS_FILE
- Dropping lower constraints testing
- Use py3 as the default runtime for tox
- Fix hacking min version to 3.0.1
- Add Python3 wallaby unit tests
- Update master for stable/victoria
- ignore reno generated artifacts
- Adding pre-commit
4.0.1
- Bump bandit version
- Update openstackdocstheme
4.0.0
- Remove the yamlutils module
- Fix pygments style
3.2.0
- Stop to use the __future__ module
- Switch to newer openstackdocstheme and reno versions
- Remove the unused coding style modules
- Remove translation sections from setup.cfg
- Align contributing doc with oslo's policy
- Bump default tox env from py37 to py38
- Add py38 package metadata
- Add release notes links to doc index
- Add Python3 victoria unit tests
- Update master for stable/ussuri
3.1.1
- Update hacking for Python3
- Use unittest.mock instead of third party mock
- Remove Python 3-specific conditional in dump_as_bytes
- Drop use of six
3.1.0
- Remove long-dead workaround
- Add releasenote to deprecate the yamlutils module
- Deprecated the yamlutils module
- Update the minversion parameter
- remove outdated header
- reword releasenote for py27 support dropping
3.0.0
- [ussuri][goal] Drop python 2.7 support and testing
- tox: Trivial cleanup
- Switch to Ussuri jobs
- tox: Keeping going with docs
- Update master for stable/train
2.29.2
- Add Python 3 Train unit tests
- Cap sphinx for py2 to match global requirements
- Sync Sphinx requirement
- Resolve some issue with tox.ini, setup.cfg
- Dropping the py35 testing
2.29.1
- Replace git.openstack.org URLs with opendev.org URLs
- Explicitly set default_flow_style to False
2.29.0
- OpenDev Migration Patch
- Update master for stable/stein
- Introduce a base yaml parser for all openstack components
2.28.2
- add python 3.7 unit test job
- Update hacking version
- Fix README
- Use template for lower-constraints
- Update mailinglist from dev to discuss
- Clean up .gitignore references to personal tools
- Update the home-page to latest
2.28.1
- add lib-forward-testing-python3 test job
- add python 3.6 unit test job
- import zuul job settings from project-config
- import zuul job settings from project-config
- py37: deal with Exception repr changes
- Update reno for stable/rocky
- Switch to stestr
- Add release notes link to README
2.27.0
- fix tox python3 overrides
- Stop using encoding='utf-8' with msgpack
2.26.0
- Unblock requirements changes
- Remove stale pip-missing-reqs tox test
- Trivial: Update pypi url to new url
- set default python to python3
- add lower-constraints job
- Updated from global requirements
2.25.0
- Follow the new PTI for document build
- Update links in README
- Update reno for stable/queens
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- msgpack-python has been renamed to msgpack
2.23.0
- Updated from global requirements
- Add bandit to pep8 job
2.22.0
- Remove -U from pip install
- Avoid tox_install.sh for constraints support
- Remove setting of version/release from releasenotes
- Updated from global requirements
- Updated from global requirements
2.21.2
- •
- When serializing an exception return its 'repr'
2.21.1
- •
- jsonutils.to_primitive(): add fallback parameter
2.21.0
- Updated from global requirements
- warn developers when can't convert value into primitive
- Updated from global requirements
- Updated from global requirements
- Update reno for stable/pike
- Updated from global requirements
2.20.0
- Revert "Explicitly raise ValueError in to_primitive"
- Update URLs in documents according to document migration
- rearrange existing documentation to fit the new standard layout
- switch from oslosphinx to openstackdocstheme
2.19.1
2.19.0
- Updated from global requirements
- Don't iterate through addresses in netaddr.IPNetwork
- Updated from global requirements
- Updated from global requirements
- Explicitly raise ValueError in to_primitive
- Updated from global requirements
- Updated from global requirements
2.18.0
- •
- Use Sphinx 1.5 warning-is-error
2.17.0
- Updated from global requirements
- [Fix gate]Update test requirement
- Updated from global requirements
- Update reno for stable/ocata
- Remove references to Python 3.4
2.16.0
- Prefer raising the python2.x type error for b64 decode errors
- Add Constraints support
- Don't raise TypeError for invalid b64
- Replace six.iteritems() with .items()
- Replaces uuid.uuid4 with uuidutils.generate_uuid()
- Show team and repo badges on README
2.15.0
- Updated from global requirements
- Updated from global requirements
- Add reno for release notes management
- Updated from global requirements
2.14.0
- Add a title to the API Documentation page
- Updated from global requirements
- modify the home-page info with the developer documentation
2.13.0
- Drop H803 in flake8 ignore list
- Fix serialization of binary strings in Python3
2.12.0
- Updated from global requirements
- Fix parameters of assertEqual are misplaced
- Add Python 3.5 classifier and venv
2.11.0
- Updated from global requirements
- Updated from global requirements
- Use {} instead of dict()
2.10.0
- •
- Updated from global requirements
2.9.0
2.8.0
- Replace TypeError by ValueError in msgpackutils
- Support serializing ipaddress objs with jsonutils
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
2.7.0
- •
- Trivial: ignore openstack/common in flake8 exclude list
2.6.0
- •
- [Trivial] Remove executable privilege of doc/source/conf.py
2.5.0
- Drop babel as requirement since its not used
- Updated from global requirements
- Unified and simplified API for all serializers
- Make msgpack registries copyable (and add __contains__)
- msgpack: fix datetime serialization
2.4.0
- •
- Updated from global requirements
2.3.0
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Remove fallback to simplejson in jsonutils
2.2.0
- Remove iso8601 dependency
- Updated from global requirements
- Updated from global requirements
- Trival: Remove 'MANIFEST.in'
2.1.0
- •
- msgpackutils: drop Python 2.6 support
2.0.0
- Updated from global requirements
- Updated from global requirements
- Remove python 2.6 classifier
- Fix bug track URL
- Remove python 2.6 and cleanup tox.ini
- Updated from global requirements
1.11.0
- Updated from global requirements
- Use versionadded and versionchanged in doc
- Fix coverage configuration and execution
1.10.0
- Add utilities for base64
- Also check for the existence of `items` in to_primitive()
- Move 'history' -> release notes section
- Add jsonutils.dump_as_bytes() function for py3
- Change ignore-errors to ignore_errors
- Updated from global requirements
1.9.0
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
1.8.0
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Updated from global requirements
- Drop stale test - test_jsonutils
1.7.0
- Stop using deprecated oslo_utils.timeutils.strtime
- Add tox target to find missing requirements
- Remove oslo namespace package
- Do not use oslo namespace for oslo.i18n lib
- Updated from global requirements
- Updated from global requirements
- Switch badges from 'pypip.in' to 'shields.io'
1.6.0
- Revert "jsonutils: replace strtime() usage with isoformat()"
- jsonutils: encode dict keys
- jsonutils: simplify simple value handling
- jsonutils: replace strtime() usage with isoformat()
- Advertise support for Python3.4 / Remove support for Python 3.3
- Do not sync run_cross_tests.sh
- Updated from global requirements
- Remove run_cross_tests.sh
- Fixes wrong folder for listing tests
1.5.0
- Expose base msgpack exceptions so users don't need to import
- Uncap library requirements for liberty
- More docstring cleanups/tweaks
- Add docstring(s) to handler registry(s)
- Add pypi download + version badges
- Cleanup README.rst and setup.cfg
- Make the msgpackutils handlers more extendable
- Updated from global requirements
1.4.0
- •
- Updated from global requirements
1.3.0
- add dependency warning to requirements.txt
- Correctly load and dump items with datetime.date(s)
- Avoid using strtime for serializing datetimes
- jsonutils: add set() tests and simplify recursive code
- jsonutils: support UUID encoding
- Use default in dumps()
- Updated from global requirements
- Update Oslo imports to remove namespace package
- Add a messagepack utils helper module
- Bump to hacking 0.10
- Updated from global requirements
- fix bug tracker link in README.rst
1.2.0
- Move files out of the namespace package
- Activate pep8 check that _ is imported
- Updated from global requirements
- Workflow documentation is now in infra-manual
1.1.0
- Add pbr to installation requirements
- Updated from global requirements
- Fix pep8, docs, requirements issues in jsonutils and tests
- Remove extraneous vim editor configuration comments
- Support building wheels (PEP-427)
- Fix coverage testing
- Updated from global requirements
- Support 'built-in' datetime module
1.0.0
- Updated from global requirements
- Add history/changelog to docs
0.2.0
- Check for namedtuple_as_object support before using it
- Work toward Python 3.4 support and testing
0.1.0
- Updated from global requirements
- Clean up docs before first release
- Updated from global requirements
- Added JSONEncoder and JSONDecoder
- Add some basic docstring(s)
- Fix docs build to run properly
- Use oslo libraries - i18n and utils
- Specify namedtuple_as_object=False when using simplejson
- Get unit tests passing
- exported from oslo-incubator by graduate.sh
- Added missing jsonutils.dump() function
- pep8: fixed multiple violations
- Add kwargs to jsonutils.load(s) functions
- Enforce unicode json output for jsonutils.load[s]()
- jsonutils.load() accepts file pointer, not string
- On Python <= 2.6, use simplejson if available
- Use oslotest instead of common test module
- Python3: enable test_jsonutils.py
- Use six.moves.xmlrpc_client instead of xmlrpclib
- Translation Message improvements
- Python3 support for xmlrpclib
- Use six.iteritems to make dict work on Python2/3
- Remove vim header
- Ensure that Message objects will be sent via RPC in unicode format
- jsonutils: make types py3 compatible
- jsonutils: do not require xmlrpclib
- Make dependency on netaddr optional
- Bump hacking to 0.7.0
- Replace using tests.utils with openstack.common.test
- Enable H302 hacking check
- Add netaddr.IPAddress support to to_primitive()
- python3: python3 binary/text data compatbility
- Convert unicode for python3 portability
- Optimise to_primitive common cases
- Update to use flake8
- Fix Copyright Headers - Rename LLC to Foundation
- Don't LOG.error on max_depth (by default)
- to_primitive imposes what seems to be an arbitary data structure depth of 3, but there is at least on case in Nova (Security group Rules) which requires a depth beyond this
- Allow to_primitive to ignore datetimes
- Replace direct use of testtools BaseTestCase
- Use testtools as test base class
- Fixes import order errors
- Make project pyflakes clean
- Add missing convert_instances arg
- Track to_primitive() depth after iteritems()
- Update iteritems test case to actually test iteritems
- Use strtime() in to_primitive() for datetime objs
- Add 'filedecoder' method to the jsonutils wrapper module
- Make jsonutils.dumps more general
- Handle xmlrpclib.DateTime types when serializing to JSON
- Revert "Add a load function wrapper to jsonutils"
- Add a load function wrapper to jsonutils
- Create openstack.common.jsonutils
Contributing
If you would like to contribute to the development of oslo's libraries, first you must take a look to this page:
If you would like to contribute to the development of OpenStack, you must follow the steps in this page: http://docs.openstack.org/infra/manual/developers.html
Once those steps have been completed, changes to OpenStack should be submitted for review via the Gerrit tool, following the workflow documented at: http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad, not GitHub: https://bugs.launchpad.net/oslo.serialization
API Reference
base64
Utilities to encode and decode Base64.
Added in version 1.10.
- oslo_serialization.base64.decode_as_bytes(encoded)
- Decode a Base64 encoded string.
- Parameters
- encoded -- bytes or text Base64 encoded string to be decoded
- Returns
- decoded bytes string (bytes)
Use decode_as_text() to get the decoded string as text.
A TypeError is raised if the input is invalid (or incorrectly padded).
- oslo_serialization.base64.decode_as_text(encoded, encoding='utf-8')
- Decode a Base64 encoded string.
Decode the Base64 string and then decode the result from encoding (UTF-8 by default).
- Parameters
- encoded -- bytes or text Base64 encoded string to be decoded
- Returns
- decoded text string (bytes)
Use decode_as_bytes() to get the decoded string as bytes.
- oslo_serialization.base64.encode_as_bytes(s, encoding='utf-8')
- Encode a string using Base64.
If s is a text string, first encode it to encoding (UTF-8 by default).
- Parameters
- s -- bytes or text string to be encoded
- encoding -- encoding used to encode s if it's a text string
- Returns
- Base64 encoded byte string (bytes)
Use encode_as_text() to get the Base64 encoded string as text.
- oslo_serialization.base64.encode_as_text(s, encoding='utf-8')
- Encode a string using Base64.
If s is a text string, first encode it to encoding (UTF-8 by default).
- Parameters
- s -- bytes or text string to be encoded
- encoding -- encoding used to encode s if it's a text string
- Returns
- Base64 encoded text string (Unicode)
Use encode_as_bytes() to get the Base64 encoded string as bytes.
jsonutils
JSON related utilities.
This module provides a few things:
- 1.
- A handy function for getting an object down to something that can be JSON serialized. See to_primitive().
- 2.
- Wrappers around loads() and dumps(). The dumps() wrapper will automatically use to_primitive() for you if needed.
- 3.
- This sets up anyjson to use the loads() and dumps() wrappers if anyjson is available.
- oslo_serialization.jsonutils.dump(obj, fp, *args, **kwargs)
- Serialize obj as a JSON formatted stream to fp
- Parameters
- obj -- object to be serialized
- fp -- a .write()-supporting file-like object
- default -- function that returns a serializable version of an object, to_primitive() is used by default.
- args -- extra arguments, please see documentation of json.dump
- kwargs --
extra named parameters, please see documentation of json.dump
Changed in version 1.3: The default parameter now uses to_primitive() by default.
- oslo_serialization.jsonutils.dump_as_bytes(obj, default=<function to_primitive>, encoding='utf-8', **kwargs)
- Serialize obj to a JSON formatted bytes.
- Parameters
- obj -- object to be serialized
- default -- function that returns a serializable version of an object, to_primitive() is used by default.
- encoding -- encoding used to encode the serialized JSON output
- kwargs -- extra named parameters, please see documentation of json.dumps
- Returns
- json formatted string
Added in version 1.10.
- oslo_serialization.jsonutils.dumps(obj, default=<function to_primitive>, **kwargs)
- Serialize obj to a JSON formatted str.
- Parameters
- obj -- object to be serialized
- default -- function that returns a serializable version of an object, to_primitive() is used by default.
- kwargs --
extra named parameters, please see documentation of json.dumps
- Returns
- json formatted string
Use dump_as_bytes() to ensure that the result type is bytes on Python 2 and Python 3.
- oslo_serialization.jsonutils.load(fp, encoding='utf-8', **kwargs)
- Deserialize fp to a Python object.
- Parameters
- fp -- a .read() -supporting file-like object
- encoding -- encoding used to interpret the string
- kwargs -- extra named parameters, please see documentation of json.loads
- Returns
- python object
- oslo_serialization.jsonutils.loads(s, encoding='utf-8', **kwargs)
- Deserialize s (a str or unicode instance containing a JSON
- Parameters
- s -- string to deserialize
- encoding -- encoding used to interpret the string
- kwargs --
extra named parameters, please see documentation of json.loads
- Returns
- python object
- oslo_serialization.jsonutils.to_primitive(value, convert_instances=False, convert_datetime=True, level=0, max_depth=3, encoding='utf-8', fallback=None)
- Convert a complex object into primitives.
Handy for JSON serialization. We can optionally handle instances, but since this is a recursive function, we could have cyclical data structures.
To handle cyclical data structures we could track the actual objects visited in a set, but not all objects are hashable. Instead we just track the depth of the object inspections and don't go too deep.
Therefore, convert_instances=True is lossy ... be aware.
If the object cannot be converted to primitive, it is returned unchanged if fallback is not set, return fallback(value) otherwise.
Changed in version 2.22: Added fallback parameter.
Changed in version 1.3: Support UUID encoding.
Changed in version 1.6: Dictionary keys are now also encoded.
msgpackutils
MessagePack related utilities.
This module provides a few things:
- 1.
- A handy registry for getting an object down to something that can be msgpack serialized. See HandlerRegistry.
- 2.
- Wrappers around loads() and dumps(). The dumps() wrapper will automatically use the default_registry for you if needed.
Added in version 1.3.
- class oslo_serialization.msgpackutils.HandlerRegistry
- Registry of type specific msgpack handlers extensions.
See: https://github.com/msgpack/msgpack/blob/master/spec.md#formats-ext
Do note that due to the current limitations in the msgpack python library we can not currently dump/load a tuple without converting it to a list.
This may be fixed in: https://github.com/msgpack/msgpack-python/pull/100
Added in version 1.5.
- copy(unfreeze=False)
- Deep copy the given registry (and its handlers).
- get(identity)
- Get the handler for the given numeric identity (or none).
- match(obj)
- Match the registries handlers to the given object (or none).
- max_value = 127
- Applications can assign 0 to 127 to store application (or library) specific type handlers; see above ranges for what is reserved by this library and what is not.
- min_value = 0
- Applications can assign 0 to 127 to store application (or library) specific type handlers; see above ranges for what is reserved by this library and what is not.
- non_reserved_extension_range = Interval(33, 127)
- These ranges are always reserved for use by applications building their own type specific handlers (the meaning of extensions in this range will typically vary depending on application).
- register(handler, reserved=False, override=False)
- Register a extension handler to handle its associated type.
- reserved_extension_range = Interval(0, 32)
- These ranges are always reserved for use by oslo.serialization and its own add-ons extensions (these extensions are meant to be generally applicable to all of python).
- class oslo_serialization.msgpackutils.Interval(min_value, max_value)
- Small and/or simple immutable integer/float interval class.
Interval checking is inclusive of the min/max boundaries.
- oslo_serialization.msgpackutils.default_registry = <oslo_serialization.msgpackutils.HandlerRegistry object>
- Default, read-only/frozen registry that will be used when none is
provided.
This registry has msgpack extensions for the following:
- DateTime objects.
- Date objects.
- UUID objects.
- itertools.count objects/iterators.
- set and frozenset container(s).
- netaddr.IPAddress objects (only if netaddr is importable).
- xmlrpclib.DateTime datetime objects.
Added in version 1.5.
- oslo_serialization.msgpackutils.dump(obj, fp, registry=None)
- Serialize obj as a messagepack formatted stream to fp.
Changed in version 1.5: Added registry parameter.
- oslo_serialization.msgpackutils.dumps(obj, registry=None)
- Serialize obj to a messagepack formatted str.
Changed in version 1.5: Added registry parameter.
- oslo_serialization.msgpackutils.load(fp, registry=None)
- Deserialize fp into a Python object.
Changed in version 1.5: Added registry parameter.
- oslo_serialization.msgpackutils.loads(s, registry=None)
- Deserialize s messagepack str into a Python object.
Changed in version 1.5: Added registry parameter.
RELEASE NOTES
Read also the oslo.serialization Release Notes.
INDICES AND TABLES
- Index
- Module Index
- Search Page
AUTHOR
Author name not set
COPYRIGHT
2025, OpenStack Foundation
| April 2, 2025 | 5.7.0 |
