amazon.ion package

Submodules

amazon.ion.core module

Ion core types.

class amazon.ion.core.DataEvent

Bases: amazon.ion.util.DataEvent

Event generated as a result of the writer or as input into the reader.

Args:

type (Enum): The type of event. data (bytes): The serialized data returned. If no data is to be serialized,

this should be the empty byte string.

class amazon.ion.core.IonEvent

Bases: amazon.ion.util.IonEvent

An parse or serialization event.

Args:

event_type (IonEventType): The type of event. ion_type (Optional(amazon.ion.core.IonType)): The Ion data model type

associated with the event.

value (Optional[any]): The data value associated with the event. field_name (Optional[Union[amazon.ion.symbols.SymbolToken, unicode]]): The field name

associated with the event.

annotations (Sequence[Union[amazon.ion.symbols.SymbolToken, unicode]]): The annotations

associated with the event.

depth (Optional[int]): The tree depth of the event if applicable.

derive_annotations(annotations)

Derives a new event from this one setting the annotations attribute.

Args:

annotations: (Sequence[Union[amazon.ion.symbols.SymbolToken, unicode]]):

The annotations associated with the derived event.

Returns:

IonEvent: The newly generated event.

derive_depth(depth)

Derives a new event from this one setting the depth attribute.

Args:

depth: (int):

The annotations associated with the derived event.

Returns:

IonEvent: The newly generated event.

derive_field_name(field_name)

Derives a new event from this one setting the field_name attribute.

Args:

field_name (Union[amazon.ion.symbols.SymbolToken, unicode]): The field name to set.

Returns:

IonEvent: The newly generated event.

derive_value(value)

Derives a new event from this one setting the value attribute.

Args:

value: (any):

The value associated with the derived event.

Returns:

IonEvent: The newly generated non-thunk event.

class amazon.ion.core.IonEventType(name, value)

Bases: amazon.ion.util.Enum

Enumeration of Ion parser or serializer events.

These types do not correspond directly to the Ion type system, but they are related. In particular, null.* will surface as a SCALAR even though they are containers.

Attributes:

INCOMPLETE: Indicates that parsing cannot be completed due to lack of input. STREAM_END: Indicates that the logical stream has terminated. VERSION_MARKER: Indicates that the Ion Version Marker has been encountered. SCALAR: Indicates an atomic value has been encountered. CONTAINER_START: Indicates that the start of a container has been reached. CONTAINER_END: Indicates that the end of a container has been reached.

CONTAINER_END = <IonEventType.CONTAINER_END: 3>

CONTAINER_START = <IonEventType.CONTAINER_START: 2>

INCOMPLETE = <IonEventType.INCOMPLETE: -2>

SCALAR = <IonEventType.SCALAR: 1>

STREAM_END = <IonEventType.STREAM_END: -1>

VERSION_MARKER = <IonEventType.VERSION_MARKER: 0>

begins_value

Indicates if the event type is a start of a value.

ends_container

Indicates if the event type terminates a container or stream.

is_stream_signal

Indicates that the event type corresponds to a stream signal.

class amazon.ion.core.IonThunkEvent

Bases: amazon.ion.core.IonEvent

An IonEvent whose value field is a thunk.

value

class amazon.ion.core.IonType(name, value)

Bases: amazon.ion.util.Enum

Enumeration of the Ion data types.

BLOB = <IonType.BLOB: 9>

BOOL = <IonType.BOOL: 1>

CLOB = <IonType.CLOB: 8>

DECIMAL = <IonType.DECIMAL: 4>

FLOAT = <IonType.FLOAT: 3>

INT = <IonType.INT: 2>

LIST = <IonType.LIST: 10>

NULL = <IonType.NULL: 0>

SEXP = <IonType.SEXP: 11>

STRING = <IonType.STRING: 7>

STRUCT = <IonType.STRUCT: 12>

SYMBOL = <IonType.SYMBOL: 6>

TIMESTAMP = <IonType.TIMESTAMP: 5>

is_container

Returns whether the type is a container.

is_lob

Returns whether the type is a LOB.

is_numeric

is_text

Returns whether the type is a Unicode textual type.

class amazon.ion.core.MemoizingThunk(delegate)

Bases: object

A callable that invokes a delegate and caches and returns the result.

class amazon.ion.core.Multimap(*args, **kwargs)

Bases: _abcoll.MutableMapping

Dictionary that can hold multiple values for the same key

In order not to break existing customers, getting and inserting elements with [] keeps the same behaviour as the built-in dict. If multiple elements are already mapped to the key, ``[]` will return the newest one.

To map multiple elements to a key, use the add_item operation. To retrieve all the values map to a key, use get_all_values.

add_item(key, value)

get_all_values(key)

items() → list of D's (key, value) pairs, as 2-tuples

iteritems() → an iterator over the (key, value) items of D

class amazon.ion.core.MultimapValue(*args)

Bases: _abcoll.MutableSequence

insert(index, value)

S.insert(index, object) – insert object before index

class amazon.ion.core.OffsetTZInfo(delta=datetime.timedelta(0))

Bases: datetime.tzinfo

A trivial UTC offset tzinfo.

dst(date_time)

datetime -> DST offset in minutes east of UTC.

tzname(date_time)

datetime -> string name of time zone.

utcoffset(date_time)

datetime -> minutes east of UTC (negative for west of UTC).

class amazon.ion.core.Timestamp

Bases: datetime.datetime

Sub-class of datetime that supports a precision field; a fractional_precision field that specifies the precision of the``microseconds`` field in datetime; and a fractional_seconds field that is a Decimal specifying the fractional seconds precisely.

Notes:

• The precision field is passed as a keyword argument of the same name.
• The fractional_precision field is passed as a keyword argument of the same name. This field only relates to to the microseconds field and can be thought of as the number of decimal digits that are significant. This is an integer that that is in the closed interval [0, 6]. If 0, microseconds must be 0 indicating no precision below seconds. This argument is optional and only valid when microseconds is not None. If the microseconds specified has more precision than this field indicates, then that is an error.
• The fractional_seconds field is passed as a keyword argument of the same name. It must be a Decimal in the left-closed, right-opened interval of [0, 1). If specified as an argument, microseconds must be None and fractional_precision must not be specified (but can be None). In addition, if microseconds is specified this argument must not be specified (but can be None). If the specified value has coefficient==0 and exponent >= 0, e.g. Decimal(0), then there is no precision beyond seconds.
• After construction, microseconds, fractional_precision, and fractional_seconds will all be present and normalized in the resulting Timestamp instance. If the precision of fractional_seconds is more than is capable of being expressed in microseconds, then the microseconds field is truncated to six digits and fractional_precision is 6.

Consider some examples:

• 2019-10-01T12:45:01Z would have the following fields set: * microseconds == 0, fractional_precision == 0, fractional_seconds == Decimal('0')
• 2019-10-01T12:45:01.100Z would have the following fields set: * microseconds == 100000, fractional_precision == 3, fractional_seconds == Decimal('0.100')
• 2019-10-01T12:45:01.123456789Z would have the following fields set: * microseconds == 123456, fractional_precision == 6, fractional_seconds == Decimal('0.123456789')

Raises:

ValueError: If any of the preconditions above are violated.

static adjust_from_utc_fields(*args, **kwargs)

Constructs a timestamp from UTC fields adjusted to the local offset if given.

fractional_precision

fractional_seconds

precision

class amazon.ion.core.TimestampPrecision(name, value)

Bases: amazon.ion.util.Enum

The different levels of precision supported in an Ion timestamp.

DAY = <TimestampPrecision.DAY: 2>

MINUTE = <TimestampPrecision.MINUTE: 3>

MONTH = <TimestampPrecision.MONTH: 1>

SECOND = <TimestampPrecision.SECOND: 4>

YEAR = <TimestampPrecision.YEAR: 0>

includes_day

Precision has at least the day field.

includes_minute

Precision has at least the minute field.

includes_month

Precision has at least the month field.

includes_second

Precision has at least the second field.

class amazon.ion.core.Transition

Bases: amazon.ion.util.Transition

A pair of event and co-routine delegate.

This is generally used as a result of a state-machine.

Args:

event (Union[DataEvent]): The event associated with the transition. delegate (Coroutine): The co-routine delegate which can be the same routine from

whence this transition came.

amazon.ion.core.timestamp(year, month=1, day=1, hour=0, minute=0, second=0, microsecond=None, off_hours=None, off_minutes=None, precision=None, fractional_precision=None, fractional_seconds=None)

Shorthand for the Timestamp constructor.

Specifically, converts off_hours and off_minutes parameters to a suitable OffsetTZInfo instance.

amazon.ion.equivalence module

Provides utilities for determining whether two objects are equivalent under the Ion data model.

amazon.ion.equivalence.ion_equals(a, b, timestamps_instants_only=False)

Tests two objects for equivalence under the Ion data model.

There are three important cases:

• When neither operand specifies its ion_type or annotations, this method will only return True when the values of both operands are equivalent under the Ion data model.
• When only one of the operands specifies its ion_type and annotations, this method will only return True when that operand has no annotations and has a value equivalent to the other operand under the Ion data model.
• When both operands specify ion_type and annotations, this method will only return True when the ion_type and annotations of both are the same and their values are equivalent under the Ion data model.

Note that the order of the operands does not matter.

Args:

a (object): The first operand. b (object): The second operand. timestamps_instants_only (Optional[bool]): False if timestamp objects (datetime and its subclasses) should be

compared according to the Ion data model (where the instant, precision, and offset must be equal); True if these objects should be considered equivalent if they simply represent the same instant.

amazon.ion.exceptions module

Exceptions for Ion Python.

exception amazon.ion.exceptions.CannotSubstituteTable

Bases: amazon.ion.exceptions.IonException

Indicates a problem with resolving symbols tables.

exception amazon.ion.exceptions.IonException

Bases: exceptions.Exception

Root exception for Ion Python.

amazon.ion.reader module

Provides common functionality for Ion binary and text readers.

class amazon.ion.reader.BufferQueue(is_unicode=False)

Bases: object

A simple circular buffer of buffers.

extend(data)

static is_eof(c)

mark_eof()

read(length, skip=False)

Consumes the first length bytes from the accumulator.

read_byte()

skip(length)

Removes length bytes and returns the number length still required to skip

unread(c)

Unread the given character, byte, or code point.

If this is a unicode buffer and the input is an int or byte, it will be interpreted as an ordinal representing a unicode code point.

If this is a binary buffer, the input must be a byte or int; a unicode character will raise an error.

class amazon.ion.reader.CodePoint(*args, **kwargs)

Bases: int

Evaluates as a code point ordinal, while also containing the unicode character representation and indicating whether the code point was escaped.

class amazon.ion.reader.CodePointArray(initial_bytes=None)

A mutable sequence of code points. Used in place of bytearray() for text values.

append(value)

as_symbol()

as_text()

extend(values)

insert(index, value)

class amazon.ion.reader.ReadEventType(name, value)

Bases: amazon.ion.util.Enum

Events that are pushed into an Ion reader co-routine.

Attributes:

DATA: Indicates more data for the reader. The expected type is bytes. NEXT: Indicates that the reader should yield the next event. SKIP: Indicates that the reader should proceed to the end of the current container.

This type is not meaningful at the top-level.

DATA = <ReadEventType.DATA: 0>

NEXT = <ReadEventType.NEXT: 1>

SKIP = <ReadEventType.SKIP: 2>

amazon.ion.reader.read_data_event(data)

Simple wrapper over the DataEvent constructor to wrap a bytes like with the DATA ReadEventType.

Args:

data (bytes|unicode): The data for the event. Bytes are accepted by both binary and text readers, while unicode

is accepted by text readers with is_unicode=True.

amazon.ion.reader_binary module

amazon.ion.reader_binary.binary_reader(queue=None)

Returns a raw binary reader co-routine.

Args:

queue (Optional[BufferQueue]): The buffer read data for parsing, if None a

new one will be created.

Yields:

IonEvent: parse events, will have an event type of INCOMPLETE if data is needed

in the middle of a value or STREAM_END if there is no data and the parser is not in the middle of parsing a value.

Receives DataEvent, with ReadEventType of NEXT or SKIP to iterate over values, or DATA if the last event was a INCOMPLETE or STREAM_END event type.

SKIP is only allowed within a container. A reader is in a container when the CONTAINER_START event type is encountered and not in a container when the CONTAINER_END event type for that container is encountered.

amazon.ion.reader_binary.raw_reader(queue=None)

Returns a raw binary reader co-routine.

Args:

queue (Optional[BufferQueue]): The buffer read data for parsing, if None a

new one will be created.

Yields:

IonEvent: parse events, will have an event type of INCOMPLETE if data is needed

in the middle of a value or STREAM_END if there is no data and the parser is not in the middle of parsing a value.

Receives DataEvent, with ReadEventType of NEXT or SKIP to iterate over values, or DATA if the last event was a INCOMPLETE or STREAM_END event type.

SKIP is only allowed within a container. A reader is in a container when the CONTAINER_START event type is encountered and not in a container when the CONTAINER_END event type for that container is encountered.

amazon.ion.reader_managed module

Provides symbol table managed processing for Ion readers.

amazon.ion.reader_text module

amazon.ion.reader_text.reader(queue=None, is_unicode=False)

Returns a raw binary reader co-routine.

Args:

queue (Optional[BufferQueue]): The buffer read data for parsing, if None a

new one will be created.

is_unicode (Optional[bool]): True if all input data to this reader will be of unicode text type; False if all

input data to this reader will be of binary type.

Yields:

IonEvent: parse events, will have an event type of INCOMPLETE if data is needed

in the middle of a value or STREAM_END if there is no data and the parser is not in the middle of parsing a value.

Receives DataEvent, with ReadEventType of NEXT or SKIP to iterate over values; DATA or NEXT if the last event type was INCOMPLETE; or DATA if the last event type was STREAM_END.

When the reader receives NEXT after yielding INCOMPLETE, this signals to the reader that no further data is coming, and that any pending data should be flushed as either parse events or errors. This is only valid at the top-level, and will only result in a parse event if the last character encountered…

• was a digit or a decimal point in a non-timestamp, non-keyword numeric value; OR
• ended a valid partial timestamp; OR
• ended a keyword value (special floats, booleans, null, and typed nulls); OR
• was part of an unquoted symbol token, or whitespace or the end of a comment following an unquoted symbol token (as long as no colons were encountered after the token); OR
• was the closing quote of a quoted symbol token, or whitespace or the end of a comment following a quoted symbol token (as long as no colons were encountered after the token); OR
• was the final closing quote of a long string, or whitespace or the end of a comment following a long string.

If the reader successfully yields a parse event as a result of this, NEXT is the only input that may immediately follow. At that point, there are only two possible responses from the reader:

• If the last character read was the closing quote of an empty symbol following a long string, the reader will emit a parse event representing a symbol value with empty text. The next reader input/output event pair must be (NEXT, STREAM_END).
• Otherwise, the reader will emit STREAM_END.

After that STREAM_END, the user may later provide DATA to resume reading. If this occurs, the new data will be interpreted as if it were at the start of the stream (i.e. it can never continue the previous value), except that it occurs within the same symbol table context. This has the following implications (where <FLUSH> stands for the (INCOMPLETE, NEXT) transaction):

• If the previously-emitted value was a numeric value (int, float, decimal, timestamp), the new data will never extend that value, even if it would be a valid continuation. For example, 123<FLUSH>456 will always be emitted as two parse events (ints 123 and 456), even though it would have been interpreted as 123456 without the <FLUSH>.
• If the previously-emitted value was a symbol value or long string, the new data will be interpreted as the start of a new value. For example, abc<FLUSH>::123 will be emitted as the symbol value 'abc', followed by an error upon encountering ‘:’ at the start of a value, even though it would have been interpreted as the int 123 annotated with 'abc' without the <FLUSH>. The input abc<FLUSH>abc will be emitted as the symbol value 'abc' (represented by a SymbolToken), followed by another symbol value 'abc' (represented by a SymbolToken with the same symbol ID), even though it would have been interpreted as 'abcabc' without the <FLUSH>. Similarly, '''abc'''<FLUSH>'''def''' will the interpreted as two strings ('abc' and 'def'), even though it would have been interpreted as 'abcdef' without the <FLUSH>.

SKIP is only allowed within a container. A reader is in a container when the CONTAINER_START event type is encountered and not in a container when the CONTAINER_END event type for that container is encountered.

amazon.ion.reader_text.text_reader(queue=None, is_unicode=False)

Returns a raw binary reader co-routine.

Args:

queue (Optional[BufferQueue]): The buffer read data for parsing, if None a

new one will be created.

is_unicode (Optional[bool]): True if all input data to this reader will be of unicode text type; False if all

input data to this reader will be of binary type.

Yields:

IonEvent: parse events, will have an event type of INCOMPLETE if data is needed

in the middle of a value or STREAM_END if there is no data and the parser is not in the middle of parsing a value.

Receives DataEvent, with ReadEventType of NEXT or SKIP to iterate over values; DATA or NEXT if the last event type was INCOMPLETE; or DATA if the last event type was STREAM_END.

When the reader receives NEXT after yielding INCOMPLETE, this signals to the reader that no further data is coming, and that any pending data should be flushed as either parse events or errors. This is only valid at the top-level, and will only result in a parse event if the last character encountered…

• was a digit or a decimal point in a non-timestamp, non-keyword numeric value; OR
• ended a valid partial timestamp; OR
• ended a keyword value (special floats, booleans, null, and typed nulls); OR
• was part of an unquoted symbol token, or whitespace or the end of a comment following an unquoted symbol token (as long as no colons were encountered after the token); OR
• was the closing quote of a quoted symbol token, or whitespace or the end of a comment following a quoted symbol token (as long as no colons were encountered after the token); OR
• was the final closing quote of a long string, or whitespace or the end of a comment following a long string.

If the reader successfully yields a parse event as a result of this, NEXT is the only input that may immediately follow. At that point, there are only two possible responses from the reader:

• If the last character read was the closing quote of an empty symbol following a long string, the reader will emit a parse event representing a symbol value with empty text. The next reader input/output event pair must be (NEXT, STREAM_END).
• Otherwise, the reader will emit STREAM_END.

After that STREAM_END, the user may later provide DATA to resume reading. If this occurs, the new data will be interpreted as if it were at the start of the stream (i.e. it can never continue the previous value), except that it occurs within the same symbol table context. This has the following implications (where <FLUSH> stands for the (INCOMPLETE, NEXT) transaction):

• If the previously-emitted value was a numeric value (int, float, decimal, timestamp), the new data will never extend that value, even if it would be a valid continuation. For example, 123<FLUSH>456 will always be emitted as two parse events (ints 123 and 456), even though it would have been interpreted as 123456 without the <FLUSH>.
• If the previously-emitted value was a symbol value or long string, the new data will be interpreted as the start of a new value. For example, abc<FLUSH>::123 will be emitted as the symbol value 'abc', followed by an error upon encountering ‘:’ at the start of a value, even though it would have been interpreted as the int 123 annotated with 'abc' without the <FLUSH>. The input abc<FLUSH>abc will be emitted as the symbol value 'abc' (represented by a SymbolToken), followed by another symbol value 'abc' (represented by a SymbolToken with the same symbol ID), even though it would have been interpreted as 'abcabc' without the <FLUSH>. Similarly, '''abc'''<FLUSH>'''def''' will the interpreted as two strings ('abc' and 'def'), even though it would have been interpreted as 'abcdef' without the <FLUSH>.

SKIP is only allowed within a container. A reader is in a container when the CONTAINER_START event type is encountered and not in a container when the CONTAINER_END event type for that container is encountered.

amazon.ion.simple_types module

The type mappings for the simplejson-like API.

In particular, this module provides the extension to native Python data types with particulars of the Ion data model.

amazon.ion.simple_types.IonPyBool

alias of amazon.ion.simple_types.IonPyInt

class amazon.ion.simple_types.IonPyBytes(*args, **kwargs)

Bases: str, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyDecimal(*args, **kwargs)

Bases: decimal.Decimal, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyDict(*args, **kwargs)

Bases: amazon.ion.core.Multimap, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyFloat(*args, **kwargs)

Bases: float, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyInt(*args, **kwargs)

Bases: long, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyList(*args, **kwargs)

Bases: list, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyNull(*args, **kwargs)

Bases: amazon.ion.simple_types._IonNature

Representation of null.

Notes:

None is a singleton and cannot be sub-classed, so we have our

own value type for it. The function is_null is the best way to test for null-ness or None-ness.

class amazon.ion.simple_types.IonPySymbol(*args, **kwargs)

Bases: amazon.ion.symbols.SymbolToken, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyText(*args, **kwargs)

Bases: unicode, amazon.ion.simple_types._IonNature

class amazon.ion.simple_types.IonPyTimestamp(*args, **kwargs)

Bases: amazon.ion.core.Timestamp, amazon.ion.simple_types._IonNature

amazon.ion.simple_types.is_null(value)

A mechanism to determine if a value is None or an Ion null.

amazon.ion.simpleion module

Provides a simplejson-like API for dumping and loading Ion data.

amazon.ion.simpleion.dump(obj, fp, imports=None, binary=True, sequence_as_stream=False, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, encoding='utf-8', default=None, use_decimal=True, namedtuple_as_object=True, tuple_as_array=True, bigint_as_string=False, sort_keys=False, item_sort_key=None, for_json=None, ignore_nan=False, int_as_string_bitcount=None, iterable_as_array=False, tuple_as_sexp=False, omit_version_marker=False, **kw)

Serialize obj as an Ion-formatted stream to fp (a file-like object), using the following conversion table:

+-------------------+-------------------+
|  Python           |       Ion         |
|-------------------+-------------------|
| None              |    null.null      |
|-------------------+-------------------|
| IonPyNull(<type>) |    null.<type>    |
|-------------------+-------------------|
| True, False,      |                   |
| IonPyInt(BOOL),   |     bool          |
| IonPyBool,        |                   |
|-------------------+-------------------|
| int (Python 2, 3) |                   |
| long (Python 2),  |      int          |
| IonPyInt(INT)     |                   |
|-------------------+-------------------|
| float, IonPyFloat |     float         |
|-------------------+-------------------|
| Decimal,          |                   |
| IonPyDecimal      |     decimal       |
|-------------------+-------------------|
| datetime,         |                   |
| Timestamp,        |    timestamp      |
| IonPyTimestamp    |                   |
|-------------------+-------------------|
| SymbolToken,      |                   |
| IonPySymbol,      |     symbol        |
| IonPyText(SYMBOL) |                   |
|-------------------+-------------------|
| str (Python 3),   |                   |
| unicode (Python2),|     string        |
| IonPyText(STRING) |                   |
|-------------------+-------------------|
| IonPyBytes(CLOB)  |     clob          |
|-------------------+-------------------|
| str (Python 2),   |                   |
| bytes (Python 3)  |     blob          |
| IonPyBytes(BLOB)  |                   |
|-------------------+-------------------|
| list,             |                   |
| tuple (when       |                   |
|  tuple_as_sexp=   |     list          |
|  False)           |                   |
| IonPyList(LIST)   |                   |
|-------------------+-------------------|
| tuple (when       |                   |
|  tuple_as_sexp=   |     sexp          |
|  True),           |                   |
| IonPyList(SEXP)   |                   |
|-------------------+-------------------|
| dict, namedtuple, |                   |
| IonPyDict         |     struct        |
+-------------------+-------------------+

Args:

obj (Any): A python object to serialize according to the above table. Any Python object which is neither an

instance of nor inherits from one of the types in the above table will raise TypeError.

fp (BaseIO): A file-like object. imports (Optional[Sequence[SymbolTable]]): A sequence of shared symbol tables to be used by by the writer. binary (Optional[True|False]): When True, outputs binary Ion. When false, outputs text Ion. sequence_as_stream (Optional[True|False]): When True, if obj is a sequence, it will be treated as a stream

of top-level Ion values (i.e. the resulting Ion data will begin with obj’s first element). Default: False.

skipkeys: NOT IMPLEMENTED ensure_ascii: NOT IMPLEMENTED check_circular: NOT IMPLEMENTED allow_nan: NOT IMPLEMENTED cls: NOT IMPLEMENTED indent (Str): If binary is False and indent is a string, then members of containers will be pretty-printed with

a newline followed by that string repeated for each level of nesting. None (the default) selects the most compact representation without any newlines. Example: to indent with four spaces per level of nesting, use '    '.

separators: NOT IMPLEMENTED encoding: NOT IMPLEMENTED default: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED namedtuple_as_object: NOT IMPLEMENTED tuple_as_array: NOT IMPLEMENTED bigint_as_string: NOT IMPLEMENTED sort_keys: NOT IMPLEMENTED item_sort_key: NOT IMPLEMENTED for_json: NOT IMPLEMENTED ignore_nan: NOT IMPLEMENTED int_as_string_bitcount: NOT IMPLEMENTED iterable_as_array: NOT IMPLEMENTED tuple_as_sexp (Optional[True|False]): When True, all tuple values will be written as Ion s-expressions.

When False, all tuple values will be written as Ion lists. Default: False.

omit_version_marker (Optional|True|False): If binary is False and omit_version_marker is True, omits the

Ion Version Marker ($ion_1_0) from the output. Default: False.

**kw: NOT IMPLEMENTED

amazon.ion.simpleion.dumps(obj, imports=None, binary=True, sequence_as_stream=False, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, encoding='utf-8', default=None, use_decimal=True, namedtuple_as_object=True, tuple_as_array=True, bigint_as_string=False, sort_keys=False, item_sort_key=None, for_json=None, ignore_nan=False, int_as_string_bitcount=None, iterable_as_array=False, tuple_as_sexp=False, omit_version_marker=False, **kw)

Serialize obj as Python string or bytes object, using the conversion table used by dump (above).

Args:

obj (Any): A python object to serialize according to the above table. Any Python object which is neither an

instance of nor inherits from one of the types in the above table will raise TypeError.

imports (Optional[Sequence[SymbolTable]]): A sequence of shared symbol tables to be used by by the writer. binary (Optional[True|False]): When True, outputs binary Ion. When false, outputs text Ion. sequence_as_stream (Optional[True|False]): When True, if obj is a sequence, it will be treated as a stream

of top-level Ion values (i.e. the resulting Ion data will begin with obj’s first element). Default: False.

skipkeys: NOT IMPLEMENTED ensure_ascii: NOT IMPLEMENTED check_circular: NOT IMPLEMENTED allow_nan: NOT IMPLEMENTED cls: NOT IMPLEMENTED indent (Str): If binary is False and indent is a string, then members of containers will be pretty-printed with

a newline followed by that string repeated for each level of nesting. None (the default) selects the most compact representation without any newlines. Example: to indent with four spaces per level of nesting, use '    '.

separators: NOT IMPLEMENTED encoding: NOT IMPLEMENTED default: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED namedtuple_as_object: NOT IMPLEMENTED tuple_as_array: NOT IMPLEMENTED bigint_as_string: NOT IMPLEMENTED sort_keys: NOT IMPLEMENTED item_sort_key: NOT IMPLEMENTED for_json: NOT IMPLEMENTED ignore_nan: NOT IMPLEMENTED int_as_string_bitcount: NOT IMPLEMENTED iterable_as_array: NOT IMPLEMENTED tuple_as_sexp (Optional[True|False]): When True, all tuple values will be written as Ion s-expressions.

When False, all tuple values will be written as Ion lists. Default: False.

omit_version_marker (Optional|True|False): If binary is False and omit_version_marker is True, omits the

Ion Version Marker ($ion_1_0) from the output. Default: False.

**kw: NOT IMPLEMENTED

Returns:

Union[str|bytes]: The string or binary representation of the data. if binary=True, this will be a

bytes object, otherwise this will be a str object (or unicode in the case of Python 2.x)

amazon.ion.simpleion.load(fp, catalog=None, single_value=True, encoding='utf-8', cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, use_decimal=None, **kw)

Deserialize fp (a file-like object), which contains a text or binary Ion stream, to a Python object using the following conversion table:

+-------------------+-------------------+
|  Ion              |     Python        |
|-------------------+-------------------|
| null.<type>       | IonPyNull(<type>) |
|-------------------+-------------------|
| bool              |    IonPyBool      |
|-------------------+-------------------|
| int               |    IonPyInt       |
|-------------------+-------------------|
| float             |    IonPyFloat     |
|-------------------+-------------------|
| decimal           |   IonPyDecimal    |
|-------------------+-------------------|
| timestamp         |  IonPyTimestamp   |
|-------------------+-------------------|
| symbol            |   IonPySymbol     |
|-------------------+-------------------|
| string            | IonPyText(STRING) |
|-------------------+-------------------|
| clob              |  IonPyBytes(CLOB) |
|-------------------+-------------------|
| blob              |  IonPyBytes(BLOB) |
|-------------------+-------------------|
| list              |   IonPyList(LIST) |
|-------------------+-------------------|
| sexp              |   IonPyList(SEXP) |
|-------------------+-------------------|
| struct            |     IonPyDict     |
+-------------------+-------------------+

Args:

fp (BaseIO): A file-like object containing Ion data. catalog (Optional[SymbolTableCatalog]): The catalog to use for resolving symbol table imports. single_value (Optional[True|False]): When True, the data in obj is interpreted as a single Ion value, and

will be returned without an enclosing container. If True and there are multiple top-level values in the Ion stream, IonException will be raised. NOTE: this means that when data is dumped using sequence_as_stream=True, it must be loaded using single_value=False. Default: True.

encoding: NOT IMPLEMENTED cls: NOT IMPLEMENTED object_hook: NOT IMPLEMENTED parse_float: NOT IMPLEMENTED parse_int: NOT IMPLEMENTED parse_constant: NOT IMPLEMENTED object_pairs_hook: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED **kw: NOT IMPLEMENTED

Returns (Any):

if single_value is True:

A Python object representing a single Ion value.

else:

A sequence of Python objects representing a stream of Ion values.

amazon.ion.simpleion.loads(ion_str, catalog=None, single_value=True, encoding='utf-8', cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, use_decimal=None, **kw)

Deserialize ion_str, which is a string representation of an Ion object, to a Python object using the conversion table used by load (above).

Args:

fp (str): A string representation of Ion data. catalog (Optional[SymbolTableCatalog]): The catalog to use for resolving symbol table imports. single_value (Optional[True|False]): When True, the data in ion_str is interpreted as a single Ion value,

and will be returned without an enclosing container. If True and there are multiple top-level values in the Ion stream, IonException will be raised. NOTE: this means that when data is dumped using sequence_as_stream=True, it must be loaded using single_value=False. Default: True.

encoding: NOT IMPLEMENTED cls: NOT IMPLEMENTED object_hook: NOT IMPLEMENTED parse_float: NOT IMPLEMENTED parse_int: NOT IMPLEMENTED parse_constant: NOT IMPLEMENTED object_pairs_hook: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED **kw: NOT IMPLEMENTED

Returns (Any):

if single_value is True:

A Python object representing a single Ion value.

else:

A sequence of Python objects representing a stream of Ion values.

amazon.ion.symbols module

Provides support for Ion symbol tokens.

class amazon.ion.symbols.ImportLocation

Bases: amazon.ion.util.ImportLocation

Represents the import location of a symbol token.

An import location can be thought of as a position independent address of an imported symbol. This metadata, if defined, can indicate identity of a symbol that came from a shared symbol token that the application does not have access to.

Note:

Version is nor part of an import location because a position in a shared table by name uniquely identifies that slot irrespective of version.

Args:

name (unicode): The name of the shared symbol table. position (int): The position in the shared symbol table.

class amazon.ion.symbols.SymbolTable(table_type, symbols, name=None, version=None, imports=None, is_substitute=False)

Bases: object

A collection of symbols that is ordered.

Symbol tables are basically an Unicode string to integer interning table.

A few things to consider about symbol tables:

• System symbol tables never have imports and are shared symbol tables themselves.
• Shared symbol tables never import the system symbol table.
• Local symbol tables implicitly import the system symbol table.

Shared symbol tables have tokens that always have a location attribute referring to themselves. Local symbol tables have tokens whose location attribute refers to either the shared symbol that it was imported from or None if the symbol was defined locally.

Note:

Shared symbol tables (which include system symbol tables) are immutable. Local symbol tables support interning as a mutable operation. The implementation doesn’t enforce making properties read-only to enforce this invariant.

Args:

table_type (_SymbolTableType): The type of symbol table. symbols (Iterable[unicode]): The symbols text associated locally to this table. imports (Optional[Iterable[SymbolTable]]): The imports of the table. name (Optional[unicode]): The name of this table. Required for shared symbol tables. version (Optional[int]): The version of this table. Required for shared symbol tables. is_substitute (Optional[bool]): Whether or not this table is substituted. A substituted symbol

table is one that is not resolvable and has placeholder entries.

get(key, default=None)

Returns a token by text or local ID, with a default.

A given text image may be associated with more than one symbol ID. This will return the first definition.

Note:

User defined symbol IDs are always one-based. Symbol zero is a special symbol that always has no text.

Args:

key (unicode | int): The key to lookup. default(Optional[SymbolToken]): The default to return if the key is not found

Returns:

SymbolToken: The token associated with the key or the default if it doesn’t exist.

intern(text)

Interns the given Unicode sequence into the symbol table.

Note:

This operation is only valid on local symbol tables.

Args:

text (unicode): The target to intern.

Returns:

SymbolToken: The mapped symbol token which may already exist in the table.

class amazon.ion.symbols.SymbolTableCatalog

Bases: object

A collection of symbol tables that can be used to resolve imports.

Note:

The catalog will return a placeholder symbol table when resolving a table that doesn’t exist. For tables that don’t exist with any version, this placeholder will be completely devoid of text mappings. For tables that exist with a non-exact version, a derived substitute will be generated.

register(table)

Adds a shared table to the catalog.

Args:

table (SymbolTable): A non-system, shared symbol table.

resolve(name, version, max_id)

Resolves the table for a given name and version.

Args:

name (unicode): The name of the table to resolve. version (int): The version of the table to resolve. max_id (Optional[int]): The maximum ID of the table requested.

May be None in which case an exact match on name and version is required.

Returns:

SymbolTable: The closest matching symbol table. This is either an exact match, a placeholder, or a derived substitute depending on what tables are registered.

class amazon.ion.symbols.SymbolToken

Bases: amazon.ion.util.SymbolToken

Representation of a symbolic token.

A symbolic token may be a part of an Ion value in several contexts:

• The field name of a value in a struct
• An annotation of a value.
• A scalar of type symbol in the Ion data model.

Args:

text (Optional[unicode]): The text image of the token. sid (Optional[int]): The local symbol ID of the token. location (Optional[ImportTableLocation]): The import source of the token.

Note:

At least one of text or sid should be non-None

amazon.ion.symbols.local_symbol_table(imports=None, symbols=())

Constructs a local symbol table.

Args:

imports (Optional[SymbolTable]): Shared symbol tables to import. symbols (Optional[Iterable[Unicode]]): Initial local symbols to add.

Returns:

SymbolTable: A mutable local symbol table with the seeded local symbols.

amazon.ion.symbols.placeholder_symbol_table(name, version, max_id)

Constructs a shared symbol table that consists symbols that all have no known text.

This is generally used for cases where a shared symbol table is not available by the application.

Args:

name (unicode): The name of the shared symbol table. version (int): The version of the shared symbol table. max_id (int): The maximum ID allocated by this symbol table, must be >= 0

Returns:

SymbolTable: The synthesized table.

amazon.ion.symbols.shared_symbol_table(name, version, symbols, imports=None)

Constructs a shared symbol table.

Args:

name (unicode): The name of the shared symbol table. version (int): The version of the shared symbol table. symbols (Iterable[unicode]): The symbols to associate with the table. imports (Optional[Iterable[SymbolTable]): The shared symbol tables to inject into this one.

Returns:

SymbolTable: The constructed table.

amazon.ion.symbols.substitute_symbol_table(table, version, max_id)

Substitutes a given shared symbol table for another version.

• If the given table has more symbols than the requested substitute, then the generated symbol table will be a subset of the given table.
• If the given table has less symbols than the requested substitute, then the generated symbol table will have symbols with unknown text generated for the difference.

Args:

table (SymbolTable): The shared table to derive from. version (int): The version to target. max_id (int): The maximum ID allocated by the substitute, must be >= 0.

Returns:

SymbolTable: The synthesized table.

amazon.ion.util module

General purpose utilities.

class amazon.ion.util.CodePoint(*args, **kwargs)

Bases: int

Evaluates as the ordinal of a code point, while also containing the unicode character representation and indicating whether the code point was escaped.

class amazon.ion.util.Enum(name, value)

Bases: int

Simple integer based enumeration type.

Examples:

The typical declaration looks like:

class MyEnum(Enum):
    A = 1
    B = 2
    C = 3

At this point MyEnum.A is an instance of MyEnum.

Note:

Proper enumerations were added in Python 3.4 (PEP 435), this is a very simplified implementation based loosely on that specification.

In particular, implicit order of the values is not supported.

Args:

value (int): the value associated with the enumeration.

Attributes:

name (str): The name of the enum. value (int): The original value associated with the enum.

amazon.ion.util.bit_length(value)

Returns the bit length of an integer

amazon.ion.util.coroutine(func)

Wraps a PEP-342 enhanced generator in a way that avoids boilerplate of the “priming” call to next.

Args:

func (Callable): The function constructing a generator to decorate.

Returns:

Callable: The decorated generator.

amazon.ion.util.record(*fields)

Constructs a type that can be extended to create immutable, value types.

Examples:

A typical declaration looks like:

class MyRecord(record('a', ('b', 1))):
    pass

The above would make a sub-class of collections.namedtuple that was named MyRecord with a constructor that had the b field set to 1 by default.

Note:

This uses meta-class machinery to rewrite the inheritance hierarchy. This is done in order to make sure that the underlying namedtuple instance is bound to the right type name and to make sure that the synthetic class that is generated to enable this machinery is not enabled for sub-classes of a user’s record class.

Args:

fields (list[str | (str, any)]): A sequence of str or pairs that

amazon.ion.util.total_seconds(td)

Timedelta total_seconds with backported support in Python 2.6

amazon.ion.util.unicode_iter(val)

Provides an iterator over the code points of the given Unicode sequence.

Notes:

Before PEP-393, Python has the potential to support Unicode as UTF-16 or UTF-32. This is reified in the property as sys.maxunicode. As a result, naive iteration of Unicode sequences will render non-character code points such as UTF-16 surrogates.

Args:

val (unicode): The unicode sequence to iterate over as integer code points in the range

0x0 to 0x10FFFF.

amazon.ion.writer module

Provides common functionality for Ion binary and text writers.

class amazon.ion.writer.WriteEventType(name, value)

Bases: amazon.ion.util.Enum

Events that can come from an Ion writer co-routine.

Attributes:

HAS_PENDING: Indicates that the writer has more pending events to yield,

and that it should be sent None as the event to flush them out. Generally, this is signalled when an internal buffer has been filled by an input event and needs to be flushed in order to make progress.

NEEDS_INPUT: Indicates that the writer has no pending events to yield

and that it should be sent a amazon.ion.core.IonEvent.

COMPLETE: Indicates that the writer has flushed out complete Ion values at the top-level.

This is similar to NEEDS_INPUT except that it signifies point that all data emitted by the writer is in sync with the events given to it.

COMPLETE = <WriteEventType.COMPLETE: 3>

HAS_PENDING = <WriteEventType.HAS_PENDING: 1>

NEEDS_INPUT = <WriteEventType.NEEDS_INPUT: 2>

amazon.ion.writer.illegal_state_null(ion_event)

amazon.ion.writer.partial_transition(data, delegate)

Generates a Transition that has an event indicating HAS_PENDING.

amazon.ion.writer.serialize_scalar(ion_event, jump_table, null_table)

amazon.ion.writer.validate_scalar_value(value, expected_types)

amazon.ion.writer_binary module

Binary Ion writer with symbol table management.

amazon.ion.writer_binary.binary_writer(imports=None)

Returns a binary writer co-routine.

Keyword Args:

imports (Optional[Sequence[SymbolTable]]): A list of shared symbol tables

to be used by this writer.

Yields:

DataEvent: serialization events to write out

Receives amazon.ion.core.IonEvent.

amazon.ion.writer_binary_raw module

Writer for raw binary Ion values, without symbol table management.

amazon.ion.writer_binary_raw_fields module

Methods for writing binary Ion Int, VarInt, UInt, and VarUInt fields.

Whenever the following words are used in variable or method names, their meaning is as defined below.

• signed: applies only to Int and VarInt - the signed fields (sign bit in first octet).
• unsigned: applies only to UInt and VarUInt - the unsigned fields (no sign bit).
• variable: applies only to VarInt and VarUInt - the variable-length fields (end bit required).
• fixed: applies only to Int and UInt - the fixed-length fields (no end bit).
• Int: applies only to the Int field - the signed, fixed-length field.
• varint: applies only to the VarInt field - the signed, variable-length field.
• uint: applies only to the UInt field - the unsigned, fixed-length field.
• varuint: applies only to the VarUInt field - the unsigned, variable-length field.

amazon.ion.writer_buffer module

Buffer for binary Ion writers.

class amazon.ion.writer_buffer.BufferTree

A tree of buffers that can be depth-first traversed to produce a correctly-ordered Ion stream.

Containers are treated as subtrees. Calling start_container creates an empty node which represents the start of a new subtree, while calling end_container steps out of that subtree, allowing for additional nodes to be added as siblings of that subtree’s start node. It is important that there is exactly one call to start_container per call to end_container. Nodes representing the start of a container subtree are kept in a stack; upon calling end_container, the subtree represented by the node that was created by the most recent call to start_container will be stepped out. Other than that, no container semantics are asserted by this class.

Scalar values are treated as child nodes of the current container subtree (which may be at the top-level if the current container subtree’s root node is the root node of the BufferTree). Calling add_scalar_value will add a new node as a child of the current container subtree. A container subtree’s children are ordered.

If None values are passed in to end_container or add_scalar_value, nodes will be added to the tree, in the same way as described above, but no value will be yielded for that node upon drain.

add_scalar_value(value_buf)

Add a node to the tree containing a scalar value.

Args:

value_buf (bytearray): bytearray containing the scalar value.

drain()

Walk the BufferTree and reset it when finished.

Yields:

any: The current node’s value.

end_container(header_buf)

Add a node containing the container’s header to the current subtree.

This node will be added as the leftmost leaf of the subtree that was started by the matching call to start_container.

Args:

header_buf (bytearray): bytearray containing the container header.

start_container()

Add a node to the tree that represents the start of a container.

Until end_container is called, any nodes added through add_scalar_value or start_container will be children of this new node.

amazon.ion.writer_text module

Implementations of Ion Text writers.

amazon.ion.writer_text.raw_writer(indent=None)

Returns a raw text writer co-routine.

Yields:

DataEvent: serialization events to write out

Receives amazon.ion.core.IonEvent or None when the co-routine yields HAS_PENDING WriteEventType events.

amazon.ion.writer_text.text_writer(indent=None)

Returns a raw text writer co-routine.

Yields:

DataEvent: serialization events to write out

Receives amazon.ion.core.IonEvent or None when the co-routine yields HAS_PENDING WriteEventType events.

Module contents