amazon.ion package¶
Submodules¶
amazon.ion.core module¶
Ion core types.
-
class
amazon.ion.core.
DataEvent
[source]¶ Bases:
amazon.ion.util.DataEvent
Event generated as a result of the writer or as input into the reader.
- Args:
type (Enum): The type of event. data (bytes): The serialized data returned. If no data is to be serialized,
this should be the empty byte string.
-
class
amazon.ion.core.
IonEvent
[source]¶ Bases:
amazon.ion.util.IonEvent
An parse or serialization event.
- Args:
event_type (IonEventType): The type of event. ion_type (Optional(amazon.ion.core.IonType)): The Ion data model type
associated with the event.value (Optional[any]): The data value associated with the event. field_name (Optional[Union[amazon.ion.symbols.SymbolToken, unicode]]): The field name
associated with the event.- annotations (Sequence[Union[amazon.ion.symbols.SymbolToken, unicode]]): The annotations
- associated with the event.
depth (Optional[int]): The tree depth of the event if applicable.
-
derive_annotations
(annotations)[source]¶ Derives a new event from this one setting the
annotations
attribute.- Args:
- annotations: (Sequence[Union[amazon.ion.symbols.SymbolToken, unicode]]):
- The annotations associated with the derived event.
- Returns:
- IonEvent: The newly generated event.
-
derive_depth
(depth)[source]¶ Derives a new event from this one setting the
depth
attribute.- Args:
- depth: (int):
- The annotations associated with the derived event.
- Returns:
- IonEvent: The newly generated event.
-
class
amazon.ion.core.
IonEventType
(name, value)[source]¶ Bases:
amazon.ion.util.Enum
Enumeration of Ion parser or serializer events.
These types do not correspond directly to the Ion type system, but they are related. In particular,
null.*
will surface as aSCALAR
even though they are containers.- Attributes:
- INCOMPLETE: Indicates that parsing cannot be completed due to lack of input. STREAM_END: Indicates that the logical stream has terminated. VERSION_MARKER: Indicates that the Ion Version Marker has been encountered. SCALAR: Indicates an atomic value has been encountered. CONTAINER_START: Indicates that the start of a container has been reached. CONTAINER_END: Indicates that the end of a container has been reached.
-
CONTAINER_END
= <IonEventType.CONTAINER_END: 3>¶
-
CONTAINER_START
= <IonEventType.CONTAINER_START: 2>¶
-
INCOMPLETE
= <IonEventType.INCOMPLETE: -2>¶
-
SCALAR
= <IonEventType.SCALAR: 1>¶
-
STREAM_END
= <IonEventType.STREAM_END: -1>¶
-
VERSION_MARKER
= <IonEventType.VERSION_MARKER: 0>¶
-
begins_value
¶ Indicates if the event type is a start of a value.
-
ends_container
¶ Indicates if the event type terminates a container or stream.
-
is_stream_signal
¶ Indicates that the event type corresponds to a stream signal.
-
class
amazon.ion.core.
IonThunkEvent
[source]¶ Bases:
amazon.ion.core.IonEvent
An
IonEvent
whosevalue
field is a thunk.-
value
¶
-
-
class
amazon.ion.core.
IonType
(name, value)[source]¶ Bases:
amazon.ion.util.Enum
Enumeration of the Ion data types.
-
BLOB
= <IonType.BLOB: 9>¶
-
BOOL
= <IonType.BOOL: 1>¶
-
CLOB
= <IonType.CLOB: 8>¶
-
DECIMAL
= <IonType.DECIMAL: 4>¶
-
FLOAT
= <IonType.FLOAT: 3>¶
-
INT
= <IonType.INT: 2>¶
-
LIST
= <IonType.LIST: 10>¶
-
NULL
= <IonType.NULL: 0>¶
-
SEXP
= <IonType.SEXP: 11>¶
-
STRING
= <IonType.STRING: 7>¶
-
STRUCT
= <IonType.STRUCT: 12>¶
-
SYMBOL
= <IonType.SYMBOL: 6>¶
-
TIMESTAMP
= <IonType.TIMESTAMP: 5>¶
-
is_container
¶ Returns whether the type is a container.
-
is_lob
¶ Returns whether the type is a LOB.
-
is_numeric
¶
-
is_text
¶ Returns whether the type is a Unicode textual type.
-
-
class
amazon.ion.core.
MemoizingThunk
(delegate)[source]¶ Bases:
object
A
callable
that invokes adelegate
and caches and returns the result.
-
class
amazon.ion.core.
Multimap
(*args, **kwargs)[source]¶ Bases:
_abcoll.MutableMapping
Dictionary that can hold multiple values for the same key
In order not to break existing customers, getting and inserting elements with
[]
keeps the same behaviour as the built-in dict. If multiple elements are already mapped to the key, ``[]` will return the newest one.To map multiple elements to a key, use the
add_item
operation. To retrieve all the values map to a key, useget_all_values
.
-
class
amazon.ion.core.
OffsetTZInfo
(delta=datetime.timedelta(0))[source]¶ Bases:
datetime.tzinfo
A trivial UTC offset
tzinfo
.
-
class
amazon.ion.core.
Timestamp
[source]¶ Bases:
datetime.datetime
Sub-class of
datetime
that supports a precision field; afractional_precision
field that specifies the precision of the``microseconds`` field indatetime
; and afractional_seconds
field that is aDecimal
specifying the fractional seconds precisely.- Notes:
- The
precision
field is passed as a keyword argument of the same name. - The
fractional_precision
field is passed as a keyword argument of the same name. This field only relates to to themicroseconds
field and can be thought of as the number of decimal digits that are significant. This is an integer that that is in the closed interval[0, 6]
. If0
,microseconds
must be0
indicating no precision below seconds. This argument is optional and only valid whenmicroseconds
is notNone
. If themicroseconds
specified has more precision than this field indicates, then that is an error. - The
fractional_seconds
field is passed as a keyword argument of the same name. It must be aDecimal
in the left-closed, right-opened interval of[0, 1)
. If specified as an argument,microseconds
must beNone
andfractional_precision
must not be specified (but can beNone
). In addition, ifmicroseconds
is specified this argument must not be specified (but can beNone
). If the specified value hascoefficient==0
andexponent >= 0
, e.g.Decimal(0)
, then there is no precision beyond seconds. - After construction,
microseconds
,fractional_precision
, andfractional_seconds
will all be present and normalized in the resultingTimestamp
instance. If the precision offractional_seconds
is more than is capable of being expressed inmicroseconds
, then themicroseconds
field is truncated to six digits andfractional_precision
is6
.
Consider some examples:
- 2019-10-01T12:45:01Z would have the following fields set:
*
microseconds == 0
,fractional_precision == 0
,fractional_seconds == Decimal('0')
- 2019-10-01T12:45:01.100Z would have the following fields set:
*
microseconds == 100000
,fractional_precision == 3
,fractional_seconds == Decimal('0.100')
- 2019-10-01T12:45:01.123456789Z would have the following fields set:
*
microseconds == 123456
,fractional_precision == 6
,fractional_seconds == Decimal('0.123456789')
- The
- Raises:
- ValueError: If any of the preconditions above are violated.
-
static
adjust_from_utc_fields
(*args, **kwargs)[source]¶ Constructs a timestamp from UTC fields adjusted to the local offset if given.
-
fractional_precision
¶
-
fractional_seconds
¶
-
precision
¶
-
class
amazon.ion.core.
TimestampPrecision
(name, value)[source]¶ Bases:
amazon.ion.util.Enum
The different levels of precision supported in an Ion timestamp.
-
DAY
= <TimestampPrecision.DAY: 2>¶
-
MINUTE
= <TimestampPrecision.MINUTE: 3>¶
-
MONTH
= <TimestampPrecision.MONTH: 1>¶
-
SECOND
= <TimestampPrecision.SECOND: 4>¶
-
YEAR
= <TimestampPrecision.YEAR: 0>¶
-
includes_day
¶ Precision has at least the
day
field.
-
includes_minute
¶ Precision has at least the
minute
field.
-
includes_month
¶ Precision has at least the
month
field.
-
includes_second
¶ Precision has at least the
second
field.
-
-
class
amazon.ion.core.
Transition
[source]¶ Bases:
amazon.ion.util.Transition
A pair of event and co-routine delegate.
This is generally used as a result of a state-machine.
- Args:
event (Union[DataEvent]): The event associated with the transition. delegate (Coroutine): The co-routine delegate which can be the same routine from
whence this transition came.
-
amazon.ion.core.
timestamp
(year, month=1, day=1, hour=0, minute=0, second=0, microsecond=None, off_hours=None, off_minutes=None, precision=None, fractional_precision=None, fractional_seconds=None)[source]¶ Shorthand for the
Timestamp
constructor.Specifically, converts
off_hours
andoff_minutes
parameters to a suitableOffsetTZInfo
instance.
amazon.ion.equivalence module¶
Provides utilities for determining whether two objects are equivalent under the Ion data model.
-
amazon.ion.equivalence.
ion_equals
(a, b, timestamps_instants_only=False)[source]¶ Tests two objects for equivalence under the Ion data model.
- There are three important cases:
- When neither operand specifies its ion_type or annotations, this method will only return True when the values of both operands are equivalent under the Ion data model.
- When only one of the operands specifies its ion_type and annotations, this method will only return True when that operand has no annotations and has a value equivalent to the other operand under the Ion data model.
- When both operands specify ion_type and annotations, this method will only return True when the ion_type and annotations of both are the same and their values are equivalent under the Ion data model.
Note that the order of the operands does not matter.
- Args:
a (object): The first operand. b (object): The second operand. timestamps_instants_only (Optional[bool]): False if timestamp objects (datetime and its subclasses) should be
compared according to the Ion data model (where the instant, precision, and offset must be equal); True if these objects should be considered equivalent if they simply represent the same instant.
amazon.ion.exceptions module¶
Exceptions for Ion Python.
-
exception
amazon.ion.exceptions.
CannotSubstituteTable
[source]¶ Bases:
amazon.ion.exceptions.IonException
Indicates a problem with resolving symbols tables.
amazon.ion.reader module¶
Provides common functionality for Ion binary and text readers.
-
class
amazon.ion.reader.
BufferQueue
(is_unicode=False)[source]¶ Bases:
object
A simple circular buffer of buffers.
-
unread
(c)[source]¶ Unread the given character, byte, or code point.
If this is a unicode buffer and the input is an int or byte, it will be interpreted as an ordinal representing a unicode code point.
If this is a binary buffer, the input must be a byte or int; a unicode character will raise an error.
-
-
class
amazon.ion.reader.
CodePoint
(*args, **kwargs)[source]¶ Bases:
int
Evaluates as a code point ordinal, while also containing the unicode character representation and indicating whether the code point was escaped.
-
class
amazon.ion.reader.
CodePointArray
(initial_bytes=None)[source]¶ A mutable sequence of code points. Used in place of bytearray() for text values.
-
class
amazon.ion.reader.
ReadEventType
(name, value)[source]¶ Bases:
amazon.ion.util.Enum
Events that are pushed into an Ion reader co-routine.
- Attributes:
DATA: Indicates more data for the reader. The expected type is
bytes
. NEXT: Indicates that the reader should yield the next event. SKIP: Indicates that the reader should proceed to the end of the current container.This type is not meaningful at the top-level.
-
DATA
= <ReadEventType.DATA: 0>¶
-
NEXT
= <ReadEventType.NEXT: 1>¶
-
SKIP
= <ReadEventType.SKIP: 2>¶
-
amazon.ion.reader.
read_data_event
(data)[source]¶ Simple wrapper over the
DataEvent
constructor to wrap abytes
like with theDATA
ReadEventType
.- Args:
- data (bytes|unicode): The data for the event. Bytes are accepted by both binary and text readers, while unicode
- is accepted by text readers with is_unicode=True.
amazon.ion.reader_binary module¶
-
amazon.ion.reader_binary.
binary_reader
(queue=None)¶ Returns a raw binary reader co-routine.
- Args:
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
None
a - new one will be created.
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
- Yields:
- IonEvent: parse events, will have an event type of
INCOMPLETE
if data is needed in the middle of a value or
STREAM_END
if there is no data and the parser is not in the middle of parsing a value.Receives
DataEvent
, withReadEventType
ofNEXT
orSKIP
to iterate over values, orDATA
if the last event was aINCOMPLETE
orSTREAM_END
event type.SKIP
is only allowed within a container. A reader is in a container when theCONTAINER_START
event type is encountered and not in a container when theCONTAINER_END
event type for that container is encountered.
- IonEvent: parse events, will have an event type of
-
amazon.ion.reader_binary.
raw_reader
(queue=None)[source]¶ Returns a raw binary reader co-routine.
- Args:
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
None
a - new one will be created.
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
- Yields:
- IonEvent: parse events, will have an event type of
INCOMPLETE
if data is needed in the middle of a value or
STREAM_END
if there is no data and the parser is not in the middle of parsing a value.Receives
DataEvent
, withReadEventType
ofNEXT
orSKIP
to iterate over values, orDATA
if the last event was aINCOMPLETE
orSTREAM_END
event type.SKIP
is only allowed within a container. A reader is in a container when theCONTAINER_START
event type is encountered and not in a container when theCONTAINER_END
event type for that container is encountered.
- IonEvent: parse events, will have an event type of
amazon.ion.reader_managed module¶
Provides symbol table managed processing for Ion readers.
amazon.ion.reader_text module¶
-
amazon.ion.reader_text.
reader
(queue=None, is_unicode=False)[source]¶ Returns a raw binary reader co-routine.
- Args:
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
None
a - new one will be created.
- is_unicode (Optional[bool]): True if all input data to this reader will be of unicode text type; False if all
- input data to this reader will be of binary type.
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
- Yields:
- IonEvent: parse events, will have an event type of
INCOMPLETE
if data is needed in the middle of a value or
STREAM_END
if there is no data and the parser is not in the middle of parsing a value.Receives
DataEvent
, withReadEventType
ofNEXT
orSKIP
to iterate over values;DATA
orNEXT
if the last event type wasINCOMPLETE
; orDATA
if the last event type wasSTREAM_END
.When the reader receives
NEXT
after yieldingINCOMPLETE
, this signals to the reader that no further data is coming, and that any pending data should be flushed as either parse events or errors. This is only valid at the top-level, and will only result in a parse event if the last character encountered…- was a digit or a decimal point in a non-timestamp, non-keyword numeric value; OR
- ended a valid partial timestamp; OR
- ended a keyword value (special floats, booleans,
null
, and typed nulls); OR - was part of an unquoted symbol token, or whitespace or the end of a comment following an unquoted symbol token (as long as no colons were encountered after the token); OR
- was the closing quote of a quoted symbol token, or whitespace or the end of a comment following a quoted symbol token (as long as no colons were encountered after the token); OR
- was the final closing quote of a long string, or whitespace or the end of a comment following a long string.
If the reader successfully yields a parse event as a result of this,
NEXT
is the only input that may immediately follow. At that point, there are only two possible responses from the reader:- If the last character read was the closing quote of an empty symbol following a long
string, the reader will emit a parse event representing a symbol value with empty text.
The next reader input/output event pair must be (
NEXT
,STREAM_END
). - Otherwise, the reader will emit
STREAM_END
.
After that
STREAM_END
, the user may later provideDATA
to resume reading. If this occurs, the new data will be interpreted as if it were at the start of the stream (i.e. it can never continue the previous value), except that it occurs within the same symbol table context. This has the following implications (where<FLUSH>
stands for the (INCOMPLETE
,NEXT
) transaction):- If the previously-emitted value was a numeric value (
int
,float
,decimal
,timestamp
), the new data will never extend that value, even if it would be a valid continuation. For example,123<FLUSH>456
will always be emitted as two parse events (ints123
and456
), even though it would have been interpreted as123456
without the<FLUSH>
. - If the previously-emitted value was a symbol value or long string, the new data will
be interpreted as the start of a new value. For example,
abc<FLUSH>::123
will be emitted as the symbol value'abc'
, followed by an error upon encountering ‘:’ at the start of a value, even though it would have been interpreted as theint
123
annotated with'abc'
without the<FLUSH>
. The inputabc<FLUSH>abc
will be emitted as the symbol value'abc'
(represented by aSymbolToken
), followed by another symbol value'abc'
(represented by aSymbolToken
with the same symbol ID), even though it would have been interpreted as'abcabc'
without the<FLUSH>
. Similarly,'''abc'''<FLUSH>'''def'''
will the interpreted as two strings ('abc'
and'def'
), even though it would have been interpreted as'abcdef'
without the<FLUSH>
.
SKIP
is only allowed within a container. A reader is in a container when theCONTAINER_START
event type is encountered and not in a container when theCONTAINER_END
event type for that container is encountered.
- IonEvent: parse events, will have an event type of
-
amazon.ion.reader_text.
text_reader
(queue=None, is_unicode=False)¶ Returns a raw binary reader co-routine.
- Args:
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
None
a - new one will be created.
- is_unicode (Optional[bool]): True if all input data to this reader will be of unicode text type; False if all
- input data to this reader will be of binary type.
- queue (Optional[BufferQueue]): The buffer read data for parsing, if
- Yields:
- IonEvent: parse events, will have an event type of
INCOMPLETE
if data is needed in the middle of a value or
STREAM_END
if there is no data and the parser is not in the middle of parsing a value.Receives
DataEvent
, withReadEventType
ofNEXT
orSKIP
to iterate over values;DATA
orNEXT
if the last event type wasINCOMPLETE
; orDATA
if the last event type wasSTREAM_END
.When the reader receives
NEXT
after yieldingINCOMPLETE
, this signals to the reader that no further data is coming, and that any pending data should be flushed as either parse events or errors. This is only valid at the top-level, and will only result in a parse event if the last character encountered…- was a digit or a decimal point in a non-timestamp, non-keyword numeric value; OR
- ended a valid partial timestamp; OR
- ended a keyword value (special floats, booleans,
null
, and typed nulls); OR - was part of an unquoted symbol token, or whitespace or the end of a comment following an unquoted symbol token (as long as no colons were encountered after the token); OR
- was the closing quote of a quoted symbol token, or whitespace or the end of a comment following a quoted symbol token (as long as no colons were encountered after the token); OR
- was the final closing quote of a long string, or whitespace or the end of a comment following a long string.
If the reader successfully yields a parse event as a result of this,
NEXT
is the only input that may immediately follow. At that point, there are only two possible responses from the reader:- If the last character read was the closing quote of an empty symbol following a long
string, the reader will emit a parse event representing a symbol value with empty text.
The next reader input/output event pair must be (
NEXT
,STREAM_END
). - Otherwise, the reader will emit
STREAM_END
.
After that
STREAM_END
, the user may later provideDATA
to resume reading. If this occurs, the new data will be interpreted as if it were at the start of the stream (i.e. it can never continue the previous value), except that it occurs within the same symbol table context. This has the following implications (where<FLUSH>
stands for the (INCOMPLETE
,NEXT
) transaction):- If the previously-emitted value was a numeric value (
int
,float
,decimal
,timestamp
), the new data will never extend that value, even if it would be a valid continuation. For example,123<FLUSH>456
will always be emitted as two parse events (ints123
and456
), even though it would have been interpreted as123456
without the<FLUSH>
. - If the previously-emitted value was a symbol value or long string, the new data will
be interpreted as the start of a new value. For example,
abc<FLUSH>::123
will be emitted as the symbol value'abc'
, followed by an error upon encountering ‘:’ at the start of a value, even though it would have been interpreted as theint
123
annotated with'abc'
without the<FLUSH>
. The inputabc<FLUSH>abc
will be emitted as the symbol value'abc'
(represented by aSymbolToken
), followed by another symbol value'abc'
(represented by aSymbolToken
with the same symbol ID), even though it would have been interpreted as'abcabc'
without the<FLUSH>
. Similarly,'''abc'''<FLUSH>'''def'''
will the interpreted as two strings ('abc'
and'def'
), even though it would have been interpreted as'abcdef'
without the<FLUSH>
.
SKIP
is only allowed within a container. A reader is in a container when theCONTAINER_START
event type is encountered and not in a container when theCONTAINER_END
event type for that container is encountered.
- IonEvent: parse events, will have an event type of
amazon.ion.simple_types module¶
The type mappings for the simplejson
-like API.
In particular, this module provides the extension to native Python data types with particulars of the Ion data model.
-
amazon.ion.simple_types.
IonPyBool
¶ alias of
amazon.ion.simple_types.IonPyInt
-
class
amazon.ion.simple_types.
IonPyBytes
(*args, **kwargs)¶ Bases:
str
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyDecimal
(*args, **kwargs)¶ Bases:
decimal.Decimal
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyDict
(*args, **kwargs)¶ Bases:
amazon.ion.core.Multimap
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyFloat
(*args, **kwargs)¶ Bases:
float
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyInt
(*args, **kwargs)¶ Bases:
long
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyList
(*args, **kwargs)¶ Bases:
list
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyNull
(*args, **kwargs)[source]¶ Bases:
amazon.ion.simple_types._IonNature
Representation of
null
.- Notes:
None
is a singleton and cannot be sub-classed, so we have our- own value type for it. The function
is_null
is the best way to test fornull
-ness orNone
-ness.
-
class
amazon.ion.simple_types.
IonPySymbol
(*args, **kwargs)[source]¶ Bases:
amazon.ion.symbols.SymbolToken
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyText
(*args, **kwargs)¶ Bases:
unicode
,amazon.ion.simple_types._IonNature
-
class
amazon.ion.simple_types.
IonPyTimestamp
(*args, **kwargs)[source]¶ Bases:
amazon.ion.core.Timestamp
,amazon.ion.simple_types._IonNature
amazon.ion.simpleion module¶
Provides a simplejson
-like API for dumping and loading Ion data.
-
amazon.ion.simpleion.
dump
(obj, fp, imports=None, binary=True, sequence_as_stream=False, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, encoding='utf-8', default=None, use_decimal=True, namedtuple_as_object=True, tuple_as_array=True, bigint_as_string=False, sort_keys=False, item_sort_key=None, for_json=None, ignore_nan=False, int_as_string_bitcount=None, iterable_as_array=False, tuple_as_sexp=False, omit_version_marker=False, **kw)[source]¶ Serialize
obj
as an Ion-formatted stream tofp
(a file-like object), using the following conversion table:+-------------------+-------------------+ | Python | Ion | |-------------------+-------------------| | None | null.null | |-------------------+-------------------| | IonPyNull(<type>) | null.<type> | |-------------------+-------------------| | True, False, | | | IonPyInt(BOOL), | bool | | IonPyBool, | | |-------------------+-------------------| | int (Python 2, 3) | | | long (Python 2), | int | | IonPyInt(INT) | | |-------------------+-------------------| | float, IonPyFloat | float | |-------------------+-------------------| | Decimal, | | | IonPyDecimal | decimal | |-------------------+-------------------| | datetime, | | | Timestamp, | timestamp | | IonPyTimestamp | | |-------------------+-------------------| | SymbolToken, | | | IonPySymbol, | symbol | | IonPyText(SYMBOL) | | |-------------------+-------------------| | str (Python 3), | | | unicode (Python2),| string | | IonPyText(STRING) | | |-------------------+-------------------| | IonPyBytes(CLOB) | clob | |-------------------+-------------------| | str (Python 2), | | | bytes (Python 3) | blob | | IonPyBytes(BLOB) | | |-------------------+-------------------| | list, | | | tuple (when | | | tuple_as_sexp= | list | | False) | | | IonPyList(LIST) | | |-------------------+-------------------| | tuple (when | | | tuple_as_sexp= | sexp | | True), | | | IonPyList(SEXP) | | |-------------------+-------------------| | dict, namedtuple, | | | IonPyDict | struct | +-------------------+-------------------+
- Args:
- obj (Any): A python object to serialize according to the above table. Any Python object which is neither an
- instance of nor inherits from one of the types in the above table will raise TypeError.
fp (BaseIO): A file-like object. imports (Optional[Sequence[SymbolTable]]): A sequence of shared symbol tables to be used by by the writer. binary (Optional[True|False]): When True, outputs binary Ion. When false, outputs text Ion. sequence_as_stream (Optional[True|False]): When True, if
obj
is a sequence, it will be treated as a streamof top-level Ion values (i.e. the resulting Ion data will begin withobj
’s first element). Default: False.skipkeys: NOT IMPLEMENTED ensure_ascii: NOT IMPLEMENTED check_circular: NOT IMPLEMENTED allow_nan: NOT IMPLEMENTED cls: NOT IMPLEMENTED indent (Str): If binary is False and indent is a string, then members of containers will be pretty-printed with
a newline followed by that string repeated for each level of nesting. None (the default) selects the most compact representation without any newlines. Example: to indent with four spaces per level of nesting, use' '
.separators: NOT IMPLEMENTED encoding: NOT IMPLEMENTED default: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED namedtuple_as_object: NOT IMPLEMENTED tuple_as_array: NOT IMPLEMENTED bigint_as_string: NOT IMPLEMENTED sort_keys: NOT IMPLEMENTED item_sort_key: NOT IMPLEMENTED for_json: NOT IMPLEMENTED ignore_nan: NOT IMPLEMENTED int_as_string_bitcount: NOT IMPLEMENTED iterable_as_array: NOT IMPLEMENTED tuple_as_sexp (Optional[True|False]): When True, all tuple values will be written as Ion s-expressions.
When False, all tuple values will be written as Ion lists. Default: False.- omit_version_marker (Optional|True|False): If binary is False and omit_version_marker is True, omits the
- Ion Version Marker ($ion_1_0) from the output. Default: False.
**kw: NOT IMPLEMENTED
-
amazon.ion.simpleion.
dumps
(obj, imports=None, binary=True, sequence_as_stream=False, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, encoding='utf-8', default=None, use_decimal=True, namedtuple_as_object=True, tuple_as_array=True, bigint_as_string=False, sort_keys=False, item_sort_key=None, for_json=None, ignore_nan=False, int_as_string_bitcount=None, iterable_as_array=False, tuple_as_sexp=False, omit_version_marker=False, **kw)[source]¶ Serialize
obj
as Pythonstring
orbytes
object, using the conversion table used bydump
(above).- Args:
- obj (Any): A python object to serialize according to the above table. Any Python object which is neither an
- instance of nor inherits from one of the types in the above table will raise TypeError.
imports (Optional[Sequence[SymbolTable]]): A sequence of shared symbol tables to be used by by the writer. binary (Optional[True|False]): When True, outputs binary Ion. When false, outputs text Ion. sequence_as_stream (Optional[True|False]): When True, if
obj
is a sequence, it will be treated as a streamof top-level Ion values (i.e. the resulting Ion data will begin withobj
’s first element). Default: False.skipkeys: NOT IMPLEMENTED ensure_ascii: NOT IMPLEMENTED check_circular: NOT IMPLEMENTED allow_nan: NOT IMPLEMENTED cls: NOT IMPLEMENTED indent (Str): If binary is False and indent is a string, then members of containers will be pretty-printed with
a newline followed by that string repeated for each level of nesting. None (the default) selects the most compact representation without any newlines. Example: to indent with four spaces per level of nesting, use' '
.separators: NOT IMPLEMENTED encoding: NOT IMPLEMENTED default: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED namedtuple_as_object: NOT IMPLEMENTED tuple_as_array: NOT IMPLEMENTED bigint_as_string: NOT IMPLEMENTED sort_keys: NOT IMPLEMENTED item_sort_key: NOT IMPLEMENTED for_json: NOT IMPLEMENTED ignore_nan: NOT IMPLEMENTED int_as_string_bitcount: NOT IMPLEMENTED iterable_as_array: NOT IMPLEMENTED tuple_as_sexp (Optional[True|False]): When True, all tuple values will be written as Ion s-expressions.
When False, all tuple values will be written as Ion lists. Default: False.- omit_version_marker (Optional|True|False): If binary is False and omit_version_marker is True, omits the
- Ion Version Marker ($ion_1_0) from the output. Default: False.
**kw: NOT IMPLEMENTED
- Returns:
- Union[str|bytes]: The string or binary representation of the data. if
binary=True
, this will be a bytes
object, otherwise this will be astr
object (orunicode
in the case of Python 2.x)
- Union[str|bytes]: The string or binary representation of the data. if
-
amazon.ion.simpleion.
load
(fp, catalog=None, single_value=True, encoding='utf-8', cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, use_decimal=None, **kw)[source]¶ Deserialize
fp
(a file-like object), which contains a text or binary Ion stream, to a Python object using the following conversion table:+-------------------+-------------------+ | Ion | Python | |-------------------+-------------------| | null.<type> | IonPyNull(<type>) | |-------------------+-------------------| | bool | IonPyBool | |-------------------+-------------------| | int | IonPyInt | |-------------------+-------------------| | float | IonPyFloat | |-------------------+-------------------| | decimal | IonPyDecimal | |-------------------+-------------------| | timestamp | IonPyTimestamp | |-------------------+-------------------| | symbol | IonPySymbol | |-------------------+-------------------| | string | IonPyText(STRING) | |-------------------+-------------------| | clob | IonPyBytes(CLOB) | |-------------------+-------------------| | blob | IonPyBytes(BLOB) | |-------------------+-------------------| | list | IonPyList(LIST) | |-------------------+-------------------| | sexp | IonPyList(SEXP) | |-------------------+-------------------| | struct | IonPyDict | +-------------------+-------------------+
- Args:
fp (BaseIO): A file-like object containing Ion data. catalog (Optional[SymbolTableCatalog]): The catalog to use for resolving symbol table imports. single_value (Optional[True|False]): When True, the data in
obj
is interpreted as a single Ion value, andwill be returned without an enclosing container. If True and there are multiple top-level values in the Ion stream, IonException will be raised. NOTE: this means that when data is dumped usingsequence_as_stream=True
, it must be loaded usingsingle_value=False
. Default: True.encoding: NOT IMPLEMENTED cls: NOT IMPLEMENTED object_hook: NOT IMPLEMENTED parse_float: NOT IMPLEMENTED parse_int: NOT IMPLEMENTED parse_constant: NOT IMPLEMENTED object_pairs_hook: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED **kw: NOT IMPLEMENTED
- Returns (Any):
- if single_value is True:
- A Python object representing a single Ion value.
- else:
- A sequence of Python objects representing a stream of Ion values.
-
amazon.ion.simpleion.
loads
(ion_str, catalog=None, single_value=True, encoding='utf-8', cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, use_decimal=None, **kw)[source]¶ Deserialize
ion_str
, which is a string representation of an Ion object, to a Python object using the conversion table used by load (above).- Args:
fp (str): A string representation of Ion data. catalog (Optional[SymbolTableCatalog]): The catalog to use for resolving symbol table imports. single_value (Optional[True|False]): When True, the data in
ion_str
is interpreted as a single Ion value,and will be returned without an enclosing container. If True and there are multiple top-level values in the Ion stream, IonException will be raised. NOTE: this means that when data is dumped usingsequence_as_stream=True
, it must be loaded usingsingle_value=False
. Default: True.encoding: NOT IMPLEMENTED cls: NOT IMPLEMENTED object_hook: NOT IMPLEMENTED parse_float: NOT IMPLEMENTED parse_int: NOT IMPLEMENTED parse_constant: NOT IMPLEMENTED object_pairs_hook: NOT IMPLEMENTED use_decimal: NOT IMPLEMENTED **kw: NOT IMPLEMENTED
- Returns (Any):
- if single_value is True:
- A Python object representing a single Ion value.
- else:
- A sequence of Python objects representing a stream of Ion values.
amazon.ion.symbols module¶
Provides support for Ion symbol tokens.
-
class
amazon.ion.symbols.
ImportLocation
[source]¶ Bases:
amazon.ion.util.ImportLocation
Represents the import location of a symbol token.
An import location can be thought of as a position independent address of an imported symbol. This metadata, if defined, can indicate identity of a symbol that came from a shared symbol token that the application does not have access to.
- Note:
- Version is nor part of an import location because a position in a shared table by name uniquely identifies that slot irrespective of version.
- Args:
- name (unicode): The name of the shared symbol table. position (int): The position in the shared symbol table.
-
class
amazon.ion.symbols.
SymbolTable
(table_type, symbols, name=None, version=None, imports=None, is_substitute=False)[source]¶ Bases:
object
A collection of symbols that is ordered.
Symbol tables are basically an Unicode string to integer interning table.
A few things to consider about symbol tables:
- System symbol tables never have imports and are shared symbol tables themselves.
- Shared symbol tables never import the system symbol table.
- Local symbol tables implicitly import the system symbol table.
Shared symbol tables have tokens that always have a
location
attribute referring to themselves. Local symbol tables have tokens whoselocation
attribute refers to either the shared symbol that it was imported from orNone
if the symbol was defined locally.- Note:
- Shared symbol tables (which include system symbol tables) are immutable. Local symbol tables support interning as a mutable operation. The implementation doesn’t enforce making properties read-only to enforce this invariant.
- Args:
table_type (_SymbolTableType): The type of symbol table. symbols (Iterable[unicode]): The symbols text associated locally to this table. imports (Optional[Iterable[SymbolTable]]): The imports of the table. name (Optional[unicode]): The name of this table. Required for shared symbol tables. version (Optional[int]): The version of this table. Required for shared symbol tables. is_substitute (Optional[bool]): Whether or not this table is substituted. A substituted symbol
table is one that is not resolvable and has placeholder entries.
-
get
(key, default=None)[source]¶ Returns a token by text or local ID, with a default.
A given text image may be associated with more than one symbol ID. This will return the first definition.
- Note:
- User defined symbol IDs are always one-based. Symbol zero is a special symbol that always has no text.
- Args:
- key (unicode | int): The key to lookup. default(Optional[SymbolToken]): The default to return if the key is not found
- Returns:
- SymbolToken: The token associated with the key or the default if it doesn’t exist.
-
class
amazon.ion.symbols.
SymbolTableCatalog
[source]¶ Bases:
object
A collection of symbol tables that can be used to resolve imports.
- Note:
- The catalog will return a placeholder symbol table when resolving a table that doesn’t exist. For tables that don’t exist with any version, this placeholder will be completely devoid of text mappings. For tables that exist with a non-exact version, a derived substitute will be generated.
-
register
(table)[source]¶ Adds a shared table to the catalog.
- Args:
- table (SymbolTable): A non-system, shared symbol table.
-
resolve
(name, version, max_id)[source]¶ Resolves the table for a given name and version.
- Args:
name (unicode): The name of the table to resolve. version (int): The version of the table to resolve. max_id (Optional[int]): The maximum ID of the table requested.
May beNone
in which case an exact match onname
andversion
is required.- Returns:
- SymbolTable: The closest matching symbol table. This is either an exact match, a placeholder, or a derived substitute depending on what tables are registered.
-
class
amazon.ion.symbols.
SymbolToken
[source]¶ Bases:
amazon.ion.util.SymbolToken
Representation of a symbolic token.
A symbolic token may be a part of an Ion value in several contexts:
- The field name of a value in a
struct
- An annotation of a value.
- A scalar of type
symbol
in the Ion data model.
- Args:
- text (Optional[unicode]): The text image of the token. sid (Optional[int]): The local symbol ID of the token. location (Optional[ImportTableLocation]): The import source of the token.
- Note:
- At least one of
text
orsid
should be non-None
- The field name of a value in a
-
amazon.ion.symbols.
local_symbol_table
(imports=None, symbols=())[source]¶ Constructs a local symbol table.
- Args:
- imports (Optional[SymbolTable]): Shared symbol tables to import. symbols (Optional[Iterable[Unicode]]): Initial local symbols to add.
- Returns:
- SymbolTable: A mutable local symbol table with the seeded local symbols.
-
amazon.ion.symbols.
placeholder_symbol_table
(name, version, max_id)[source]¶ Constructs a shared symbol table that consists symbols that all have no known text.
This is generally used for cases where a shared symbol table is not available by the application.
- Args:
- name (unicode): The name of the shared symbol table.
version (int): The version of the shared symbol table.
max_id (int): The maximum ID allocated by this symbol table, must be
>= 0
- Returns:
- SymbolTable: The synthesized table.
Constructs a shared symbol table.
- Args:
- name (unicode): The name of the shared symbol table. version (int): The version of the shared symbol table. symbols (Iterable[unicode]): The symbols to associate with the table. imports (Optional[Iterable[SymbolTable]): The shared symbol tables to inject into this one.
- Returns:
- SymbolTable: The constructed table.
-
amazon.ion.symbols.
substitute_symbol_table
(table, version, max_id)[source]¶ Substitutes a given shared symbol table for another version.
- If the given table has more symbols than the requested substitute, then the generated symbol table will be a subset of the given table.
- If the given table has less symbols than the requested substitute, then the generated symbol table will have symbols with unknown text generated for the difference.
- Args:
- table (SymbolTable): The shared table to derive from.
version (int): The version to target.
max_id (int): The maximum ID allocated by the substitute, must be
>= 0
. - Returns:
- SymbolTable: The synthesized table.
amazon.ion.util module¶
General purpose utilities.
-
class
amazon.ion.util.
CodePoint
(*args, **kwargs)[source]¶ Bases:
int
Evaluates as the ordinal of a code point, while also containing the unicode character representation and indicating whether the code point was escaped.
-
class
amazon.ion.util.
Enum
(name, value)[source]¶ Bases:
int
Simple integer based enumeration type.
- Examples:
The typical declaration looks like:
class MyEnum(Enum): A = 1 B = 2 C = 3
At this point
MyEnum.A
is an instance ofMyEnum
.- Note:
Proper enumerations were added in Python 3.4 (PEP 435), this is a very simplified implementation based loosely on that specification.
In particular, implicit order of the values is not supported.
- Args:
- value (int): the value associated with the enumeration.
- Attributes:
- name (str): The name of the enum. value (int): The original value associated with the enum.
-
amazon.ion.util.
coroutine
(func)[source]¶ Wraps a PEP-342 enhanced generator in a way that avoids boilerplate of the “priming” call to
next
.- Args:
- func (Callable): The function constructing a generator to decorate.
- Returns:
- Callable: The decorated generator.
-
amazon.ion.util.
record
(*fields)[source]¶ Constructs a type that can be extended to create immutable, value types.
- Examples:
A typical declaration looks like:
class MyRecord(record('a', ('b', 1))): pass
The above would make a sub-class of
collections.namedtuple
that was namedMyRecord
with a constructor that had theb
field set to 1 by default.- Note:
- This uses meta-class machinery to rewrite the inheritance hierarchy.
This is done in order to make sure that the underlying
namedtuple
instance is bound to the right type name and to make sure that the synthetic class that is generated to enable this machinery is not enabled for sub-classes of a user’s record class. - Args:
- fields (list[str | (str, any)]): A sequence of str or pairs that
-
amazon.ion.util.
total_seconds
(td)[source]¶ Timedelta
total_seconds
with backported support in Python 2.6
-
amazon.ion.util.
unicode_iter
(val)[source]¶ Provides an iterator over the code points of the given Unicode sequence.
- Notes:
- Before PEP-393, Python has the potential to support Unicode as UTF-16 or UTF-32.
This is reified in the property as
sys.maxunicode
. As a result, naive iteration of Unicode sequences will render non-character code points such as UTF-16 surrogates. - Args:
- val (unicode): The unicode sequence to iterate over as integer code points in the range
0x0
to0x10FFFF
.
amazon.ion.writer module¶
Provides common functionality for Ion binary and text writers.
-
class
amazon.ion.writer.
WriteEventType
(name, value)[source]¶ Bases:
amazon.ion.util.Enum
Events that can come from an Ion writer co-routine.
- Attributes:
- HAS_PENDING: Indicates that the writer has more pending events to yield,
- and that it should be sent
None
as the event to flush them out. Generally, this is signalled when an internal buffer has been filled by an input event and needs to be flushed in order to make progress. - NEEDS_INPUT: Indicates that the writer has no pending events to yield
- and that it should be sent a
amazon.ion.core.IonEvent
. - COMPLETE: Indicates that the writer has flushed out complete Ion values at the top-level.
- This is similar to
NEEDS_INPUT
except that it signifies point that all data emitted by the writer is in sync with the events given to it.
-
COMPLETE
= <WriteEventType.COMPLETE: 3>¶
-
HAS_PENDING
= <WriteEventType.HAS_PENDING: 1>¶
-
NEEDS_INPUT
= <WriteEventType.NEEDS_INPUT: 2>¶
amazon.ion.writer_binary module¶
Binary Ion writer with symbol table management.
-
amazon.ion.writer_binary.
binary_writer
(imports=None)[source]¶ Returns a binary writer co-routine.
- Keyword Args:
- imports (Optional[Sequence[SymbolTable]]): A list of shared symbol tables
- to be used by this writer.
- Yields:
DataEvent: serialization events to write out
Receives
amazon.ion.core.IonEvent
.
amazon.ion.writer_binary_raw module¶
Writer for raw binary Ion values, without symbol table management.
amazon.ion.writer_binary_raw_fields module¶
Methods for writing binary Ion Int, VarInt, UInt, and VarUInt fields.
Whenever the following words are used in variable or method names, their meaning is as defined below.
signed
: applies only to Int and VarInt - the signed fields (sign bit in first octet).unsigned
: applies only to UInt and VarUInt - the unsigned fields (no sign bit).variable
: applies only to VarInt and VarUInt - the variable-length fields (end bit required).fixed
: applies only to Int and UInt - the fixed-length fields (no end bit).Int
: applies only to the Int field - the signed, fixed-length field.varint
: applies only to the VarInt field - the signed, variable-length field.uint
: applies only to the UInt field - the unsigned, fixed-length field.varuint
: applies only to the VarUInt field - the unsigned, variable-length field.
amazon.ion.writer_buffer module¶
Buffer for binary Ion writers.
-
class
amazon.ion.writer_buffer.
BufferTree
[source]¶ A tree of buffers that can be depth-first traversed to produce a correctly-ordered Ion stream.
Containers are treated as subtrees. Calling
start_container
creates an empty node which represents the start of a new subtree, while callingend_container
steps out of that subtree, allowing for additional nodes to be added as siblings of that subtree’s start node. It is important that there is exactly one call to start_container per call toend_container
. Nodes representing the start of a container subtree are kept in a stack; upon callingend_container
, the subtree represented by the node that was created by the most recent call to start_container will be stepped out. Other than that, no container semantics are asserted by this class.Scalar values are treated as child nodes of the current container subtree (which may be at the top-level if the current container subtree’s root node is the root node of the BufferTree). Calling add_scalar_value will add a new node as a child of the current container subtree. A container subtree’s children are ordered.
If None values are passed in to end_container or add_scalar_value, nodes will be added to the tree, in the same way as described above, but no value will be yielded for that node upon drain.
-
add_scalar_value
(value_buf)[source]¶ Add a node to the tree containing a scalar value.
- Args:
- value_buf (bytearray): bytearray containing the scalar value.
-
drain
()[source]¶ Walk the BufferTree and reset it when finished.
- Yields:
- any: The current node’s value.
-
amazon.ion.writer_text module¶
Implementations of Ion Text writers.
-
amazon.ion.writer_text.
raw_writer
(indent=None)[source]¶ Returns a raw text writer co-routine.
- Yields:
DataEvent: serialization events to write out
Receives
amazon.ion.core.IonEvent
orNone
when the co-routine yieldsHAS_PENDING
WriteEventType
events.
-
amazon.ion.writer_text.
text_writer
(indent=None)¶ Returns a raw text writer co-routine.
- Yields:
DataEvent: serialization events to write out
Receives
amazon.ion.core.IonEvent
orNone
when the co-routine yieldsHAS_PENDING
WriteEventType
events.