scistag.datastag.data_stag_connection.DataStagConnection¶

class DataStagConnection(url='', local=True, _request_client=None)[source]¶

Bases: object

Defines a connection to a DataStack vault - locally or remote

Initializer

Parameters

url (str) – The connection url of a remote server
local – Defines if the local in memory database shall be used
_request_client (FlaskClient) – For unit tests only. Uses Flask’s internal test client to test the API.

Methods

`add`	Adds given value to the element stored in the database.
`collect_garbage`	Implicit garbage collection request
`data_to_json`	Converts data of a binary type, e.g.
`delete`	Deletes an element :type name: `str` :param name: The element's name.
`delete_multiple`	Deletes a set of elements
`exists`	Verifies if given element exists
`find`	Finds a list of elements by name
`get`	Tries to read an element from the database :type name: `str` :param name: The element's name :type default: `Union`[`str`, `bool`, `int`, `float`, `bytes`, `dict`, `list`, `ndarray`, `None`] :param default: The default return value if the element does nto exist :rtype: `Union`[`str`, `bool`, `int`, `float`, `bytes`, `dict`, `list`, `ndarray`, `None`] :return: The element
`get_ex`	Tries to read an element from the database.
`get_status`	Returns the database status
`get_ts`	Returns an object's update time stamp (if it was set via set_ts)
`get_ts_modified`	Returns the object if it was modified since timestamp :param name: The element's name :param timestamp: The previous timestamp :return: The timestamp or counter, the data on success, otherwise the old timestamp, None
`get_values_by_name`	Returns the data of a set of elements by name.
`json_to_data`	Converts an element converted to JSOn back to it's original type.
`lelements`	Tries to receive a list of elements from the vault
`llen`	Tries to retrieve the data of a list
`pop`	Tries to remove an element at the beginning of a list
`push`	Inserts an element at the beginning of a list
`set`	Stores a named element in the database
`set_ts`	Stores a named element in the database and adds a timestamp to it in a neighbour entry named :timeStamp
`start_transaction`	Starts a new transaction. All write operations will be bundled and
`ts_lock`	Locks the access to the vault and bundles upcoming write events
`ts_unlock`	Unlocks the access to the vault and bundles upcoming write events

Attributes

`_ADVANCED`
`_COMMAND`
`_COMMAND_ADD`
`_COMMAND_COLLECT_GARBAGE`
`_COMMAND_DELETE`
`_COMMAND_DELETE_MULTIPLE`
`_COMMAND_EXISTS`
`_COMMAND_FIND`
`_COMMAND_GET`
`_COMMAND_GET_EX`
`_COMMAND_GET_VALUES_BY_NAME`
`_COMMAND_LELEMENTS`
`_COMMAND_LLEN`
`_COMMAND_POP`
`_COMMAND_PUSH`
`_COMMAND_SET`
`_COMMAND_STATUS`
`_COUNTER_IDENTIFIER`	Name of a timestamp neighbour entry in which the time when an entry was written is stored
`_DATA`
`_DEFAULT`
`_ELEMENTS`
`_END`
`_FLAT`
`_INDEX`
`_LIMIT`
`_MASK`
`_NAME`
`_RECURSIVE`
`_RELATIVE_NAMES`
`_SEARCH_MASKS`
`_START`
`_TIMESTAMP_IDENTIFIER`
`_TIME_OUT`
`_TYPE`
`_TYPE_BYTES`
`_TYPE_NUMPY`
`_VALUE`
`_VERSION_COUNTER`
`__dict__`
`__doc__`
`__module__`
`__weakref__`	list of weak references to the object (if defined)
`local`	Defines if a local storage is being used
`vault`	Link to the local data vault
`target_url`	The target url (when not using a local db)
`request_client`	When testing locally this links to the local test client
`_cur_transaction`	The current transaction

_execute_remote(command)[source]¶

Executes a command remotely and returns it’s result

Parameters: command (dict | list) – The command or list of commands
Return type: dict | list | str | float | int | bool | None
Returns: The result

_verify_bool_result(result)[source]¶

Verifies if the result is of type bool

Parameters: result (object) – The result
Return type: bool
Returns: The result value if the type is a bool, otherwise None

_verify_ct_result(result)[source]¶

Verifies if the result is of any allowed storable type

Parameters: result (dict) – The result
Return type: int
Returns: The result value if the type is allowed, otherwise None

_verify_dict_result(result)[source]¶

Verifies if the result is of type dict

Parameters: result (object) – The result
Return type: dict
Returns: The result value if the type is a dict, otherwise None

_verify_int_result(result)[source]¶

Verifies if the result is of type int

Parameters: result (object) – The result
Return type: int
Returns: The result value if the type is an integer, otherwise None

classmethod _verify_result(result, supported_types)[source]¶

Verifies if the result is of a given type

Parameters

result – The result
supported_types (list) – The list of types which are allowed for this time

Return type

Returns

The result value if the type is correct, otherwise None

add(name, value=1, timeout_s=None, default=0)[source]¶

Adds given value to the element stored in the database. If it does not exist yet, it will be initialized with default.

Parameters

name – The element’s name
value – The value to be added. 1 by default.
timeout_s – The timeout for automatic deletion
default – The default value

Returns

The new value

collect_garbage()[source]¶

Implicit garbage collection request

Return type: bool
Returns: True if at least one time interval passed and something could get collected

classmethod data_to_json(data)[source]¶

Converts data of a binary type, e.g. np.ndarray or bytes to a JSON representation

Parameters: data (Union[str, bool, int, float, bytes, dict, ndarray, list]) – The data
Return type: Union[str, bool, int, float, bytes, dict, list, ndarray, None]
Returns: The JSON representation which can decoded on the receiver side again using json_to_data

delete(name)[source]¶: Deletes an element :type name: str :param name: The element’s name. :rtype: bool :return: True on success

delete_multiple(search_masks, recursive=False)[source]¶

Deletes a set of elements

Parameters

search_masks (list[str]) – The element’s names or search masks. May not point directly to the root directory
recursive (bool) – Defines if the search shall be executed recursive

Return type

int

Returns

The count of removed elements

exists(name)[source]¶

Verifies if given element exists

Parameters: name (str) – The element’s name
Return type: bool
Returns: True if the element exists

find(mask, limit=100, relative_names=False, recursive=False)[source]¶

Finds a list of elements by name

Parameters

mask (str) – The search mask. If it contains a folder the mask is only applied to the nested element
limit (int) – The maximum count of entries
relative_names (bool) – Defines if the relative names shall be returned
recursive (bool) – Defines if the search shall be executed recursive

Return type

list[Union[str, bool, int, float, bytes, dict, ndarray, list]]

Returns

A list of elements of all valid elements matching the search mask

get(name, default=None)[source]¶: Tries to read an element from the database :type name: str :param name: The element’s name :type default: Union[str, bool, int, float, bytes, dict, list, ndarray, None] :param default: The default return value if the element does nto exist :rtype: Union[str, bool, int, float, bytes, dict, list, ndarray, None] :return: The element

get_ex(name, default=None, version_counter=-1)[source]¶: Tries to read an element from the database. Allows to add a version check so only data will be returned if it changed since the last get_ex. :param name: The element’s name :param default: The default return value if the element does not exist :param version_counter: If set then a value will only be returned if the element’s update counter does not match :return: The element’s version, The element

get_status(advanced=False)[source]¶

Returns the database status

Parameters: advanced (bool) – Defines if advanced details shall be received as well
Returns: A dictionary containing the status

get_ts(name, default=0.0)[source]¶

Returns an object’s update time stamp (if it was set via set_ts)

Parameters

name (str) – The element’s name
default (float) – The default return value

Return type

float

Returns

The timestamp on success

get_ts_modified(name, timestamp=0.0)[source]¶: Returns the object if it was modified since timestamp :param name: The element’s name :param timestamp: The previous timestamp :return: The timestamp or counter, the data on success, otherwise the

old timestamp, None

get_values_by_name(mask, limit=100, flat=True)[source]¶

Returns the data of a set of elements by name.

Parameters

mask (str) – The search mask. If it contains a folder the mask is only applied to the nested element
limit (int) – The maximum count of entries
flat (bool) – Returns a list of all values received without providing the element names

Returns

A list containing the data and names of all valid elements

classmethod json_to_data(data)[source]¶

Converts an element converted to JSOn back to it’s original type. Special types, e.g. binary representations are flagged as those using a _TYPE and a _VALUE element in a dictionary.

Parameters: data – The result
Return type: Union[str, bool, int, float, bytes, dict, list, ndarray, None]
Returns: The data in the original type, e.g. byte or np.ndarray

lelements(name, start=0, end=None)[source]¶

Tries to receive a list of elements from the vault

Parameters

name (str) – The list’s name
start (int) – The first element’s index
end (int | None) – The stop index (not included anymore). -1 = end of the list

Return type

list[StagDataTypes]

Returns

The list’s content in given range

llen(name)[source]¶

Tries to retrieve the data of a list

Parameters: name (str) – The list’s name
Return type: int
Returns: The list length’s if it does exist. Otherwise 0.

pop(name, default=None, index=0)[source]¶

Tries to remove an element at the beginning of a list

Parameters

name (str) – The name’s list
default (Union[str, bool, int, float, bytes, dict, list, ndarray, None]) – The default return value if the list is empty or does not exist
index (int) – The index from which the data shall be received. By default at the front

Return type

Union[str, bool, int, float, bytes, dict, list, ndarray, None]

Returns

The element which was removed

push(name, data, timeout_s=None, index=-1)[source]¶

Inserts an element at the beginning of a list

Parameters

name (str) – The element’s name
data (list[StagDataTypes] | StagDataTypes) – The data to be added. See StagDataTypes for supported types
timeout_s (float | None) – The timeout for automatic deletion
index (int) – The index at which the data shall be inserted. By default at the end.

Return type

int

Returns

The new length of the list

set(name, data, timeout_s=None)[source]¶

Stores a named element in the database

Parameters

name (str) – The element’s name
data (StagDataTypes) – The data to be added. See StagDataTypes for supported types
timeout_s (float | None) – The timeout for automatic deletion

Return type

bool

Returns

True on success

set_ts(name, data, timeout_s=None, timestamp=None)[source]¶

Stores a named element in the database and adds a timestamp to it in a neighbour entry named :timeStamp

Parameters

name (str) – The element’s name
data (StagDataTypes) – The data to be added. See StagDataTypes for supported types
timeout_s (float | None) – The timeout for automatic deletion
timestamp (float | None) – A custom timestamp. If None is provided the current time will be used

Return type

bool

Returns

True on success

start_transaction()[source]¶

Starts a new transaction. All write operations will be bundled and: executed in a single lock.

Use like: with my_connection.start_transaction(): … :return: The transaction object

ts_lock(target)[source]¶

Locks the access to the vault and bundles upcoming write events

Parameters: target (DataStagTransaction) – The current transaction

ts_unlock()[source]¶: Unlocks the access to the vault and bundles upcoming write events

_COUNTER_IDENTIFIER = ':counter'¶: Name of a timestamp neighbour entry in which the time when an entry was written is stored

_cur_transaction: 'DataStagTransaction' | None¶: The current transaction

local¶: Defines if a local storage is being used

request_client: FlaskClient¶: When testing locally this links to the local test client

target_url: str¶: The target url (when not using a local db)

vault: DataStagVault | None¶: Link to the local data vault