scistag.vislog.visual_log.VisualLog¶
- class VisualLog(title='SciStag - VisualLog', target_dir='./logs', formats_out=None, ref_dir=None, tmp_dir=None, clear_target_dir=False, log_to_disk=True, log_to_stdout=True, embed_images=None, continuous_write=False, refresh_time_s=0.5, max_fig_size=None, image_format='png', image_quality=90, cache_dir=None, cache_version=1, cache_name='', auto_reload=False)[source]¶
Bases:
objectThe VisualLog class enables you to create detailed, static data processing documentation logs in various formats such as html, md and pdf up to complex interactive browser based HTML and JavaScript applications forwarding all user inputs via JavaScript to your Python logic and the updates you trigger back to the user’s view in the browser.
To view the log live in your IDE next to your code: - Build the log via run_server. - For a detailed example see the camera-demo at
- PyCharm: Ctrl-Shift-A -> Open Source Code From URL -> Paste the live
url -> Click on the small PyCharm icon in the upper right corner
- VS Code: Not supported yet. Open the /live URL above in a browser and
align your IDE and browser windows side by side, e.g. with Win Key+Left and Win Key+Right
- Parameters
target_dir (str) – The output directory
title (str) – The log’s name
formats_out (set[str] | None) –
A set of the formats to export.
”html”, “txt” (pure Text) and “md” (markdown) are supported.
By default only html files will be created.
ref_dir (str | None) – The directory in which the reference data objects can be stored.
tmp_dir (str | None) – A directory in which temporary files can be stored. Will be deleted upon finalization.
clear_target_dir (bool) – Defines if the target dir shall be deleted before starting (take care!)
log_to_disk – Defines if the logger shall write it’s results to disk. True by default.
log_to_stdout – Defines if the system shall automatically log to stdout via print as well
embed_images (bool | None) –
Defines if images shall be directly embedded into the HTML log instead of being stored as separate files.
By default True if Markdown is not set as one of the “formats_out”, otherwise False by default as Markdown will need the files on disk.
continuous_write – Defines if the log shall be written to disk after every added element. False by default.
refresh_time_s – The time interval in seconds in which the auto-reloader html page (liveView.html) tries to refresh the page. The lower the time the more often the page is refreshed.
max_fig_size (Size2DTypes | None) – The optimum, maximum width and height for embedded figures and images
image_format (str | tuple[str, int]) –
The default output image format to store images and figures with. “png” by default.
You can also pass the image format and image quality in a tuple such as (“jpg”, 60).
Alternatively “jpg” or “bmp” can be used (to minimize the bandwidth or in the later case if you are the intranet w/ unlimited bandwidth and want to host it live at maximum performance).
image_quality (int) –
The default image output quality. 90 by default.
Values between 0 and 100 are valid.
cache_version (int) –
The cache version. 1 by default.
When ever you change this version all old cache values will be removed and/or ignored from the cache.
cache_dir (str | None) –
The directory in which data which shall be cached between multiple execution sessions shall be dumped to disk.
By default “{target_dir}/.stscache”.
cache_name (str) – The cache’s identifier. If multiple logs store data into the same logging directory this can be used to ensure their caching directories don’t accidentally overlap w/o having to provide the whole path via cache_dir.
auto_reload (bool | BuilderTypes) – Defines if this log will be executed in auto_reload mode in its cache should be update and restored each turn.
Methods
Adds a button to the log which can be clicked and raise a click event.
Adds an advanced console as target to the log
Adds an event to the event queue which will be handled before and after the next re-build (or loop turn in case of a continuous log).
Provides a file statically, e.g. to provide it via a
Pushes the current log target to create a sub log.
Clears the whole log (excluding headers and footers)
Checks if the log limited exceeded and clips old logs if necessary.
Creates a web service which provides (for example) a blueprint you can add to an arbitrary Flask server.
Embeds another VisualLog's content into this one
Ends a sub log, aggregates all logs which participated to the current target and stores the content in sub_log_data[target] which can then be used to customize def get_body()
Finalizes the report and writes it to disk
Writes the current state to disk
Returns the latest body data.
Returns the current list of events
Tries to receive a file created by this log, either stored locally or in memory via
add_static_file().Receives the newest update of the page of given output type.
Returns statistics about the log
Returns the temporary file path.
Handles a single event and forwards it to the correct widget
Handles all queued events and clears the event queue
Flag this log as invalidate for inform the auto-reloader that this log should be reloaded
Returns if the file calling this method was the main entry point before the module got reloaded.
Kills the http server running in the background.
Tries to load the old logs from disk so they can be hosted via
Prepapres the builder to be used for this log
Registers a widget which is able to receive events
Renders all pages - so combines the main log with the sub logs of the supported output types (html, txt, md etc.) and stores them.
Reserves a unique name within the log, e.g.
Helper function to update the log via a callback function.
Hosts the log as web service.
Stores a copy of the latest page.
Changes the maximum count of log rows for the current sub log.
Creates a set of files in the defined directory which contain replacements for the essential logging classes such as VisualLog, VisualLogBuilder etc.
Sleeps and handles input events until the application is either terminated by Ctrl-L or by an exit button added to the log or a quit call triggered.
Sets the termination state to true so that if the log was initialized with the flag continuous=True it can be terminated from within the logging function.
The HTML code to add
The markdown code to add
Writes the rendered pages from all (or all specified) formats to disk.
Adds text code to the txt / console log
Attributes
__dict____doc____module____weakref__list of weak references to the object (if defined)
Returns the log's cache object to cache computation data between functions repetitions or even multiple execution sessions.
Returns if this log was invalidated and should be rebuilt
Returns if this builder is a minimalistic logger with limited functionality.
The maximum figure size in pixels
Returns the server (if one was created) and started via
run_server().The log's data cache to store computation results between execution sessions
The log's title
The directory in which the logs shall be stored
The directory in which reference files for comparison shall be stored
Output directory for temporary files
Defines if the images and the html data shall be written to disk
Defines if images shall be logged to disk
The time interval with which the log shall be refreshed when using the liveViewer (see Live_view)
Defines if text shall be logged
A stag for temporary switching log targets and to created 'nested" logs.
Contains the content of each "sub log", see
begin_sub_log().Contains the log data for each output type
If defined the output logs will be updated after every log
Defines if markdown shall support html embedding
Defines if images shall also be logged to text files as ASCII
Defines if tabulate may be used
Defines if pretty html shall be used
The pretty html style to be used
The text table format to use in tabulate
The markdown table format to use
The default image type to use for storage
The image compression quality
Defines if HTML gets exported
Defines if markdown gets exported
Defines if txt gets exported
The name of the txt file to which we shall save
The name of the html file to which we shall save
The name of the markdown file to which we shall save
Attached consoles to which the data shall be logged
The current log limit (maximum number of rows before starting deleting the oldest ones)
The renderers for the single supported formats
Lock for multithread secure access to the latest page update
A backup of the latest rendered page of each dynamic data type (excluding PDFs and PNGs which are just created on demand)
A backup of the latest body renderings for each content type
Statically hosted files for a pure web based provision of the log
Defines if the log service shall be terminated, e.g if it's running endlessly via
run()orrun_server().The web server (if one was being started via
run_server())The time stamp of when the log was creation
List of unhandled events
Set of widgets
Defines if all log messages shall also be send to stdout via print
Defines if this log was invalidated via
invalidate()Counter for file names to prevent writing to the same file twice
Counter for titles to numerate the if appearing twice
The total number of updates to this log
THe last time the _update rate was computed as time stamp
The default builder.
Defines if the log is run in test mode and e.g.
- _add_to_console(txt_code)[source]¶
Adds text code to the console log
- Parameters
txt_code (
str) – The text to add- Returns
True if txt logging is enabled
- _build_body(base_log)[source]¶
Requests to combine all logs and sub logs to a single page which can be logged to the disk or provided in the browser. (excluding html headers and footers), so just “the body” of the HTML page.
- Parameters
base_log (
dict[slice(<class ‘str’>, <class ‘bytes’>, None)]) – The byte stream of all concatenated logs for each output type.- Returns
The finalized page, e.g. by combining base_log w/ sub_logs as shown in the
VisualLiveLogclass.
- static _get_module_path()[source]¶
Returns the path of the VisualStag module
- Return type
- Returns
The path
- _provide_live_view()[source]¶
Assembles a website file which automatically updates the logged html file as often as possible when ever it is updated on disk.
- _run_builder(builder=None)[source]¶
Runs the associated builder
- Parameters
builder (BuilderTypes | None) – The builder to be called from rebuild the log
- _run_continuous(auto_clear, builder)[source]¶
Runs the builder until
terminate()is called.- Parameters
auto_clear (
bool) – Defines if the log shall be cleared each turnbuilder (
Callable[[VisualLogBuilder],None]) – The builder function to call
- _setup_cache(auto_reload, cache_version, cache_dir, cache_name)[source]¶
Configures the data cache
- Parameters
auto_reload – Auto-reloading used?
cache_version –
The cache version. 1 by default.
When ever you change this version all old cache values will be removed and/or ignored from the cache.
cache_dir – The cache target directory on disk
cache_name – The unique name of the cache, e.g. for the case multiple logs use the same logging directory
- _update_statistics(cur_time)[source]¶
Updates the statistics if necessary
- Parameters
cur_time (
float) – The current system time (in seconds)
- add_button(name, caption, on_click=None)[source]¶
Adds a button to the log which can be clicked and raise a click event.
- add_console(console)[source]¶
Adds an advanced console as target to the log
- Parameters
console (
Console) – The console to add
- add_event(event)[source]¶
Adds an event to the event queue which will be handled before and after the next re-build (or loop turn in case of a continuous log).
- Parameters
event – The new event
- add_static_file(filename, content)[source]¶
- Provides a file statically, e.g. to provide it via a
VisualLiveLogServer.
Multi-thread safe function.
- begin_sub_log(target, max_fig_size=None)[source]¶
Pushes the current log target to create a sub log.
You can call this method for the same target multiple times so the logs get attached to each other. When ever
end_sub_log()is called self.sub_log_data is updated with all elements on the stack which participate towards the same target. These can then (for example) be used to combine them to a custom html or txt log via customizing the get_body function.Usage:
- Parameters
target (str) – The sub log’s name in which the content shall be stored. See
sub_log_data.max_fig_size (Size2DTypes | None) – Defines the maximum size of visual elements
- Return type
- create_web_service(support_flask=False, url_prefix='')[source]¶
Creates a web service which provides (for example) a blueprint you can add to an arbitrary Flask server.
- Parameters
support_flask (
bool) – Support Flask (and setup a blueprint?)url_prefix (
str) –The url prefix at which the service shall be hosted.
”” = At http://server “log/” = At http://server/log
- Return type
- Returns
The service object containing the services for the request backends (e.g. flask, fastapi etc.)
- embed(log_data)[source]¶
Embeds another VisualLog’s content into this one
- Parameters
log_data (
VisualLog) – The source log
- end_sub_log()[source]¶
Ends a sub log, aggregates all logs which participated to the current target and stores the content in sub_log_data[target] which can then be used to customize def get_body()
- finalize()[source]¶
Finalizes the report and writes it to disk
- Return type
- Returns
The VisualLog object
- get_body(format_type)[source]¶
Returns the latest body data.
Contains only the part of that format w/ header and footer. Can be used to for example embed one log’s content in another log such as main_log.html(sub_log.get_body(“html”))
Assumes that render() or write_to_disk() or render was called before since the last change. This is not necessary if continuous_write is enabled.
- get_file(filename)[source]¶
Tries to receive a file created by this log, either stored locally or in memory via
add_static_file().
- get_page(format_type)[source]¶
Receives the newest update of the page of given output type.
If not done automatically (e.g. when using a VisualLiveLog) you might have to call render_pages yourself.
This method is multi-threading secure.
Assumes that render() or write_to_disk() or render was called before since the last change. This is not necessary if continuous_write is enabled.
- get_statistics()[source]¶
Returns statistics about the log
- Return type
- Returns
A dictionary with statistics about the log such as - totalUpdateCount - How often was the log updated? - updatesPerSecond - How often was the log updated per second - upTime - How long is the log being updated?
- get_temp_path(relative=None)[source]¶
Returns the temporary file path. The data will be wiped upon the call of finalize.
- handle_event(event)[source]¶
Handles a single event and forwards it to the correct widget
- Parameters
event (
LogEvent) – The event to be handled
- invalidate()[source]¶
Flag this log as invalidate for inform the auto-reloader that this log should be reloaded
- classmethod is_main()[source]¶
Returns if the file calling this method was the main entry point before the module got reloaded.
Only available if auto-reloading is being used.
- Return type
- Returns
True if the calling method is in the main module.
- kill_server()[source]¶
Kills the http server running in the background.
If you hosted this log as a webserver running in the background using run_server(mt=True) you can use this method to (by force) kill the server being used. Note that this may lead to memory leaks and should only be used to really shut down an application and to for example prevent Flask keeping the process alive upon Ctrl-C.
- Return type
- Returns
True on success
- load_old_logs()[source]¶
- Tries to load the old logs from disk so they can be hosted via
- Return type
- Returns
True if the logs could be loaded
- prepare_builder(builder)[source]¶
Prepapres the builder to be used for this log
- Parameters
builder (
Union[Callable[[VisualLogBuilder],None],VisualLogBuilder,Type[VisualLogBuilder]]) – The build helper, either a function which fills the log or an ancestor of VisualLogBuilder implementing at least the build_body method to do the same.- Returns
The prepared build object
- render(formats=None)[source]¶
Renders all pages - so combines the main log with the sub logs of the supported output types (html, txt, md etc.) and stores them.
The page data for each type can be received via
get_latest_page().
- reserve_unique_name(name)[source]¶
Reserves a unique name within the log, e.g. to store an object to a unique file.
- Parameters
name (
str) – The desired name- Returns
The effective name with which the data shall be stored
- run(builder, continuous=None, auto_clear=None, overwrite=None, auto_reload=False)[source]¶
Helper function to update the log via a callback function.
This helps you to - Create the log only once when it was not yet created.
Pass overwrite=False.
- Run the method `continuous`ly and update or extend it on disk with a
defined frequency.
- When run in “continuous”-mode:
To just extend the log use auto_clear = False
To completely rewrite the log every turn pass auto_clear = True
- Parameters
builder (BuilderCallback) –
The function to be called to build or (repetitively) rebuild the log’s content.
The function can be called once - if continuous=False was passed, continuously with a frequency of
refresh_time_s(as passed to the constructor) if continuous=True was passed.continuous (bool | None) –
Defines if the run_server shall run until
terminate()was called to update the logs content continuously and write them to disk each turn.False by default.
auto_clear (bool | None) – Defines if then log shall be cleared automatically when being rebuild with continuous=True.
overwrite (bool | None) –
If set to False it will only call the builder function if there is no recent version of the log stored on disk.
If a valid log was found the builder method passed will never be called.
auto_reload (bool) –
If swt to True the module calling this function will automatically be reloaded on-the-fly when ever it is modified and saved and the log will be rebuilt from scratch.
Note that this will override many of the other objects specified in the call of this function such as
mt - As multithreading is required to use this feature
continous - which is is currently not supported yet.
…
- Return type
- Returns
False if overwrite=False was passed and a log could successfully be loaded, so that no run was required.
- run_server(host_name='127.0.0.1', port=8010, url_prefix='', public_ips=None, builder=None, continuous=None, wait=False, auto_clear=None, overwrite=None, mt=True, test=False, server_logs=False, show_urls=True, auto_reload=False, auto_reload_stag_level=1, **kwargs)[source]¶
Hosts the log as web service.
This way you can either provide the log as a static website or even update it dynamically and
- Parameters
host_name (str) –
The IP(s) to listen at.
127.0.0.1 = Local access only (default) as “there is no place like localhost”.
”0.0.0.0” = Listen at all local network adapters
port (int | tuple[int, int]) – The port ot listen at or a port range to select the first port within. 8010 by default. 0 for a random port.
url_prefix (str) –
The url prefix at which the service shall be hosted.
”” = At http://server “log/” = At http://server/log
public_ips (str | list[str] | None) –
If you run the service on a virtual machine in the cloud you can pass its public IPs to log the correct connection URls to the console.
If you pass “auto” as ip the public IP will be auto-detected via ipify.
builder (BuilderTypes | None) –
An (optional) function to be called to build or (repetitively) rebuild the log’s content.
The function can be called once - if continuous=False was passed, continuously with a frequency of
refresh_time_s(as passed to the constructor) if continuous=True was passed.Instead of passing a builder callback you can as well as also just fill the log with content before running
run_server().continuous (bool | None) –
Defines if the run_server shall run until
terminate()was called to update the logs content continuously.False by default.
wait (bool) –
Defines if also a non-continuous log shall wait till the log has been terminated. (via
terminate()) or the application was killed via Ctrl-C.Has no effect if threaded is False (because the server will anyway block the further execution then) or if continuous is set to True.
Basically it acts like the “continuous” mode just with the difference that the builder function is just called once.
auto_clear (bool | None) – Defines if then log shall be cleared automatically when being rebuild with continuous=True.
overwrite (bool | None) –
If set to False it will only call the builder function if there is no recent version of the log stored on disk.
This way you can host the log results of a (potentially) long-running data engineering or ML training session without accidentally re-running it.
mt (bool) –
If set to true the server will be started in a background thread and the method will return asap.
You have to pass mt=True if this log shall be updated `continuous`ly.
If the log is dynamic, but you do not want to be stuck in this function you can - instead of passing a builder - (optionally) just call
clear_logs()to clear the log and when you are done updating the logwrite_to_disk()or justrender()the update the page provided by the server,Example:
test (bool) – Defines if the server shall be created in test mode (just “virtually”)
server_logs (bool) – Defines if the Flask and/or FastAPI logs shall be enabled.
show_urls (bool) – Defines if the URLs at which the server can be reached shall be shown upon start
auto_reload –
If swt to True the module calling this function will automatically be reloaded on-the-fly when ever it is modified and saved and the log will be rebuilt from scratch.
Note that this will override many of the other objects specified in the call of this function such as
mt - As multithreading is required to use this feature
continuous - which is not supported yet.
…
auto_reload_stag_level (1) –
Defines which module shall be observed and reloaded upon modifications.
By default it is the method of the calling module (1). If you need e.g. to track the caller’s caller (2) increase this number accordingly.
kwargs – Additional parameters which shall be passed to the WebStagServer upon creation.
- set_latest_page(page_type, content)[source]¶
Stores a copy of the latest page.
This method is multi-threading secure.
- set_log_limit(limit)[source]¶
Changes the maximum count of log rows for the current sub log.
If the number gets exceeded it will automatically start deleting the oldest logs.
- Parameters
limit (
int) – The new limit. -1 = None
- static setup_mocks(target_dir='./')[source]¶
Creates a set of files in the defined directory which contain replacements for the essential logging classes such as VisualLog, VisualLogBuilder etc. which can be used on systems without a valid SciStag installation such as MicroPython.
- sleep()[source]¶
Sleeps and handles input events until the application is either terminated by Ctrl-L or by an exit button added to the log or a quit call triggered.
- terminate()[source]¶
Sets the termination state to true so that if the log was initialized with the flag continuous=True it can be terminated from within the logging function.
- write_html(html_code)[source]¶
The HTML code to add
- Parameters
html_code (
str) – The html code- Returns
True if txt logging is enabled
- write_to_disk(formats=None, render=True)[source]¶
Writes the rendered pages from all (or all specified) formats to disk.
- _events¶
List of unhandled events
- _html_export¶
Defines if HTML gets exported
- _html_filename¶
The name of the html file to which we shall save
- _invalid¶
Defines if this log was invalidated via
invalidate()
- _last_statistic_update¶
THe last time the _update rate was computed as time stamp
- _log_limit¶
The current log limit (maximum number of rows before starting deleting the oldest ones)
- _md_filename¶
The name of the markdown file to which we shall save
- _page_backups: dict[str, bytes]¶
A backup of the latest rendered page of each dynamic data type (excluding PDFs and PNGs which are just created on demand)
- _page_lock¶
Lock for multithread secure access to the latest page update
- _server: Union['WebStagServer', None]¶
The web server (if one was being started via
run_server())
- _shall_terminate¶
Defines if the log service shall be terminated, e.g if it’s running endlessly via
run()orrun_server().
- _title¶
The log’s title
- _total_update_counter¶
The total number of updates to this log
- _txt_filename¶
The name of the txt file to which we shall save
- _widgets¶
Set of widgets
- property cache: Cache¶
Returns the log’s cache object to cache computation data between functions repetitions or even multiple execution sessions.
- Return type
- continuous_write¶
If defined the output logs will be updated after every log
- default_builder: VisualLogBuilder¶
The default builder. It let’s you easily add content the log without the need of any callbacks.
- html_table_style¶
The pretty html style to be used
- image_format¶
The default image type to use for storage
- image_quality¶
The image compression quality
- property is_micro: bool¶
Returns if this builder is a minimalistic logger with limited functionality.
See
setup_mocks()- Return type
- Returns
True if it is a mock
- log_formats¶
Defines if text shall be logged
- log_images¶
Defines if images shall be logged to disk
- log_to_disk¶
Defines if the images and the html data shall be written to disk
- log_to_stdout¶
Defines if all log messages shall also be send to stdout via print
- log_txt_images¶
Defines if images shall also be logged to text files as ASCII
- markdown_html¶
Defines if markdown shall support html embedding
- md_export¶
Defines if markdown gets exported
- md_table_format¶
The markdown table format to use
- name_counter¶
Counter for file names to prevent writing to the same file twice
- ref_dir¶
The directory in which reference files for comparison shall be stored
- refresh_time_s¶
The time interval with which the log shall be refreshed when using the liveViewer (see Live_view)
- property server: WebStagServer¶
Returns the server (if one was created) and started via
run_server().- Return type
- start_time¶
The time stamp of when the log was creation
- sub_log_data: dict[str, dict[str, bytes]]¶
Contains the content of each “sub log”, see
begin_sub_log().
- target_dir¶
The directory in which the logs shall be stored
- testing¶
Defines if the log is run in test mode and e.g. shall not spawn a real http server
- title_counter¶
Counter for titles to numerate the if appearing twice
- tmp_path¶
Output directory for temporary files
- txt_export¶
Defines if txt gets exported
- txt_table_format¶
The text table format to use in tabulate
- use_pretty_html_table¶
Defines if pretty html shall be used
- use_tabulate¶
Defines if tabulate may be used