emm.loggers package

Submodules

emm.loggers.logger module

We define the logger that is going to be used in the entire package. We should not configure the logger, that is the responsibility of the user. By default, in Python the log level is set to WARNING.

emm.loggers.logger.logSchema(df)

emm.loggers.logger.logShow(df, n=20, truncate=True, vertical=False)

Equivalent of show() but for logging Copy pasted from https://spark.apache.org/docs/latest/api/python/_modules/pyspark/sql/dataframe.html#DataFrame.show

Parameters:

• n (int)

• truncate (bool)

• vertical (bool)

emm.loggers.logger.set_logger(level=20, format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s')

emm.loggers.timer module

class emm.loggers.timer.Timer(label)

Bases: ContextDecorator

Context manager that logs the timing of labelled blocks of code

Example:

>>> with Timer("label") as timer:
>>>     timer.label("part 1")
>>>     ...
>>>
>>>     timer.label("part 2")
>>>     ...

difference()

end()

label(name)

Labelled checkpoint

Args:

name: label for block of code

Raises:

ValueError: if reserved or used name is provided

Parameters:

name (str)

Return type:

None

log_param(key, value)

Parameters:

• key (str)

• value (Any)

log_params(value)

Parameters:

value (dict)

start()

emm.loggers.timer.format_values(values)

Parameters:

values (dict[str, Any])

Return type:

str

Module contents

class emm.loggers.Timer(label)

Bases: ContextDecorator

Context manager that logs the timing of labelled blocks of code

Example:

>>> with Timer("label") as timer:
>>>     timer.label("part 1")
>>>     ...
>>>
>>>     timer.label("part 2")
>>>     ...

difference()

end()

label(name)

Labelled checkpoint

Args:

name: label for block of code

Raises:

ValueError: if reserved or used name is provided

Parameters:

name (str)

Return type:

None

log_param(key, value)

Parameters:

• key (str)

• value (Any)

log_params(value)

Parameters:

value (dict)

start()