You could use the following approach:
- Redirect all log messages in the root logger (which you will get by calling
getLogger()
) to a QueueHandler
to accumulate the log messages in a queue.Queue
.
- In the intended output cell, start a
QueueListener
that wraps a StreamHandler
. The QueueListener
, as its name implies, will listen to new items on the logging queue. It will pass new items to the StreamHandler
, which will actually print them.
Assuming we want to print below cell 1, this could look as follows:
# Cell 1
import logging, queue, threading, time
from logging.handlers import QueueHandler, QueueListener
log_queue = queue.Queue(-1)
logging.getLogger().addHandler(QueueHandler(log_queue))
listener = QueueListener(log_queue, logging.StreamHandler())
listener.start()
In cell 2, we will simulate some activity:
# Cell 2
def log_activity_1():
while True:
logging.getLogger().warning("Activity 1")
time.sleep(1)
threading.Thread(target=log_activity_1, daemon=True).start()
And likewise, in cell 3:
# Cell 3
def log_activity_2():
while True:
logging.getLogger().warning("Activity 2")
time.sleep(2)
threading.Thread(target=log_activity_2, daemon=True).start()
The output will happen, basically in real time, under the cell that contains the listener.start()
call, thus under cell 1 (and only there) in our case. It will look as expected: For each logged "Activity 2", we will see "Activity 1" logged in alternation and approximately twice as often, as we sleep 2 seconds in the former, and 1 second in the latter:
Once processing has finished, we can stop the QueueListener
(either programmatically or manually) via listener.stop()
– or rather, we should stop the listener this way, following its documentation: if you don’t call [stop()
] before your application exits, there may be some records still left on the queue, which won’t be processed.