Photo by Christhian Gruhn on Unsplash

Data flow is a recurrent use case faced by many companies and institutions. There is always the need to handle an incoming source of data, transform it somehow, and send it to another system. These data flows evolve quickly and are sometimes mission-critical.

Monitoring, changing, investigating, and fixing your ETL jobs could be challenging. Mostly if your ETLs are a bunch of scripts triggered via a CRON scheduler or when a new message arrives in your queue messaging system. Most of the time no user interface is available to visualize your Data flows to investigate what’s going on. In that…

Photo by Emily Morter on Unsplash

It’s easy to write asynchronous Python code that does not behave like you think it would. I recently faced a situation where a Sanic web application was crashing/restarting in Kubernetes very often, but without any trace.

Why was it crashing/restarting without any logs? I was clueless. The only possible cause that came to my mind was the readiness/liveness probe that I configured on that application. Then, it would have meant that application was not able to answer health checks — at least not fast enough.

It could also be the main process that was stuck doing something, and not able…

Photo by Trevin Rudy on Unsplash

This article will describe creation of two simple micro-services using Sanic with multiple workers, and Jaeger configuration that will help visualizing flows them.

From Sanic official Documentation :

“ Sanic is a Python 3.6+ web server and web framework that’s written to go fast. It allows the usage of the async/await syntax added in Python 3.5, which makes your code non-blocking and speedy.”

In fact, writing an API with Sanic is as simple as :

from sanic import Sanic
from sanic import response
import random
app = Sanic(__name__)@app.route("/example-api") async def example(request): random_value = random.randint(0, 100000) return response.json(random_value) if __name__…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store