All Posts

  • Pipenv Cheat Sheet. Pipenv is a packaging tool for Python that solves some common problems associated with the typical workflow using pip, virtualenv, and the good old requirements.txt. It automatically creates and manages a virtual environment for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever-important Pipfile.lock, which is used to produce deterministic builds.
  • Top 10 Python Libraries for Data Engineering. Data science is rapidly growing and providing immense opportunities for organizations to leverage data insights for strategic decision-making. Python is gaining popularity as the programming language of choice for data science projects. One of the primary reasons for this trend is the availability of various Python libraries that offer efficient solutions for data science tasks. In this article, we will discuss the top 10 Python libraries for data science.
  • Airflow is a platform to programmatically author, schedule and monitor workflows. Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.