Note: Make sure to generate the SSH key using RSA algorithm. Apache Airflow uses a git-sync container to keep its collection of DAGs in synch with the content of the Git Repository and the SSH key is used to authenticate.
How to develop a component catalog connector.Enabling Elyra pipelines in an existing Apache Airflow deployment.Deploying Airflow on a new Kubernetes cluster.Configuring Apache Airflow on Kubernetes for use with Elyra.Visualizing output from your notebooks or Python scripts in the Kubeflow Pipelines UI.Creating a custom runtime container image.
Running Elyra in an air-gapped environment.Why Airflow on Kubernetes Since its inception, Airflow's greatest strength has been its flexibility. You can define dependencies, programmatically construct complex workflows, and monitor scheduled jobs in an easy to read UI. Using Elyra with the Kubeflow Notebook Server Airflow allows users to launch multi-step pipelines using a simple Python object DAG (Directed Acyclic Graph).Deploying Kubeflow Pipelines Locally for Elyra.Deploying Elyra & JupyterHub in a Kubernetes environment.Best practices for file-based pipeline nodes.
Requirements and best practices for custom pipeline components.Articles, blog posts, and other resources.