Project overview

At the core of the Stackable Platform are the Kubernetes operators used to install and manage various data products, like the nifi-operator for example. You can find all of operators if you search the organization repositories.

All the operators are written in Rust and the source code is found in the rust directory. tests contains the integration tests which use kuttl and our own test template beku.py. Documentation is written in Antora and found in the docs directory, see also Documentation further down the page. deploy and docker contain files used to package the operator into a Docker image and Helm chart. Some files in these repositories are actually templated:

The operator-templating repository contains a template for all operator repositories, where shared files are distributed from. You can read the README in that repository to find out more about the details. Whenever common files are changed, a GitHub action is used to distribute the changes to all operator repositories.

The operator-rs repository contains the common framework library for all operators. It is a Rust library that is used by all operators and contains shared structs and shared functionality.

The docker-images repository contains Dockerfiles for all the products that are supported by the SDP. The actual product artifacts are pulled from the product artifacs store and packaged into images. The images are pushed into an image registry.

The stackablectl commandline tool and the Stackable Cockpit UI are both found in the stackable-cockpit repository, and they both share some code. The structure of the repository is documented in its README.

The documentation is built with Antora and the playbook file to build it is located in the documentation repository, among some common platform documentation. The UI for the documentation is found in the documentation-ui repository; it is included as a submodule in the documentation repository. The documentation pulls in operator documentation files from the operator repositories. The documentation is found at https://docs.stackable.tech/.

There is also https://crds.stackable.tech/ where you can find generated documentation for all the CustomResourceDefinitions on the platform. The code to generate this page is found in the crddocs repository.

On top of the mentioned repositories, there are various smaller tools or product extensions that Stackable developed, they are all linked to from relevant places. Two examples are the image-tools used to build Docker images and the druid-opa-authorizer which is a Druid extensions that enables OPA support for Druid.

T2 - Test & Troubleshoot Platform is used for integration testing across different versions and cloud providers, find more information in the README of the repository.

The issues repository exists solely for the purpose of tracking issues related to the Stackable Platform in general. Large topics that impact many or even all of the platform components are discussed here. There is no code in this repository.

A lot of artifacts are stored in the Nexus repo. Under packages you find product binaries, the different helm-* stores contain the Helm Charts and Docker images are stored there too.

Docker images are stored in Nexus as mentioned above, but also in https://oci.stackable.tech - which is where we want to move to in the future.