Druid

How to Configure Druid to Use Minio as Deep Storage

June 20, 2018 Monzurul Haque Shimul

Druid relies on a distributed filesystem or binary object store for data storage. The most commonly used deep storage implementations are S3 (popular for those on AWS) and HDFS (popular if you already have a Hadoop deployment). In this post, I will show you how to configure non-Amazon S3 deep storage for druid cluster. And for this, I will use Minio as S3 deep storage for druid cluster.

Minio is a high performance distributed object storage server, designed for large-scale private cloud infrastructure. Amazon S3 API is the de facto standard for object storage. Minio implements Amazon S3 v2/v4 API. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.

Setting Up A Horizontally Scalable, Fault-Tolerant Druid Cluster

June 19, 2018 Monzurul Haque Shimul

Druid is designed to be deployed as a horizontally scalable, fault-tolerant cluster. In this post, we’ll set up a simple cluster and discuss how it can be further configured to meet your needs. This simple cluster will feature scalable, fault-tolerant Data servers for ingesting and storing data, a single Query server, and a single Master server. Later, we’ll discuss how this simple cluster can be configured for high availability and to scale out all server types. We will use imply druid distribution.

Running Imply Druid Distribution inside Docker Container

June 18, 2018 Monzurul Haque Shimul

Druid is an open-source data store designed for sub-second queries on real-time and historical data. Druid can scale to store trillion of events and ingest millions of events per second. Druid is best used to power user-facing data applications.

Imply is an analytics solution powered by druid. The Imply Analytics platform includes Druid bundled with all its dependencies, an exploratory analytics UI, and a SQL layer. It also provides additional tools and scripts for easy management of druid nodes.

Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud.

In this post, I will demonstrate how to install and run Imply Druid distribution inside a Docker container.

Druid quickstart using Imply distribution

June 18, 2018 Monzurul Haque Shimul

Druid is an open-source data store designed for sub-second queries on real-time and historical data. It is primarily used for business intelligence/OLAP queries on event data. Druid enables arbitrary data exploration, low latency data ingestion, and fast aggregations at scale. Druid can scale to store trillion of events and ingest millions of events per second. Druid is best used to power user-facing data applications.
In this post, we will download Druid distribution from Imply, set it up on a single machine, load some data, and query and visualize the data.