Analytics on Kafka Event Streams Using Druid, Elasticsearch, and Rockset

,
Everything you need to get started analyzing Kafka Event Streams

Events are messages that are sent by a system to notify operators or other systems about a change in its domain. With event-driven architectures powered by systems like Apache Kafka becoming more prominent, there are now many applications in the modern software stack that make use of events and messages to operate effectively. In this blog, we will examine the use of three different data backends for event data – Apache Druid, Elasticsearch, and Rockset.

Using Event Data

Events are commonly used by systems in the following ways:

Source de l’article sur DZONE

/par

Introducing Wormhole: Fast Dockerized Presto and Alluxio Setups

,
Just like a real wormhole, this tool is all about speed.

This blog introduces Wormhole, an open-source Dockerized solution for deploying Presto and Alluxio clusters for blazing-fast analytics on file system (we use S3, GCS, OSS). When it comes to analytics, generally people are hands-on in writing SQL queries and love to analyze data that resides in a warehouse (e.g. MySQL database). But as data grows, these stores start failing and there arises a need for getting the faster results in the same or a shorter time frame. This can be solved by distributed computing and Presto is designed for that. When attached to Alluxio, it works even more, faster. That’s what Wormhole is all about.

You may also enjoy:  Alluxio Cluster Setup Using Docker

Here is the high-level architecture diagram of solution:

Source de l’article sur DZONE

/par

Jenkins CI/CD With Git Secrets

,
Connect all your Git secrets to the Jenkins pipeline.

It’s a common practice to encrypt the secrets/credentials we use in our code and then save it in some secure place. We have a number of options to achieve this, with tools like Vault, Git-crypt, and more. However, git-secret is one simple awesome tool that we can use to store our secrets in our Git repo. Git secret uses gpg for encryption and decryption of secrets. 

Here’s how git-secret works. Go to the folder in your repo in which you have files to encrypt. Then, run git init && git secret init. This would initialize your .gitsecret folder. Then you run git secret tell $email , and if you want other users to decrypt the secrets file you have to import their gpg public key and again run git secret tell $otheruseremailid . Now you can run git secret add $secretfilename and git secret hide,which creates $yoursecretfile.secret file, which is an encrypted secret file.

Source de l’article sur DZONE

/par

Big Data File Formats Explained

,

Apache Spark supports many different data formats, such as the ubiquitous CSV format and web-friendly JSON format. Common formats used primarily for big data analytical purposes are Apache Parquet and Apache Avro.

In this post, we’re going to cover the properties of these four formats — CSV, JSON, Parquet, and Avro with Apache Spark.

Source de l’article sur DZONE

/par

Applying CI/CD to Java Apps Using Spring Boot

,

I am very excited to share my experiences building Continuous Integration/Continuous Delivery (CI/CD) into Spring-Boot-based Java applications. First, let’s establish everything we will learn in this tutorial: 

Step 1) Create a Spring Boot Java App using Spring Initializr

Source de l’article sur DZONE

/par

Building Blazor  »Hello, Blinky » IoT Application

,
Build your Blazor IoT app in the blink of an eye!

I thought my first ASP.NET Core edition of Hello, Blinky would be my last, at least for a long time. But then something reminded me of Blazor, and I thought why not build a Blazor edition of Hello, Blinky for Windows IoT Core and Raspberry Pi? After some hacking, I made it work. Here’s my Hello, Blinky for Blazor.

You may also like: Building an ASP.NET Core  »Hello, Blinky » IoT Application

Who the Heck Is Hello, Blinky?

Hello, Blinky is sort of like the "Hello, World" from Raspberry Pi and other microboards. While it’s possible to do a "Hello, World" with these boards, why not do something more interesting with connected electronics? Isn’t that what those boards were made for anyway?

Source de l’article sur DZONE

/par

Cloud, Fog, and Edge Computing: 3 Differences That Matter

,
The three distinct flavors of cloud computing.

Internet of Things (IoT) has transformed the way businesses work, and the industry has seen a massive shift from on-premise software to cloud computing. IoT has sprawled across several industries catering to consumers at a global level.

From smart voice assistants to smart homes, brands are expanding their range of services and experimenting with different ideas to enhance the customer experience. Data on customer behavior is now collected through diverse and innovative ways.

Source de l’article sur DZONE

/par

Distributed Multi-Document ACID Transactions in Couchbase

,

The Couchbase CTO, Ravi Mayuram, announced the Beta of Distributed Multi-document ACID Transactions in Couchbase Server 6.5. I highly recommend reading Ravi’s blog, which highlights how Couchbase transactions are an innovative union of ACID guarantees with scale, high-availability, performance, and flexibility.

In this article, I will dive deeper into our distributed ACID transactions functionality.

Source de l’article sur DZONE

/par

How to Implement Video Recording Using Multi-Camera in Your iOS App (Part 2)

,

In the first part of this tutorial, we have seen how to configure multi-camera for recording a video.

As we know that in the previous iOS version, Apple did not permit to record videos using the front and back cameras at the same time. This time, with the release of iOS 13, they have introduced this feature that allows users to capture video recording using the back and front camera.

Source de l’article sur DZONE

/par

Enterprise Architecture – Building a Robust Business IT Landscape

,
Simple or ornate, the enterprise architecture forms the structure for IT.
“The goal of enterprise architecture is boundary-less information flow where all systems, IT and non-IT, interoperate.” – Allen Brown

Today, when technology has proven its necessity amongst almost all industry segments around the globe, digitalization seems to be having a great influence on enterprise architecture (EA). Businesses are expanding beyond enterprise limits and IT solutions are encompassing enterprise, clients, stakeholder, ecologies and more. At such times, it is tough to manage a traditional monolithic framework. Now is the time to have a process that offers enough space for planning and managing the entire digital wave.

You may also enjoy: Role of Enterprise Architecture in DevOps Adoption

With this concept in mind, around the 1960s began the start of enterprise architecture. Initiated by Professor Dewey Walker and taken forward by his student John Zachmann, enterprise architecture found its entry into the tech world. Somewhere in the 1980s, enterprises realized that they would need a perfect planning approach to match pace with the fast-growing technological web. That gave further impetus to enterprise architecture, to extend beyond mere IT, trying to encompass all important ingredients of the business. The focus area was large organizations who are already in the digitization mode and need to have a seamless integration of legacy apps and processes.

Source de l’article sur DZONE

/par