Open in app

Sign In

Write

Sign In

Taufiq Ibrahim
Taufiq Ibrahim

149 Followers

Home

Lists

About

Published in

Dev Genius

·Aug 2, 2022

Realtime Change Data Capture To BigQuery — No Kafka, No Dataflow

A proof-of-concept running CDC from MySQL to BigQuery in realtime with Debezium and Pub/Sub… —

Pub Sub

8 min read

Realtime Change Data Capture To BigQuery — No Kafka, No Dataflow
Realtime Change Data Capture To BigQuery — No Kafka, No Dataflow
Pub Sub

8 min read


Published in

Towards AWS

·May 4, 2021

Serverless Kafka Streaming Application Using AWS MSK + AWS Lambda + Serverless Framework

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed, highly available service that uses Apache Kafka to process real-time streaming data. In August 2020, AWS launched support for Amazon Managed Streaming Kafka as an event source for Amazon Lambda. As a cloud giant, this service will attract…

AWS

7 min read

Serverless Kafka Streaming Application Using AWS MSK + AWS Lambda + Serverless Framework
Serverless Kafka Streaming Application Using AWS MSK + AWS Lambda + Serverless Framework
AWS

7 min read


Mar 12, 2021

Data Pipeline Service Graph Using AWS Neptune Part 1

Background todo Goals todo The Journeys todo 1 Creating Solution Architecture todo 2 Deploying AWS Neptune DB We will deploy a Neptune instance on AWS using Cloudformation below. We also need to have a file called neptune.params.prod.json, which define all parameters required by Cloudformation template. Below is one example. To deploy the stack, let’s use below command: aws cloudformation create-stack \ --stack-name…

AWS

2 min read

Data Pipeline Service Graph Using AWS Neptune Part 1
Data Pipeline Service Graph Using AWS Neptune Part 1
AWS

2 min read


May 16, 2020

Apache Flink: SQL Client Walkthrough

This article shall give you walkthrough in using Apache Flink SQL Client based on my experience on learning and deploy Flink streaming applications. The Mission Supposed that we have a Kafka topic called order_billing which has schema like this: TODO JSON SCHEMA And, this is the example of the records using kafkacat utility. TODO SAMPLE RECORDS We want to have an aggregated view on do_no. The fields need to be aggregated are:

1 min read

1 min read


Mar 17, 2020

Getting Started with Kubernetes

So, what is this all about? Another hype? Another trending tech things? Nope, this is just another simple article to show you on how to get started with Kubernetes and understand some simple operations on it. …

Kubernetes

10 min read

Getting Started with Kubernetes
Getting Started with Kubernetes
Kubernetes

10 min read


Oct 11, 2017

Set Up Continuous Integration Pipelines with Gitlab CI on Ubuntu — Part 1

This series shall guides you for setting up Gitlab CI on Ubuntu. The first part of the series covers how to set up Gitlab CE. Gitlab Runner shall be covered on the second part of this series. Prerequisites The published GitLab hardware requirements recommend using a server with: 2 cores 4GB…

Ubuntu

2 min read

Set Up Continuous Integration Pipelines with Gitlab CI on Ubuntu — Part 1
Set Up Continuous Integration Pipelines with Gitlab CI on Ubuntu — Part 1
Ubuntu

2 min read


Oct 2, 2017

Apache Airflow Installation on Ubuntu

This is the documentation of Apache Airflow installation using Ubuntu on Windows. Preparation pip Apache Airflow requires pip for installation. Run below command. sudo apt-get install python-pip Perhaps if the default installed pip is not the up-to-date version, you may want to consider updating it: sudo pip install --upgrade pip Installing Database Backend (PostgreSQL) sudo apt-get…

Python

4 min read

Apache Airflow Installation on Ubuntu
Apache Airflow Installation on Ubuntu
Python

4 min read


Sep 27, 2017

Installing Apache JMeter on Windows 10

This is the documentation for Apache JMeter installation on Windows 10. Disclaimer: This article was written on September 2017 Latest version of Apache JMeter was 3.3 Java 8 was required My Windows 10 did not have any Java installed Step 1: Install Java 8 Download latest Java 8 from Oracle website. You will need to…

Apache

2 min read

Installing Apache JMeter on Windows 10
Installing Apache JMeter on Windows 10
Apache

2 min read


Sep 1, 2017

Ubuntu On Windows 10

Requirements Your PC must be running a 64-bit version of Windows 10 Anniversary Update or later (build 1607+) Read These Links Before Begin Install the Linux Subsystem on Windows 10 Your PC must be running a 64-bit version of Windows 10 Anniversary Update or later (build 1607+). To find your PC's…msdn.microsoft.com Install the Windows Subsystem for Linux Open PowerShell as Administrator and run: Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Windows 10

2 min read

Ubuntu On Windows 10
Ubuntu On Windows 10
Windows 10

2 min read


Aug 1, 2017

500 Meter Rectangular Grids Using PostgreSQL

Suddenly yesterday afternoon my client asked me to create a report involving 500 meter rectangular grid. Well, that was quite massive, since the data is covering the whole Indonesia. Yes, the whole Indonesia. Indonesia is basically a very large area. …

Sql

3 min read

500 Meter Rectangular Grids Using PostgreSQL
500 Meter Rectangular Grids Using PostgreSQL
Sql

3 min read

Taufiq Ibrahim

Taufiq Ibrahim

149 Followers

Data & Telco

Following
  • Stark Raving

    Stark Raving

  • Pinterest Engineering

    Pinterest Engineering

  • Sarah Stroh

    Sarah Stroh

  • Seth Miller

    Seth Miller

  • Hashmap, an NTT DATA Company

    Hashmap, an NTT DATA Company

See all (59)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech

Teams