Open in app

Sign In

Write

Sign In

Jay
Jay

231 Followers

Home

Lists

About

Published in

AWS in Plain English

·Pinned

MWAA: Apache Airflow on AWS

Stop Worrying, Start DAG’ing — Airflow is an amazing platform to programmatically author, schedule, and monitor workflows. Check out MWAA: Apache Airflow on AWS Part 1 to get an overview of what Airflow is and how we can use it to automate our data pipelines and workflows. Nothing makes sense until you start implementing, so…

AWS

6 min read

Stop Worrying, Start DAG’ing. MWAA: Apache Airflow on AWS par2.
Stop Worrying, Start DAG’ing. MWAA: Apache Airflow on AWS par2.
AWS

6 min read


Published in

SelectFrom

·Pinned

Apache Spark: All about Serialization

Overview of How You Can Tune Your Spark Jobs to Improve Performance In distributed systems, data transfer over the network is the most common task. If this is not handled efficiently, you may end up facing numerous problems, like high memory usage, network bottlenecks, and performance issues. Serialization plays an…

Apache Spark

3 min read

Apache Spark: All about Serialization
Apache Spark: All about Serialization
Apache Spark

3 min read


Pinned

Introduction to Akka Typed with Scala for beginners.

Learn to Build distributed and fault tolerant systems with me and Dead-pool. Learn best in class technology upon which many cutting edge frameworks are build(Play Framework, lagom,cloudFlow, etc…) the list goes on… I would like to share my insights on how to get there.when i started my journey to learn…

Scala

8 min read

Akka Typed with Scala for beginners.
Akka Typed with Scala for beginners.
Scala

8 min read


Sep 8

Kaggle-Databricks-Snowflake: End-to-End Hands-on ETL demo

Effortless Data Integration: Kaggle, Databricks, and Snowflake in Action There has been a notable shift in focus, primarily towards the collection and storage of data, due to advancements in artificial intelligence and machine learning. The goal is to gather as many data points as possible. But it’s crucial to understand…

Data

6 min read

Kaggle-Databricks-Snowflake: End-to-End Hands-on ETL demo
Kaggle-Databricks-Snowflake: End-to-End Hands-on ETL demo
Data

6 min read


Published in

ITNEXT

·Sep 7

Security at Scale: Sensitive Data Supervision with Regulatory Confidence

Data — a dynamic entity that orchestrates our current course — navigates us through our present and opens up endless future possibilities. It is at the epicenter of all our endeavors, making data security essential for reliability and trustworthiness. …

Data

5 min read

Security at Scale: Sensitive Data Supervision with Regulatory Confidence
Security at Scale: Sensitive Data Supervision with Regulatory Confidence
Data

5 min read


Published in

ITNEXT

·Aug 7

Mitigating CVE Shock, AKA Navigating the Perils of Vulnerability Hell

DevOps teams customarily abide by a typical pattern concerning overall security and vulnerabilities: they overburden themselves with unwanted vulnerability intricacies. Commonly referred to as “CVE shock”, this is a phase in which DevOps teams get distracted and feel helpless when exposed to an overwhelming list of vulnerabilities to mitigate and…

DevOps

5 min read

Mitigating CVE Shock, AKA Navigating the Perils of Vulnerability Hell
Mitigating CVE Shock, AKA Navigating the Perils of Vulnerability Hell
DevOps

5 min read


Jun 21

Building a custom WhatsApp AI chatbot with ChatGPT and Selenium.

Deep dive into ChatAgent development and Prompt Engineering. Introduction: Today we are going to see how we can integrate and automate chatCPT on WhatsApp and convert it into an AI chatbot. It sometimes feels overwhelming to connect to OpenAI or dall-E to generate responses or images right. I felt it too…

ChatGPT

2 min read

Building a custom WhatsApp AI chatbot with ChatGPT and Selenium.
Building a custom WhatsApp AI chatbot with ChatGPT and Selenium.
ChatGPT

2 min read


Apr 30

Dynamic data load to AWS S3 with Snowpark.

The data war between Databricks and Snowflake are inevitable and are rushing the platforms to adapt and onboard new features at an alarming pace. They both have their pros and cons. The advantages are appealing only based on your role and the tasks you want to complete from the platform. …

Snowflake

4 min read

Dynamic data load to AWS S3 with Snowpark.
Dynamic data load to AWS S3 with Snowpark.
Snowflake

4 min read


Published in

AWS in Plain English

·Apr 5

Develop and Invoke AWS Lambda Functions programmatically.

Introduction: AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers and you can pay only for the compute when the lambda is invoked. This sounds convenient, but the pricing differs when we opt for general-purpose and on-demand computing. When you have a…

AWS

4 min read

Develop and Invoke AWS Lambda Functions programmatically.
Develop and Invoke AWS Lambda Functions programmatically.
AWS

4 min read


Published in

DataTrek

·Feb 9

Mastering Tools and Libraries are Trivial.

Choose wisely, learn mindfully, implement efficiently, and grow immensely. Today, everybody wants to solve a problem, be it technical or otherwise, and tools and libraries are emerging at an alarming pace like never before. We live in an era where there is no room for the word impossible. …

Programming

3 min read

Mastering Tools and Libraries are Trivial.
Mastering Tools and Libraries are Trivial.
Programming

3 min read

Jay

Jay

231 Followers

Databricks platform lead. MLOps and DataOps. databracket.substack.com youtube.com/@data_bracket

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech

Teams