Abhishek Soni

Spark query performance on Databricks is driven by a multi-layer optimization stack: Catalyst transforms SQL into optimized execution plans.

Build an AI agent that processes real-time events with Amazon Bedrock and a serverless AWS architecture powered by Kinesis, DynamoDB, and S3.

A practical guide to feature engineering at scale with Azure Databricks, covering distributed data processing with Spark and reliable storage with Delta Lake.

This article walks through building a modern Databricks pipeline using the Medallion Architecture, explains Delta Lake's transaction log and ACID guarantees.

Azure Databricks and Microsoft Fabric overlap, but they're built for different priorities. Databricks for data engineering, ML, open-source, and Spark workloads.

This article walks through building a production-ready multi-agent AI system using Microsoft's AI stack, focusing on the operational capabilities.

Secure Microsoft Foundry agents with zero-trust networking, managed identities, RBAC, private networking, and least-privilege access for enterprise AI.

Discover architectural patterns for integrating AI into enterprise systems without sacrificing governance, reliability, observability, or loose coupling.

Learn key concepts like embeddings, vector search, chunking, hybrid search, semantic ranking, and how these pieces fit into a scalable, enterprise-grade AI application.

Learn Spark performance tuning on Databricks with shuffle optimization, skew handling, AQE, broadcast joins, and Delta Lake Z-Ordering best practices.