Resources

🔒 Security

Coarse-grained Data Access Control

• Secure access to S3 buckets using instance profiles | Databricks on AWS

Fine-grained Data Access Control

• SAML 2.0 Federation credential passthrough
• Access S3 buckets using IAM credential passthrough with Databricks SCIM | Databricks on AWS
• Manage cluster policies | Databricks on AWS
• Restrict access to PII Data - Encrypt columns

🔖 Documentation

• Auto Loader | Databricks on AWS
• Optimize performance with file management | Databricks on AWS

📇 Other Resources

• Databricks Academy - Home - Databricks
• Delta Live Tables Demo - Databricks
• How to Create an R&D Lakehouse Based on FAIR Data Principles
• Security Best Practices for Databricks on AWS
• databricks/security-bucket-brigade
• MrPowers/spark-daria
• databricks/tech-talks
• YouTube | Databricks
• Online User Community
• Medallion Architecture
• Query Watchdog: Handling Disruptive Queries in Spark SQL
• Best Practices - Videos
• Disaster Recovery Overview
• Attack of the Delta Clones - Against Disaster Recovery Availability Complexity
• Using Deep Clone Disaster Recovery Delta Lake
• Overwatch - Audit Logs
• DeltaOMS - Delta operational metrics store
• Audit Logs
• Auto-Loader Schema Inference
• Databricks Terraform Docs

Videos

• Vimeo | Databricks
• Introduction to Instance Pools

Training Notebooks

• Data Engineering with Databricks
• Apache Spark Programming with Databricks
• Data Analysis with Databricks
• Advanced Data Engineering with Databricks
• ML In Production

Self-Paced Courses

• Data Engineering with Databricks
• Data Engineering Learning Plan
• Partner Learning Catalog

Unsorted

• Slides - Welcome to Databricks Lakehouse
• Slides - Data Engineering Technical Overview
• Bootcamp Github