کتاب معماری کاربردی لیک‌هاوس

مقدمه‌ای بر معماری لیک‌هاوس

Understanding Data Architecture • What Is Data Architecture? • How Does Data Architecture Help Build a Data Platform? • Core Components of a Data Platform • Why Do We Need a New Data Architecture? • Lakehouse Architecture: A New Pattern • The Lakehouse: Best of Both Worlds • Understanding Lakehouse Architecture • Lakehouse Architecture Characteristics • Lakehouse Architecture Benefits • Key Takeaways • References

معماری‌های سنتی و پلتفرم‌های داده مدرن

Traditional Architectures: Data Lakes and Data Warehouses • Data Warehouse Fundamentals • Data Lake Fundamentals • Modern Data Platforms • Finding Answers in the Cloud • Standalone Approach • Combined Approach • Expectations of Modern Data Platforms • Comparison: Data Warehouse, Data Lake, Lakehouse • Capabilities and Limitations • Implementation Activities • Administration and Management • Business Outcomes • Lakehouse Architecture: The Default Choice for Future Data Platforms? • Key Takeaways • References

ذخیره‌سازی: قلب تپنده لیک‌هاوس

Lakehouse Storage: Key Concepts • Row Versus Columnar Storage • Storage-based Performance Optimization • Lakehouse Storage Components • Cloud Object Storage • File Formats • Table Formats • Key Design Considerations • Ecosystem Support • Community Support • Supported File Formats • Supported Compute Engines • Supported Features • Commercial Product Support • Current and Future Versions • Performance Benchmarking • Comparisons • Sharing Features • Key Takeaways • References

کاتالوگ‌های داده

Understanding Metadata • Technical Metadata • Business Metadata • How Metastores and Data Catalogs Work Together • Features of a Data Catalog • Search, Explore, and Discover Data • Data Classification • Data Governance and Security • Data Lineage • Unified Data Catalog • Challenges of Siloed Metadata Management • What Is a Unified Data Catalog? • Benefits of a Unified Data Catalog • Implementing a Data Catalog: Key Design Considerations and Options • Using Hive metastore • Using AWS Services • Using Azure Services • Using GCP Services • Using Databricks • Key Takeaways • References

موتورهای محاسباتی برای معماری‌های لیک‌هاوس

Data Computation Benefits of Lakehouse Architecture • Independent Scaling • Cross-region, Cross-account Access • Unified Batch and Real-Time Processing • Enhanced BI Performance • Freedom to Choose Different Engine Types • Cross-zone Analysis • Compute Engine Options for Lakehouse Platforms • Open Source Tools • Cloud Services • Third-Party Platforms • Key Design Considerations • Open Table Format Support • Supported Version and Features • Ecosystem Support • Persona-Based Preferences • Managed Open Source Versus Cloud Native Versus Third-Party Products • Data Consumption Workloads • Key Takeaways • References

حاکمیت داده (و هوش مصنوعی) و امنیت در معماری لیک‌هاوس

What Is Data Governance and Data Security? • Benefits of Data Governance and Data Security • Unified Governance and Security in Lakehouse Architecture • Governance and Security Processes in Lakehouse Architecture • Metadata Management • Compliance and Regulations • Data and ML Model Quality • Lineage Across Data and AI assets • Data and AI Asset Sharing • Data Ownership • Auditing and Monitoring • Access Management • Data Protection • Handling Sensitive Data • What’s Your Role? • Key Takeaways • References

تصویر بزرگ: طراحی و پیاده‌سازی یک پلتفرم لیک‌هاوس

Pre-design Activities • Understanding Platform Requirements • Studying Existing System • Understanding the Organization’s Vision and Data Strategy • Conducting Workshops and Interviews • Choosing the Right Architecture • Establishing Guiding Principles • Data Ecosystem • Scalability and Performance • Cost Control and Optimization • Platform Operations • Governance and Security • Design Considerations and Implementation Best Practices • Architecture Blueprint • Data Ingestion • Data Storage • Data Processing • Data Consumption and Delivery • Common Services • Design References • Step-by-Step Design Guide • Design Questionnaire • Key Takeaways • References

لیک‌هاوس در دنیای واقعی

Delivering a Real-World Lakehouse • Estimation and Planning Phase • Estimation • Planning • Analysis and Design Phase • Analyzing the Existing System • Data Modeling • Finalizing the Tech Stack • Implementation and Test Phase • Historical Data Migration • Data Reconciliation and Testing • Reverse Engineering • Data Quality and Handling Sensitive Data • Support and Maintenance Phase • Auditing and Tracking • Disaster Recovery Strategy • Decommissioning the Old System • Delivery References • Project Deliverables • Reference Architectures • Key Takeaways • References

لیک‌هاوس آینده

Warehouse to Lakehouse: What’s Next? • Data Mesh • HTAP • Zero ETL • Interoperability and New Formats • Universal Format (UniForm) • Apache XTable • Upcoming File and Table Formats • Managed Platforms for Public and Private Clouds • Microsoft Fabric and Other Platforms • Managed Lakehouse for Private Cloud Platform • AI in a Lakehouse • Key Takeaways • Book Conclusion • References