کتاب معماری پلتفرم‌های داده و یادگیری ماشین

مدرن‌سازی پلتفرم داده: مروری مقدماتی

The Data Lifecycle • The Journey to Wisdom • Water Pipes Analogy • Collect • Store • Process/Transform • Analyze/Visualize • Activate • Limitations of Traditional Approaches • Antipattern: Breaking Down Silos Through ETL • Antipattern: Centralization of Control • Antipattern: Data Marts and Hadoop • Creating a Unified Analytics Platform • Cloud Instead of On-Premises • Drawbacks of Data Marts and Data Lakes • Convergence of DWHs and Data Lakes • Hybrid Cloud • Reasons Why Hybrid Is Necessary • Challenges of Hybrid Cloud • Why Hybrid Can Work • Edge Computing • Applying AI • Machine Learning • Uses of ML • Why Cloud for AI? • Cloud Infrastructure • Democratization • Real Time • MLOps • Core Principles • Summary

گام‌های استراتژیک برای نوآوری با داده

Step 1: Strategy and Planning • Strategic Goals • Identify Stakeholders • Change Management • Step 2: Reduce Total Cost of Ownership by Adopting a Cloud Approach • Why Cloud Costs Less • How Much Are the Savings? • When Does Cloud Help? • Step 3: Break Down Silos • Unifying Data Access • Choosing Storage • Semantic Layer • Step 4: Make Decisions in Context Faster • Batch to Stream • Contextual Information • Cost Management • Step 5: Leapfrog with Packaged AI Solutions • Predictive Analytics • Understanding and Generating Unstructured Data • Personalization • Packaged Solutions • Step 6: Operationalize AI-Driven Workflows • Identifying the Right Balance of Automation and Assistance • Building a Data Culture • Populating Your Data Science Team • Step 7: Product Management for Data • Applying Product Management Principles to Data • 1. Understand and Maintain a Map of Data Flows in the Enterprise • 2. Identify Key Metrics • 3. Agreed Criteria, Committed Roadmap, and Visionary Backlog • 4. Build for the Customers You Have • 5. Don’t Shift the Burden of Change Management • 6. Interview Customers to Discover Their Data Needs • 7. Whiteboard and Prototype Extensively • 8. Build Only What Will Be Used Immediately • 9. Standardize Common Entities and KPIs • 10. Provide Self-Service Capabilities in Your Data Platform • Summary

طراحی برای تیم داده

Classifying Data Processing Organizations • Data Analysis–Driven Organization • The Vision • The Personas • The Technological Framework • Data Engineering–Driven Organization • The Vision • The Personas • The Technological Framework • Data Science–Driven Organization • The Vision • The Personas • The Technological Framework • Summary

چارچوب مهاجرت

Modernize Data Workflows • Holistic View • Modernize Workflows • Transform the Workflow Itself • A Four-Step Migration Framework • Prepare and Discover • Assess and Plan • Execute • Optimize • Estimating the Overall Cost of the Solution • Audit of the Existing Infrastructure • Request for Information/Proposal and Quotation • Proof of Concept/Minimum Viable Product • Setting Up Security and Data Governance • Framework • Artifacts • Governance over the Life of the Data • Schema, Pipeline, and Data Migration • Schema Migration • Pipeline Migration • Data Migration • Migration Stages • Summary

معماری دریاچه داده

Data Lake and the Cloud—A Perfect Marriage • Challenges with On-Premises Data Lakes • Benefits of Cloud Data Lakes • Design and Implementation • Batch and Stream • Data Catalog • Hadoop Landscape • Cloud Data Lake Reference Architecture • Integrating the Data Lake: The Real Superpower • APIs to Extend the Lake • The Evolution of Data Lake with Apache Iceberg, Apache Hudi, and Delta Lake • Interactive Analytics with Notebooks • Democratizing Data Processing and Reporting • Build Trust in the Data • Data Ingestion Is Still an IT Matter • ML in the Data Lake • Training on Raw Data • Predicting in the Data Lake • Summary

نوآوری با انبار داده سازمانی

A Modern Data Platform • Organizational Goals • Technological Challenges • Technology Trends and Tools • Hub-and-Spoke Architecture • Data Ingest • Business Intelligence • Transformations • Organizational Structure • DWH to Enable Data Scientists • Query Interface • Storage API • ML Without Moving Your Data • Summary

همگرایی به سمت لیک‌هاوس

The Need for a Unique Architecture • User Personas • Antipattern: Disconnected Systems • Antipattern: Duplicated Data • Converged Architecture • Two Forms • Lakehouse on Cloud Storage • SQL-First Lakehouse • The Benefits of Convergence • Summary

معماری‌های جریان داده

The Value of Streaming • Industry Use Cases • Streaming Use Cases • Streaming Ingest • Streaming ETL • Streaming ELT • Streaming Insert • Streaming from Edge Devices (IoT) • Streaming Sinks • Real-Time Dashboards • Live Querying • Materialize Some Views • Stream Analytics • Time-Series Analytics • Clickstream Analytics • Anomaly Detection • Resilient Streaming • Continuous Intelligence Through ML • Training Model on Streaming Data • Streaming ML Inference • Automated Actions • Summary

توسعه پلتفرم داده با استفاده از محیط‌های ترکیبی و لبه

Why Multicloud? • A Single Cloud Is Simpler and Cost-Effective • Multicloud Is Inevitable • Multicloud Could Be Strategic • Multicloud Architectural Patterns • Single Pane of Glass • Write Once, Run Anywhere • Bursting from On Premises to Cloud • Pass-Through from On Premises to Cloud • Data Integration Through Streaming • Adopting Multicloud • Framework • Time Scale • Define a Target Multicloud Architecture • Why Edge Computing? • Bandwidth, Latency, and Patchy Connectivity • Use Cases • Benefits • Challenges • Edge Computing Architectural Patterns • Smart Devices • Smart Gateways • ML Activation • Adopting Edge Computing • The Initial Context • The Project • The Final Outcomes and Next Steps • Summary

معماری کاربردی هوش مصنوعی

Is This an AI/ML Problem? • Subfields of AI • Generative AI • Problems Fit for ML • Buy, Adapt, or Build? • Data Considerations • When to Buy • What Can You Buy? • How Adapting Works • AI Architectures • Understanding Unstructured Data • Generating Unstructured Data • Predicting Outcomes • Forecasting Values • Anomaly Detection • Personalization • Automation • Responsible AI • AI Principles • ML Fairness • Explainability • Summary

معماری پلتفرم یادگیری ماشین

ML Activities • Developing ML Models • Labeling Environment • Development Environment • User Environment • Preparing Data • Training ML Models • Deploying ML Models • Deploying to an Endpoint • Evaluate Model • Hybrid and Multicloud • Training-Serving Skew • Automation • Automate Training and Deployment • Orchestration with Pipelines • Continuous Evaluation and Training • Choosing the ML Framework • Team Skills • Task Considerations • User-Centric • Summary

مدرن‌سازی پلتفرم داده: یک مورد مطالعاتی

New Technology for a New Era • The Need for Change • It Is Not Only a Matter of Technology • The Beginning of the Journey • The Current Environment • The Target Environment • The PoC Use Case • The RFP Responses Proposed by Cloud Vendors • The Target Environment • The Approach on Migration • The RFP Evaluation Process • The Scope of the PoC • The Execution of the PoC • The Final Decision • Peroration • Summary