优秀的可伸缩性
这是一份经过精心整理的阅读清单，它详细说明了在大型系统中如何保障系统的可伸缩性，可靠性以及高性能的常见模式。 该阅读清单由知名工程文章及来源可靠的资源组成。 这些内容对可伸缩性等相关概念均有很好的阐述。清单中研究的案例也 都来自于服务于数百万，乃至数十亿用户且经过实战检验过的系统。
内容
原则
- 巨型服务的经验教训--加州大学伯克利分校和谷歌公司的Eric Brewer
- 构建大型分布式系统的设计、经验和建议 - Jeff Dean, Google
- 如何设计一个好的API以及它的重要性--Joshua Bloch，CMU和Google。
- 关于效率、可靠性、扩展--James Hamilton，AWS公司副总裁
- 构建企业平台时应注意的事项 - Heidi Williams，Box公司平台副总裁
- 混沌工程原理
- 在混沌中寻找秩序
- 十二要素应用
- 清晰架构
- 高内聚、低耦合
- 单体和微服务
- CAP定理和权衡
- CP 数据库 和 AP 数据库
- 无状态与有状态的可伸缩性
- 垂直伸缩和水平伸缩
- 垂直伸缩对比水平伸缩: 隐形成本
- ACID 与 BASE
- 阻塞/非阻塞 和 同步/异步
- 数据库的性能和可伸缩性
- 数据库隔离级别及对性能和可扩展性的影响
- 大型集群中数据丢失的概率分析
- 高可扩展解决方案的数据访问。使用SQL、NoSQL和Polyglot持久性技术
- SQL vs NoSQL
- SQL vs NoSQL - Salesforce的经验教训
- NoSQL 数据库: 调查和决策指南
- 分片是如何工作的
- 一致性哈希
- 一致性哈希：算法权衡
- 不要被哈希技巧所欺骗
- 在Netflix的统一一致的哈希
- 最终一致性 - Werner Vogels,Amazon公司CTO
- 缓存为王
- 反缓存
- 理解延迟
- 每一个程序员都应该知道的延迟数字
- 服务可用性微积分
- 扩展Web应用时的架构问题。瓶颈、数据库、CPU、IO
- 常见瓶颈
- 分布式交易背后的秘密
- 依靠软件在不同层级可靠地重定向流量
- 有目的的打破固有事物
- 避免过度工程
- 可伸缩性最差实践
- 使用稳定的技术--不要重新发明轮子--保持简单!
- Simplicity by Distributing Complexity
- 为什么过度重用是坏事
- 性能是一个特性
- 让性能成为工作流程的一部分
- 服务器端渲染比客户端渲染的优点
- 自动化和抽象：Facebook的教训
- AWS的做与不做
- UI设计不可伸缩性 - Stanley Wood, Design Director at Spotify
- Linux 性能
- 构建快速而有弹性的Web应用 - Ilya Grigorik
- 接受部分故障，将服务损失降到最低限度
- 弹性设计
- 自我修复设计
- 水平伸缩设计
- 演化设计
- 从错误中学习
可伸缩性
- 微服务和服务编排
- Uber公司面向领域的微服务架构
- Riot Games的容器 (8部分)实践
- 容器化在Pinterest的应用
- 容器在Netflix使用的演变
- 在Uber实现MySQL的Docker化
- Spotify的微服务测试
- Treehouse的在生产中Docker实践
- SoundCloud的微服务
- 在Stripe可靠地运行Kubernetes
- 在Trivago使用Istio进行跨集群流量镜像
- 英国广播公司的Nanoservices
- PowerfulSeal: 彭博社Kubernetes集群的测试工具
- Conductor: Netflix的微服务编排
- Docker容器为Shopify的100.000多个在线商店提供支撑
- Medium公司的微服务架构
- From bare-metal to Kubernetes at Betabrand
- Tinder公司的K8s实践
- Quora公司的K8s实践
- Pinterest公司的K8s平台
- Nubank的微服务实践
- GRIT: eBay跨微服务的分布式协议
- Rubix: Palantir的K8s实践
- 分布式缓存
- EVCache: Netflix的分布式内存缓存技术
- EVCache Cache Warmer基础架构在Netflix的应用
- Memsniff: Box公司强大的Memcache流量分析器
- Caching with Consistent Hashing and Cache Smearing at Etsy
- Facebook的图片缓存分析
- 在Facebook的缓存效率练习
- tCache: Scalable Data-aware Java Caching at Trivago
- Pycache: Quora的进程内缓存
- 在Trivago将Memcached内存使用量减少50%
- 缓存内部服务,Yelp公司
- 在Allegro使用大数据估算缓存效率
- Zalando的分布式缓存
- NetFlix公司从RAM到SSD的应用数据缓存化
- Skyscanner重复缓存的权衡
- Avoiding Cache Stampede at DoorDash
- 在Yext使用Quadtrees进行位置缓存
- Vimeo的视频元数据缓存功能
- 在Twitter上扩展Redis
- 在Slack上使用Redis扩展作业队列
- 在Github上将持久性数据从Redis中移出
- 在Instagram的Redis中存储数以亿计的简单键值对
- Trivago公司的Redis实践
- 优化Deliveroo的Redis存储
- Redis中的内存优化在Wattpad上的应用
- Redis Fleet at Heroku
- 在SoundCloud解决远程构建缓存缺失的问题（2部分）
- Prefetch Caching of Items at eBay
- HTTP缓存和CDN
- Zynga Geo Proxy: Reducing Mobile Game Latency at Zynga
- Google AMP at Condé Nast
- A/B Tests on Hosting Infrastructure (CDNs) at Deliveroo
- HAProxy with Kubernetes for User-facing Traffic at SoundCloud
- Bandaid: Service Proxy at Dropbox
- CDN in LIVE's Encoder Layer at LINE
- Service Workers at Slack
- CDN Services at Spotify
- 分布式锁
- 分布式跟踪，追踪，测量
- Zipkin: Distributed Systems Tracing at Twitter
- Improve Zipkin Traces using Kubernetes Pod Metadata at SoundCloud
- Canopy: Scalable Distributed Tracing & Analysis at Facebook
- Pintrace: Distributed Tracing at Pinterest
- XCMetrics: All-in-One Tool for Tracking Xcode Build Metrics at Spotify
- Real-time Distributed Tracing at LinkedIn
- Tracking Service Infrastructure at Scale at Shopify
- Distributed Tracing at HelloFresh
- Analyzing Distributed Trace Data at Pinterest
- Distributed Tracing at Uber
- JVM Profiler: Tracing Distributed JVM Applications at Uber
- Data Checking at Dropbox
- Tracing Distributed Systems at Showmax
- osquery Across the Enterprise at Palantir
- StatsD at Etsy
- StatsD at DoorDash
- 分布式调度
- Distributed Task Scheduling (3 parts) at PagerDuty
- Building Cron at Google
- Distributed Cron Architecture at Quora
- Chronos: A Replacement for Cron at Airbnb
- Scheduler at Nextdoor
- Peloton: Unified Resource Scheduler for Diverse Cluster Workloads at Uber
- Fenzo: OSS Scheduler for Apache Mesos Frameworks at Netflix
- Airflow - Workflow Orchestration
- 分布式监控和警告
- Unicorn: Remediation System at eBay
- M3: Metrics and Monitoring Platform at Uber
- Athena: Automated Build Health Management System at Dropbox
- Vortex: Monitoring Server Applications at Dropbox
- Nuage: Cloud Management Service at LinkedIn
- Telltale: Application Monitoring at Netflix
- ThirdEye: Monitoring Platform at LinkedIn
- Periskop: Exception Monitoring Service at SoundCloud
- Securitybot: Distributed Alerting Bot at Dropbox
- Monitoring System at Alibaba
- Real User Monitoring at Dailymotion
- Alerting Ecosystem at Uber
- Alerting Framework at Airbnb
- Alerting on Service-Level Objectives (SLOs) at SoundCloud
- Job-based Forecasting Workflow for Observability Anomaly Detection at Uber
- Monitoring and Alert System using Graphite and Cabot at HackerEarth
- Observability (2 parts) at Twitter
- Distributed Security Alerting at Slack
- Real-Time News Alerting at Bloomberg
- Data Pipeline Monitoring System at LinkedIn
- Monitoring and Observability at Picnic
- 分布式安全
- Approach to Security at Scale at Dropbox
- Aardvark and Repokid: AWS Least Privilege for Distributed, High-Velocity Development at Netflix
- LISA: Distributed Firewall at LinkedIn
- Secure Infrastructure To Store Bitcoin In The Cloud at Coinbase
- BinaryAlert: Real-time Serverless Malware Detection at Airbnb
- Scalable IAM Architecture to Secure Access to 100 AWS Accounts at Segment
- OAuth Audit Toolbox at Indeed
- Active Directory Password Blacklisting at Yelp
- Syscall Auditing at Scale at Slack
- Athenz: Fine-Grained, Role-Based Access Control at Yahoo
- WebAuthn Support for Secure Sign In at Dropbox
- Security Development Lifecycle at Slack
- Unprivileged Container Builds at Kinvolk
- Diffy: Differencing Engine for Digital Forensics in the Cloud at Netflix
- Detecting Credential Compromise in AWS at Netflix
- Scalable User Privacy at Spotify
- AVA: Audit Web Applications at Indeed
- TTL as a Service: Automatic Revocation of Stale Privileges at Yelp
- Enterprise Key Management at Slack
- Scalability and Authentication at Twitch
- 分布式消息，队列，事件流
- Cape: Event Stream Processing Framework at Dropbox
- Brooklin: Distributed Service for Near Real-Time Data Streaming at LinkedIn
- Samza: Stream Processing System for Latency Insighs at LinkedIn
- Bullet: Forward-Looking Query Engine for Streaming Data at Yahoo
- EventHorizon: Tool for Watching Events Streaming at Etsy
- Qmessage: Distributed, Asynchronous Task Queue at Quora
- Cherami: Message Queue System for Transporting Async Tasks at Uber
- Dynein: Distributed Delayed Job Queueing System at Airbnb
- Messaging Service at Riot Games
- Debugging Production with Event Logging at Zillow
- Cross-platform In-app Messaging Orchestration Service at Netflix
- Video Gatekeeper at Netflix
- Scaling Push Messaging for Millions of Devices at Netflix
- Delaying Asynchronous Message Processing with RabbitMQ at Indeed
- Benchmarking Streaming Computation Engines at Yahoo
- Improving Stream Data Quality With Protobuf Schema Validation at Deliveroo
- Scaling Email Infrastructure at Medium
- Event Stream Database at Nike
- Event-Driven Messaging
- Pub-Sub Messaging
- Kafka - Message Broker
- Kafka at LinkedIn
- Kafka at Pinterest
- Kafka at Trello
- Kafka at Salesforce
- Kafka at The New York Times
- Kafka at Yelp
- Kafka at Criteo
- Kafka on Kubernetes at Shopify
- Migrating Kafka's Zookeeper with No Downtime at Yelp
- Reprocessing and Dead Letter Queues with Kafka at Uber
- Chaperone: Audit Kafka End-to-End at Uber
- Finding Kafka throughput limit in infrastructure at Dropbox
- Cost Orchestration at Walmart
- InfluxDB and Kafka to Scale to Over 1 Million Metrics a Second at Hulu
- Stream Data Deduplication
- 分布式日志
- Logging at LinkedIn
- Scalable and Reliable Log Ingestion at Pinterest
- High-performance Replicated Log Service at Twitter
- Logging Service with Spark at CERN Accelerator
- Logging and Aggregation at Quora
- Collection and Analysis of Daemon Logs at Badoo
- Log Parsing with Static Code Analysis at Palantir
- Centralized Application Logging at eBay
- Enrich VPC Flow Logs at Hyper Scale to provide Network Insight at Netflix
- BookKeeper: Distributed Log Storage at Yahoo
- LogDevice: Distributed Data Store for Logs at Facebook
- LogFeeder: Log Collection System at Yelp
- DBLog: Generic Change-Data-Capture Framework at Netflix
- 分布式搜索
- Search Architecture at Instagram
- Search Architecture at eBay
- Search Architecture at Box
- Search Discovery Indexing Platform at Coupang
- Universal Search System at Pinterest
- Improving Search Engine Efficiency by over 25% at eBay
- Indexing and Querying Telemetry Logs with Lucene at Palantir
- Query Understanding at TripAdvisor
- Search Federation Architecture at LinkedIn (2018)
- Search at Slack
- Search and Recommendations at DoorDash
- Search Service at Twitter (2014)
- Autocomplete Search (2 parts) at Traveloka
- Data-Driven Autocorrection System at Canva
- Adapting Search to Indian Phonetics at Flipkart
- Nautilus: Search Engine at Dropbox
- Galene: Search Architecture of LinkedIn
- Manas: High Performing Customized Search System at Pinterest
- Sherlock: Near Real Time Search Indexing at Flipkart
- Nebula: Storage Platform to Build Search Backends at Airbnb
- ELK (Elasticsearch, Logstash, Kibana) Stack
- Predictions in Real Time with ELK at Uber
- Building a scalable ELK stack at Envato
- ELK at Robinhood
- Scaling Elasticsearch Clusters at Uber
- Elasticsearch Performance Tuning Practice at eBay
- Improve Performance using Elasticsearch Plugins (2 parts) at Tinder
- Elasticsearch at Kickstarter
- Elasticsearch at Target
- Log Parsing with Logstash and Google Protocol Buffers at Trivago
- Fast Order Search using Data Pipeline and Elasticsearch at Yelp
- Moving Core Business Search to Elasticsearch at Yelp
- Sharding out Elasticsearch at Vinted
- Self-Ranking Search with Elasticsearch at Wattpad
- Vulcanizer: a library for operating Elasticsearch at Github
- 分布式存储
- In-memory Storage
- MemSQL Architecture - The Fast (MVCC, InMem, LockFree, CodeGen) And Familiar (SQL)
- Optimizing Memcached Efficiency at Quora
- Real-Time Data Warehouse with MemSQL on Cisco UCS
- Moving to MemSQL at Tapjoy
- MemSQL and Kinesis for Real-time Insights at Disney
- MemSQL to Query Hundreds of Billions of Rows in a Dashboard at Pandora
- Object Storage
- Scaling HDFS at Uber
- Reasons for Choosing S3 over HDFS at Databricks
- File System on Amazon S3 at Quantcast
- Image Recovery at Scale Using S3 Versioning at Trivago
- Cloud Object Store at Yahoo
- Ambry: Distributed Immutable Object Store at LinkedIn
- Dynamometer: Scale Testing HDFS on Minimal Hardware with Maximum Fidelity at LinkedIn
- Hammerspace: Persistent, Concurrent, Off-heap Storage at Airbnb
- MezzFS: Mounting Object Storage in Media Processing Platform at Netflix
- Magic Pocket: In-house Multi-exabyte Storage System at Dropbox
- In-memory Storage
- 关系数据库
- MySQL for Schema-less Data at FriendFeed
- MySQL at Pinterest
- PostgreSQL at Twitch
- Scaling MySQL-based Financial Reporting System at Airbnb
- Scaling MySQL at Wix
- MaxScale (MySQL) Database Proxy at Airbnb
- Switching from Postgres to MySQL at Uber
- Handling Growth with Postgres at Instagram
- Scaling the Analytics Database (Postgres) at TransferWise
- Updating a 50 Terabyte PostgreSQL Database at Adyen
- Scaling Database Access for 100s of Billions of Queries per Day at PayPal
- Minimizing Read-Write MySQL Downtime at Yelp
- Replication
- MySQL Parallel Replication (4 parts) at Booking.com
- Mitigating MySQL Replication Lag and Reducing Read Load at Github
- Black-Box Auditing: Verifying End-to-End Replication Integrity between MySQL and Redshift at Yelp
- Partitioning Main MySQL Database at Airbnb
- Herb: Multi-DC Replication Engine for Schemaless Datastore at Uber
- Sharding
- Sharding MySQL at Pinterest
- Sharding MySQL at Twilio
- Sharding MySQL at Square
- Sharding MySQL at Quora
- Sharding Layer of Schemaless Datastore at Uber
- Sharding & IDs at Instagram
- Solr: Improving Performance for Batch Indexing at Box
- Geosharded Recommendations (3 parts) at Tinder
- Scaling Services with Shard Manager at Facebook
- Presto the Distributed SQL Query Engine
- NoSQL 数据库
- Key-Value Databases
- DynamoDB at Nike
- DynamoDB at Segment
- DynamoDB at Mapbox
- Manhattan: Distributed Key-Value Database at Twitter
- Sherpa: Distributed NoSQL Key-Value Store at Yahoo
- HaloDB: Embedded Key-Value Storage Engine at Yahoo
- MPH: Fast and Compact Immutable Key-Value Stores at Indeed
- Venice: Distributed Key-Value Database at Linkedin
- Columnar Databases
- Cassandra
- Cassandra at Instagram
- Storing Images in Cassandra at Walmart
- Storing Messages with Cassandra at Discord
- Scaling Cassandra Cluster at Walmart
- Scaling Ad Analytics with Cassandra at Yelp
- Scaling to 100+ Million Reads/Writes using Spark and Cassandra at Dream11
- Moving Food Feed from Redis to Cassandra at Zomato
- Benchmarking Cassandra Scalability on AWS at Netflix
- Service Decomposition at Scale with Cassandra at Intuit QuickBooks
- Cassandra for Keeping Counts In Sync at SoundCloud
- cstar: Cassandra Orchestration Tool at Spotify
- HBase
- Redshift
- Cassandra
- Document Databases
- eBay: Building Mission-Critical Multi-Data Center Applications with MongoDB
- MongoDB at Baidu: Multi-Tenant Cluster Storing 200+ Billion Documents across 160 Shards
- Migrating Mongo Data at Addepar
- The AWS and MongoDB Infrastructure of Parse (acquired by Facebook)
- Migrating Mountains of Mongo Data at Addepar
- Couchbase Ecosystem at LinkedIn
- SimpleDB at Zendesk
- Espresso: Distributed Document Store at LinkedIn
- Graph Databases
- Key-Value Databases
- 时间序列数据库
- Beringei: High-performance Time Series Storage Engine at Facebook
- MetricsDB: TimeSeries Database for storing metrics at Twitter
- Atlas: In-memory Dimensional Time Series Database at Netflix
- Heroic: Time Series Database at Spotify
- Roshi: Distributed Storage System for Time-Series Event at SoundCloud
- Goku: Time Series Database at Pinterest
- Scaling Time Series Data Storage (2 parts) at Netflix
- Druid - Real-time Analytics Database
- 分布式存储库、依赖性和配置管理
- DGit: Distributed Git at Github
- Stemma: Distributed Git Server at Palantir
- Configuration Management for Distributed Systems at Flickr
- Git Repository at Microsoft
- Solve Git Problem with Large Repositories at Microsoft
- Single Repository at Google
- Scaling Infrastructure and (Git) Workflow at Adyen
- Dotfiles Distribution at Booking.com
- Secret Detector: Preventing Secrets in Source Code at Yelp
- Managing Software Dependency at Scale at LinkedIn
- Merging Code in High-velocity Repositories at LinkedIn
- Dynamic Configuration at Twitter
- Dynamic Configuration at Mixpanel
- Dynamic Configuration at GoDaddy
- 扩展持续集成和持续交付
- Continuous Integration Stack at Facebook
- Continuous Integration with Distributed Repositories and Dependencies at Netflix
- Continuous Integration and Deployment with Bazel at Dropbox
- Continuous Deployments at BuzzFeed
- Screwdriver: Continuous Delivery Build System for Dynamic Infrastructure at Yahoo
- CI/CD at Betterment
- CI/CD at Brainly
- Scaling iOS CI with Anka at Shopify
- Scaling Jira Server at Yelp
- Auto-scaling CI/CD cluster at Flexport
可用性
- Resilience Engineering: Learning to Embrace Failure
- Resilience Engineering with Project Waterbear at LinkedIn
- Resiliency against Traffic Oversaturation at iHeartRadio
- Resiliency in Distributed Systems at GO-JEK
- Practical NoSQL Resilience Design Pattern for the Enterprise at eBay
- Ensuring Resilience to Disaster at Quora
- Site Resiliency at Expedia
- Resiliency and Disaster Recovery with Kafka at eBay
- Disaster Recovery for Multi-Region Kafka at Uber
- Failover
- The Evolution of Global Traffic Routing and Failover
- Testing for Disaster Recovery Failover Testing
- Designing a Microservices Architecture for Failure
- ELB for Automatic Failover at GoSquared
- Eliminate the Database for Higher Availability at American Express
- Failover with Redis Sentinel at Vinted
- High-availability SaaS Infrastructure at FreeAgent
- MySQL High Availability at GitHub
- Business Continuity & Disaster Recovery at Walmart
- Load Balancing
- Introduction to Modern Network Load Balancing and Proxying
- Top Five (Load Balancing) Scalability Patterns
- Load Balancing infrastructure to support more than 1.3 billion users at Facebook
- DHCPLB: DHCP Load Balancer at Facebook
- Katran: Scalable Network Load Balancer at Facebook
- Deterministic Aperture: A Distributed, Load Balancing Algorithm at Twitter
- Load Balancing with Eureka at Netflix
- Edge Load Balancing at Netflix
- Zuul 2: Cloud Gateway at Netflix
- Load Balancing at Yelp
- Load Balancing at Github
- Consistent Hashing to Improve Load Balancing at Vimeo
- UDP Load Balancing at 500 pixel
- QALM: QoS Load Management Framework at Uber
- Traffic Steering using Rum DNS at LinkedIn
- Traffic Infrastructure (Edge Network) at Dropbox
- Intelligent DNS based load balancing at Dropbox
- Monitor DNS systems at Stripe
- Multi-DNS Architecture (3 parts) at Monday
- Rate Limiting
- Autoscaling
- Autoscaling Pinterest
- Autoscaling Based on Request Queuing at Square
- Autoscaling Jenkins at Trivago
- Autoscaling Pub-Sub Consumers at Spotify
- Autoscaling Bigtable Clusters based on CPU Load at Spotify
- Autoscaling AWS Step Functions Activities at Yelp
- Scryer: Predictive Auto Scaling Engine at Netflix
- Bouncer: Simple AWS Auto Scaling Rollovers at Palantir
- Clusterman: Autoscaling Mesos Clusters at Yelp
- Availability in Globally Distributed Storage Systems at Google
- NodeJS High Availability at Yahoo
- Operations (11 parts) at LinkedIn
- Monitoring Powers High Availability for LinkedIn Feed
- Supporting Global Events at Facebook
- High Availability at BlaBlaCar
- High Availability at Netflix
- High Availability Cloud Infrastructure at Twilio
- Automating Datacenter Operations at Dropbox
- Globalizing Player Accounts at Riot Games
稳定性
- Circuit Breaker
- Circuit Breaking in Distributed Systems
- Circuit Breakers for Distributed Services at LINE
- Applying Circuit Breaker to Channel Gateway at LINE
- Lessons in Resilience at SoundCloud
- Circuit Breaker for Scaling Containers
- Protector: Circuit Breaker for Time Series Databases at Trivago
- Improved Production Stability with Circuit Breakers at Heroku
- Circuit Breakers at Zendesk
- Circuit Breakers at Traveloka
- Timeouts
- Crash-safe Replication for MySQL at Booking.com
- Bulkheads: Partition and Tolerate Failure in One Part
- Steady State: Always Put Logs on Separate Disk
- Throttling: Maintain a Steady Pace
- Multi-Clustering: Improving Resiliency and Stability of a Large-scale Monolithic API Service at LinkedIn
- Determinism (4 parts) in League of Legends Server
性能
- Performance Optimization on OS, Storage, Database, Network
- Improving Performance with Background Data Prefetching at Instagram
- Compression Techniques to Solve Network I/O Bottlenecks at eBay
- Optimizing Web Servers for High Throughput and Low Latency at Dropbox
- Linux Performance Analysis in 60.000 Milliseconds at Netflix
- Live Downsizing Google Cloud Persistent Disks (PD-SSD) at Mixpanel
- Decreasing RAM Usage by 40% Using jemalloc with Python & Celery at Zapier
- Reducing Memory Footprint at Slack
- Performance Improvements at Pinterest
- Server Side Rendering at Wix
- 30x Performance Improvements on MySQLStreamer at Yelp
- Optimizing APIs at Netflix
- Performance Monitoring with Riemann and Clojure at Walmart
- Performance Tracking Dashboard for Live Games at Zynga
- Optimizing CAL Report Hadoop MapReduce Jobs at eBay
- Performance Tuning on Quartz Scheduler at eBay
- Profiling C++ (Part 1: Optimization, Part 2: Measurement and Analysis) at Riot Games
- Profiling React Server-Side Rendering at HomeAway
- Hardware-Assisted Video Transcoding at Dailymotion
- Cross Shard Transactions at 10 Million RPS at Dropbox
- API Profiling at Pinterest
- Pagelets Parallelize Server-side Processing at Yelp
- Improving key expiration in Redis at Twitter
- Ad Delivery Network Performance Optimization with Flame Graphs at MindGeek
- Predictive CPU isolation of containers at Netflix
- Cloud Jewels: Estimating kWh in the Cloud at Etsy
- Unthrottled: Fixing CPU Limits in the Cloud (2 parts) at Indeed
- Performance Optimization by Tuning Garbage Collection
- Garbage Collection in Java Applications at LinkedIn
- Garbage Collection in High-Throughput, Low-Latency Machine Learning Services at Adobe
- Garbage Collection in Redux Applications at SoundCloud
- Garbage Collection in Go Application at Twitch
- Analyzing V8 Garbage Collection Logs at Alibaba
- Python Garbage Collection for Dropping 50% Memory Growth Per Request at Instagram
- Performance Impact of Removing Out of Band Garbage Collector (OOBGC) at Github
- Debugging Java Memory Leaks at Allegro
- Optimizing JVM at Alibaba
- Tuning JVM Memory for Large-scale Services at Uber
- Performance Optimization on Image, Video, Page Load
- Optimizing 360 Photos at Scale at Facebook
- Reducing Image File Size in the Photos Infrastructure at Etsy
- Improving GIF Performance at Pinterest
- Optimizing Video Playback Performance at Pinterest
- Optimizing Video Stream for Low Bandwidth with Dynamic Optimizer at Netflix
- Adaptive Video Streaming at YouTube
- Reducing Video Loading Time at Dailymotion
- Improving Homepage Performance at Zillow
- The Process of Optimizing for Client Performance at Expedia
- Web Performance at BBC
- Performance Optimization by Brotli Compression
- Performance Optimization on Languages and Frameworks
智能
- 大数据
- Uber的数据平台
- 宝马的数据平台
- Netflix的数据平台
- Flipkart的数据平台
- Coupang的数据平台
- DoorDash的数据平台
- Khan Academy数据平台
- Airbnb的数据基础设施
- LinkedIn的数据基础设施
- GO-JEK的数据基础设施
- Pinterest的数据挖掘基础设施
- Pinterest的数据分析架构
- Spotify的大数据处理(2部分)
- Uber的大数据处理
- Lyft的分析管道
- Grammarly的分析管道
- Teads的分析管道
- 用于PayPal的实时欺诈防范的机器学习数据管道
- LinkedIn的大数据分析和机器学习技术
- LinkedIn的Hadoop自助报告平台
- LinkedIn的隐私保护分析与报告
- 用于跟踪沃尔玛商品供应情况的分析平台
- HALO：Facebook的硬件分析和生命周期优化
- RBEA：King的实时分析平台
- AresDB：Uber的GPU驱动实时分析引擎
- AthenaX：Uber的流媒体分析平台
- Delta: Netflix的数据同步和增强平台
- Keystone.Netflix的实时流处理平台
- Databook: 在Uber用元数据将大数据转化为知识
- Amundsen：Lyft的数据发现和元数据引擎
- Maze：Uber的漏斗可视化平台
- Metacat: 让大数据在Netflix变得可发现和有意义
- SpinalTap：Airbnb的变化数据采集系统
- Accelerator: eBay的快速数据处理框架
- Omid: 雅虎交易处理平台
- TensorFlowOnSpark: 在雅虎大数据集群上进行分布式深度学习
- Spark on Scala: Adobe公司的分析参考架构
- 在Spotify的实验平台（2部分）
- Airbnb的实验平台
- Zalando的智能产品平台
- LINE公司的日志分析平台
- Myntra的数据可视化平台
- 在Netflix构建和扩展数据
- 为Pinterest的计算机视觉任务构建一个可扩展的数据管理系统
- Etsy的结构化数据
- 拓展成熟的数据管道--Airbnb的管理开销
- Airbnb的Spark分区策略
- Distributed Machine Learning
- Aroma: Using ML for Code Recommendation at Facebook
- Flyte: Cloud Native Machine Learning and Data Processing Platform at Lyft
- Michelangelo: Machine Learning Platform at Uber
- Scaling Michelangelo
- Machine Learning Platform at Yelp
- Horovod: Open Source Distributed Deep Learning Framework for TensorFlow at Uber
- COTA: Improving Customer Care with NLP & Machine Learning at Uber
- Manifold: Model-Agnostic Visual Debugging Tool for Machine Learning at Uber
- Repo-Topix: Topic Extraction Framework at Github
- Concourse: Generating Personalized Content Notifications in Near-Real-Time at LinkedIn
- Altus Care: Applying a Chatbot to Platform Engineering at eBay
- PyKrylov: Accelerating Machine Learning Research at eBay
- Box Graph: Spontaneous Social Network at Box
- PricingNet: Pricing Modelling with Neural Networks at Skyscanner
- PinText: Multitask Text Embedding System at Pinterest
- Cannes: ML saves $1.7M a year on document previews at Dropbox
- Scaling Gradient Boosted Trees for Click-Through-Rate Prediction at Yelp
- Learning with Privacy at Scale at Apple
- Deep Learning for Image Classification Experiment at Mercari
- Deep Learning for Frame Detection in Product Images at Allegro
- Content-based Video Relevance Prediction at Hulu
- Improving Photo Selection With Deep Learning at TripAdvisor
- Personalized Recommendations for Experiences Using Deep Learning at TripAdvisor
- Personalised Recommender Systems at BBC
- Machine Learning (2 parts) at Condé Nast
- Natural Language Processing and Content Analysis (2 parts) at Condé Nast
- Mapping the World of Music Using Machine Learning (2 parts) at iHeartRadio
- Machine Learning to Improve Streaming Quality at Netflix
- Machine Learning to Match Drivers & Riders at GO-JEK
- Improving Video Thumbnails with Deep Neural Nets at YouTube
- Quantile Regression for Delivering On Time at Instacart
- Cross-Lingual End-to-End Product Search with Deep Learning at Zalando
- Machine Learning at Jane Street
- Machine Learning for Ranking Answers End-to-End at Quora
- Clustering Similar Stories Using LDA at Flipboard
- Similarity Search at Flickr
- Large-Scale Machine Learning Pipeline for Job Recommendations at Indeed
- Deep Learning from Prototype to Production at Taboola
- Atom Smashing using Machine Learning at CERN
- Mapping Tags at Medium
- Clustering with the Dirichlet Process Mixture Model in Scala at Monsanto
- Map Pins with DBSCAN & Random Forests at Foursquare
- Detecting and Preventing Fraud at Uber
- Forecasting at Uber
- Financial Forecasting at Uber
- Productionizing ML with Workflows at Twitter
- GUI Testing Powered by Deep Learning at eBay
- Scaling Machine Learning to Recommend Driving Routes at Pivotal
- Real-Time Predictions at DoorDash
- Machine Intelligence at Dropbox
- Machine Learning for Indexing Text from Billions of Images at Dropbox
- Modeling User Journeys via Semantic Embeddings at Etsy
- Automated Fake Account Detection at LinkedIn
- Building Knowledge Graph at Airbnb
- Core Modeling at Instagram
- Neural Architecture Search (NAS) for Prohibited Item Detection at Mercari
- Computer Vision at Airbnb
- 3D Home Backend Algorithms at Zillow
- Long-term Forecasts at Lyft
- Discovering Popular Dishes with Deep Learning at Yelp
- SplitNet Architecture for Ad Candidate Ranking at Twitter
- Jobs Filter at Indeed
- Architecting Restaurant Wait Time Predictions at Yelp
- Music Personalization at Spotify
- Deep Learning for Domain Name Valuation at GoDaddy
- Similarity Clustering to Catch Fraud Rings at Stripe
- Personalized Search at Etsy
- ML Feature Serving Infrastructure at Lyft
架构
- Systems We Make
- Tech Stack (2 parts) at Uber
- Tech Stack at Medium
- Tech Stack at Shopify
- Building Services (4 parts) at Airbnb
- Architecture of Evernote
- Architecture of Chat Service (3 parts) at Riot Games
- Architecture of League of Legends Client Update
- Architecture of Ad Platform at Twitter
- Basic Architecture of Slack
- Back-end at LinkedIn
- Back-end at Flickr
- Infrastructure (3 parts) at Zendesk
- Cloud Infrastructure at Grubhub
- Real-time Presence Platform at LinkedIn
- Settings Platform at LinkedIn
- Nearline System for Scale and Performance (2 parts) at Glassdoor
- Real-time User Action Counting System for Ads at Pinterest
- API Platform at Riot Games
- Games Platform at The New York Times
- Kabootar: Communication Platform at Swiggy
- Simone: Distributed Simulation Service at Netflix
- Seagull: Distributed System that Helps Running > 20 Million Tests Per Day at Yelp
- PriceAggregator: Intelligent System for Hotel Price Fetching (3 parts) at Agoda
- Phoenix: Testing Platform (3 parts) at Tinder
- Hexagonal Architecture at Netflix
- Architecture of Play API Service at Netflix
- Architecture of Sticker Services at LINE
- Stack Overflow Enterprise at Palantir
- Architecture of Following Feed, Interest Feed, and Picked For You at Pinterest
- API Specification Workflow at WeWork
- Media Database at Netflix
- Member Transaction History Architecture at Walmart
- Sync Engine (2 parts) at Dropbox
- Architectures of Finance and Banking Systems
面试
- Designing Large-Scale Systems
- My Scaling Hero - Jeff Atwood (a dose of Endorphins before your interview, JK)
- Software Engineering Advice from Building Large-Scale Distributed Systems - Jeff Dean
- Introduction to Architecting Systems for Scale
- Anatomy of a System Design Interview
- 8 Things You Need to Know Before a System Design Interview
- Top 10 System Design Interview Questions
- Top 10 Common Large-Scale Software Architectural Patterns in a Nutshell
- Cloud Big Data Design Patterns - Lynn Langit
- How NOT to design Netflix in your 45-minute System Design Interview?
- API Best Practices: Webhooks, Deprecation, and Design
- Explaining Low-Level Systems (OS, Network/Protocol, Database, Storage)
- "What Happens When... and How" Questions
组织
- Engineering Levels at SoundCloud
- Engineering Roles at Palantir
- Scaling Engineering Teams at Twitter
- Scaling Decision-Making Across Teams at LinkedIn
- Scaling Data Science Team at GOJEK
- Scaling Agile at Zalando
- Scaling Agile at bol.com
- Lessons Learned from Scaling a Product Team at Intercom
- Hiring, Managing, and Scaling Engineering Teams at Typeform
- Scaling the Datagram Team at Instagram
- Scaling the Design Team at Flexport
- Team Model for Scaling a Design System at Salesforce
- Building Analytics Team (4 parts) at Wish
- From 2 Founders to 1000 Employees at Transferwise
- Lessons Learned Growing a UX Team from 10 to 170 at Adobe
- Five Lessons from Scaling at Pinterest
- Approach Engineering at Vinted
- Using Metrics to Improve the Development Process (and Coach People) at Indeed
- Mistakes to Avoid while Creating an Internal Product at Skyscanner
- RACI (Responsible, Accountable, Consulted, Informed) at Etsy
- Four Pillars of Leading People (Empathy, Inspiration, Trust, Honesty) at Zalando
- Pair Programming at Shopify
- Distributed Responsibility at Asana
- Rotating Engineers at Zalando
- Experiment Idea Review at Pinterest
- Tech Migrations at Spotify
- Improving Code Ownership at Yelp
- Agile Code Base at eBay
- Code Review
访谈
- Distributed Systems in One Lesson - Tim Berglund, Senior Director of Developer Experience at Confluent
- Building Real Time Infrastructure at Facebook - Jeff Barber and Shie Erlich, Software Engineer at Facebook
- Building Reliable Social Infrastructure for Google - Marc Alvidrez, Senior Manager at Google
- Building a Distributed Build System at Google Scale - Aysylu Greenberg, SDE at Google
- Site Reliability Engineering at Dropbox - Tammy Butow, Site Reliability Engineering Manager at Dropbox
- How Google Does Planet-Scale for Planet-Scale Infra - Melissa Binde, SRE Director for Google Cloud Platform
- Netflix Guide to Microservices - Josh Evans, Director of Operations Engineering at Netflix
- Achieving Rapid Response Times in Large Online Services - Jeff Dean, Google Senior Fellow
- Architecture to Handle 80K RPS Celebrity Sales at Shopify - Simon Eskildsen, Engineering Lead at Shopify
- Lessons of Scale at Facebook - Bobby Johnson, Director of Engineering at Facebook
- Performance Optimization for the Greater China Region at Salesforce - Jeff Cheng, Enterprise Architect at Salesforce
- How GIPHY Delivers a GIF to 300 Millions Users - Alex Hoang and Nima Khoshini, Services Engineers at GIPHY
- High Performance Packet Processing Platform at Alibaba - Haiyong Wang, Senior Director at Alibaba
- Solving Large-scale Data Center and Cloud Interconnection Problems - Ihab Tarazi, CTO at Equinix
- Scaling Dropbox - Kevin Modzelewski, Back-end Engineer at Dropbox
- Scaling Reliability at Dropbox - Sat Kriya Khalsa, SRE at Dropbox
- Scaling with Performance at Facebook - Bill Jia, VP of Infrastructure at Facebook
- Scaling Live Videos to a Billion Users at Facebook - Sachin Kulkarni, Director of Engineering at Facebook
- Scaling Infrastructure at Instagram - Lisa Guo, Instagram Engineering
- Scaling Infrastructure at Twitter - Yao Yue, Staff Software Engineer at Twitter
- Scaling Infrastructure at Etsy - Bethany Macri, Engineering Manager at Etsy
- Scaling Real-time Infrastructure at Alibaba for Global Shopping Holiday - Xiaowei Jiang, Senior Director at Alibaba
- Scaling Data Infrastructure at Spotify - Matti (Lepistö) Pehrs, Spotify
- Scaling Pinterest - Marty Weiner, Pinterest’s founding engineer
- Scaling Slack - Bing Wei, Software Engineer (Infrastructure) at Slack
- Scaling Backend at Youtube - Sugu Sougoumarane, SDE at Youtube
- Scaling Backend at Uber - Matt Ranney, Chief Systems Architect at Uber
- Scaling Global CDN at Netflix - Dave Temkin, Director of Global Networks at Netflix
- Scaling Load Balancing Infra to Support 1.3 Billion Users at Facebook - Patrick Shuff, Production Engineer at Facebook
- Scaling (a NSFW site) to 200 Million Views A Day And Beyond - Eric Pickup, Lead Platform Developer at MindGeek
- Scaling Counting Infrastructure at Quora - Chun-Ho Hung and Nikhil Gar, SEs at Quora
- Scaling Git at Microsoft - Saeed Noursalehi, Principal Program Manager at Microsoft
- Scaling Multitenant Architecture Across Multiple Data Centres at Shopify - Weingarten, Engineering Lead at Shopify
书籍
- Big Data, Web Ops & DevOps Ebooks - O'Reilly (Online - Free)
- Google Site Reliability Engineering (Online - Free)
- Distributed Systems for Fun and Profit (Online - Free)
- What Every Developer Should Know About SQL Performance (Online - Free)
- Beyond the Twelve-Factor App - Exploring the DNA of Highly Scalable, Resilient Cloud Applications (Free)
- Chaos Engineering - Building Confidence in System Behavior through Experiments (Free)
- The Art of Scalability
- Web Scalability for Startup Engineers
- Scalability Rules: 50 Principles for Scaling Web Sites
(全文完)