How Uber Created Cinnamon

🌙 Hello world ☀️ 

Check out the GitHub Innovation Graph if you haven't already. It offers insights based on eight metrics, showcasing JavaScript's continued dominance as the top programming language, followed by Python.

Notably, TypeScript has impressively climbed from 11th to 5th place, while Ruby has seen a significant drop from 5th to the last position at 12th.

My prediction is that TypeScript will be among the top three within the next two years.

In this week’s email:

• Microservices: How Uber created Cinnamon, an advanced load shedder library.

• JavaScript: A new optimizing compiler that enhances JavaScript execution efficiency in V8.

• Database: Insights into the comprehensive process, challenges, and strategies involved in upgrading GitHub’s large-scale MySQL infrastructure.

• Kubernetes: Learn how taints and tolerations work.

• Best Practices: Learn about the purpose, structure, and benefits of RFCs.

MICROSERVICES
How Uber Created Cinnamon

Jakob Holdgaard Thomsen

Introduction

In the fast-paced world of Uber's microservices, catering to approximately 130 million monthly customers involves managing an immense volume of online and offline traffic.

This complexity inevitably leads to scenarios where services become overloaded due to factors like traffic spikes or database slowdowns.

To ensure seamless user experience despite these challenges, Uber developed Cinnamon, an advanced load shedder library, building upon the legacy of its predecessor, QALM.

The Necessity of Graceful Degradation

The key to managing service overload lies in gracefully degrading system performance by shedding lower-priority requests, such as batch jobs, to preserve the quality of high-priority services like customer requests.

Uber's vast array of services demands an automatic, configuration-free solution for graceful degradation, given the prohibitive cost of configuring each service individually.

Introducing Cinnamon

Cinnamon is the culmination of Uber’s efforts to enhance service stability.

It dynamically adjusts service capacity using a modified TCP-Vegas algorithm and determines load shedding levels using a PID controller concept, dating back to the 17th century.

Compared to previous methods, Cinnamon effectively manages larger overloads while maintaining reasonable latencies, adding minimal overhead per request.

Crucially, it requires no manual configuration, facilitating rapid adoption across Uber's services.

Understanding QALM

Before delving into Cinnamon, it's essential to grasp the workings of QALM, its forerunner.

QALM employed the Controlled Delay (CoDel) algorithm to manage overload by dropping lower-priority requests.

It monitored request queue times and, upon detecting delays, would reject low-priority requests.

However, its reliance on service configuration and priority classification posed challenges to widespread adoption.

Cinnamon's Architecture

Cinnamon functions as an RPC middleware in Uber’s Go services.

It tags each request with a priority at the edge, which is then propagated through the service stack.

This priority tagging helps services understand the importance of each request, enabling more informed load-shedding decisions.

Priorities in Cinnamon

Cinnamon’s prioritization mechanism consists of two elements: tier levels and cohorts.

The tier level indicates the request's importance, with six tiers used at Uber.

Cohorts further segment requests within a tier, allowing consistent load shedding across services.

This system, inspired by WeChat, involves 768 different priority levels.

Library Architecture

Cinnamon operates in stages as a request passes through its middleware:

1. Priority Assignment: Assigns or verifies the priority of a request.

2. Rejection Decision: Determines if a request should be rejected based on system load.

3. Priority Queue Management: Prioritizes requests in the queue.

4. Concurrent Request Management: Limits the number of parallel requests.

5. Timeout Management: Handles request timeouts in the queue.

Additionally, Cinnamon's background threads, the PID controller, and the auto-tuner, adjust rejection thresholds and concurrent request limits to optimize system performance.

Experimental Results

Testing Cinnamon against QALM revealed its superior capacity in managing service load and protecting core flows.

It demonstrated higher goodput (successful request rate) and maintained better latencies under heavy loads compared to QALM.

Interaction between PID and Auto-Tuner

Cinnamon’s PID controller and auto-tuner work in tandem to balance rejection rates and service throughput.

This interaction ensures that the system dynamically adapts to varying loads, maintaining optimal service levels.

Conclusion

Cinnamon represents a significant leap in load shedding technology at Uber.

By leveraging time-tested algorithms and incorporating modern enhancements, it offers a robust, efficient, and configuration-free solution to manage service overloads.

Cinnamon's development, inspired by diverse technologies and ideas, stands as a testament to Uber's commitment to innovative and effective software solutions in a complex, high-demand environment.

P.S. If you’re enjoying the content of this newsletter, please share it with your network: https://www.fullstackexpress.io/subscribe

JAVASCRIPT
V8’s Fastest Optimizing JIT

Toon Verwaest

Big picture: Maglev, a new optimizing compiler introduced in Chrome M117, enhances JavaScript execution efficiency in V8.

What you’ll learn: Discover how Maglev bridges the gap between Sparkplug's instant compilation and TurboFan's peak performance optimization, using sophisticated techniques like SSA-based JIT compilation for improved web performance.

Why this matters: Maglev's introduction is pivotal for web developers and users alike, offering faster, more efficient JavaScript execution in browsers, reducing resource consumption, and improving user experience on web applications.

DATABASE
Upgrading GitHub.com To MySQL 8.0

Jiaqi Liu

Big picture: GitHub's transition to MySQL 8.0 for their production fleet is a detailed journey of upgrading over 1200 MySQL hosts, enhancing data storage, and ensuring high availability for the vast relational data of GitHub.com.

What you’ll learn: Insights into the comprehensive process, challenges, and strategies involved in upgrading a large-scale MySQL infrastructure, emphasizing the importance of maintaining service level objectives and handling complex data workloads.

Why this matters: This upgrade demonstrates the critical role of database management in large tech platforms, offering valuable lessons in scalability, reliability, and the necessity of continuous software maintenance and improvement for operational efficiency and future growth.

KUBERNETES
Importance Of Kubernetes And The Need For Tainting Nodes

Abhisman Sarkar

Big picture: The article details the importance of using taints and tolerations in Kubernetes for efficient cluster management and optimal workload distribution.

What you’ll learn: Understand how taints and tolerations work in Kubernetes, enabling them to control pod placements on nodes, which is essential for specific use cases like multi-tenancy, backup strategies, and environment-specific deployments.

Why this matters: Mastering the concept of tainting and tolerating nodes in Kubernetes is crucial for developers and system administrators to ensure that containerized applications are deployed effectively, meeting the diverse requirements of modern software environments.

BEST PRACTICES
On The Importance of RFCs In Programming

Matija Sosic

Big picture: The article emphasizes the critical role of Request For Comments (RFC) documents in programming, particularly in collaborative environments and complex projects.

What you’ll learn: You'll learn about the purpose, structure, and benefits of RFCs, and how they facilitate informed decision-making and efficient implementation in software development by preemptively addressing potential issues and gathering team insights.

Why this matters: Understanding and utilizing RFCs is essential for developers, as they streamline the coding process, ensure clarity of purpose, and foster a collaborative approach to problem-solving, ultimately leading to more effective and successful project outcomes.

DATA STRUCTURES & ALGORITHMS
Middle Of the Linked List

Missed the solution to the latest coding challenge?

This question is asked by Amazon and Apple. Learn how to solve this problem here.

JAVASCRIPT ECOSYSTEM
JS Weekly Pulse

📢 Supabase: Supabase GraphQL (pg_graphql) version 1.4+ now supports User Defined Functions (UDFs) in Postgres, enhancing GraphQL's flexibility and allowing custom SQL logic within GraphQL queries.

📜 Chain Reaction: Dan Abramov demonstrates the significance of naming conventions in HTML and CSS, and how they simplify the browser’s process of displaying web elements.

📜 Postman Collections: A new format for storing Postman collections using a directory structure, addressing challenges like difficult pull request reviews and limited editor options, and enhancing the ease of maintaining and updating API tests in CI pipelines.

📜 Notifications: How to set up automated notifications for CI builds in Slack and Twilio, using NestJS and CircleCI

🚀 Node 21.4.0: Fixed regression affecting fs.writeFileSync method, introduces dirent.parentPath, and default w flag for writeFileSync with utf8 encoding.

🚀 Astro 4.0: Dev Toolbar, internationalization routing, new view transition APIs, and incremental content caching.

🚀 Electron 28.0.0: Support for ECMAScript modules (ESM), additional properties to the display object, and support for ELECTRON_OZONE_PLATFORM_HINT on Linux.

🚀 React Native 0.73: Debugging improvements, stable symlink support in Metro, Kotlin template on Android, Android 14 support, and more.

🚀 Redux Toolkit 2.0.0: Enhanced store configuration and middleware options, improved package compatibility, and TypeScript improvements.

🚀 Million 3.0: 34% faster initial compile, ~2% for code changes on medum-sized Next.js projects, and 100% improvement in hydration times.

🚀 GitHub Enterprise Server 3.11: Generally available, enhanced code security with tools for application security testing, new Activity view for repository history, advanced secret leak prevention, and GitHub CLI extension.

RECOMMENDATIONS
Around the Web

✅ Interesting: Visualization and walkthrough of ChatGPT’s LLM algorithm.

✅ Software Engineering: A review of the original 1970 paper on the waterfall model in software engineering, discussing its key components, the shift in management's role, optimization in development, and iterative approaches.

✅ Learn: Programming challenges of text editors, focusing on reworking fundamental data structures, particularly for handling text.

✅ Career Growth: Ryan Peterman reflects on his transition from mid-level to senior engineer at Meta.

✅ Internet Trends: Cloudflare shares their 2023 Radar Year in Review, a comprehensive analysis of annual global and regional Internet trends based on various metrics.

COMMUNITY SPOTLIGHT
Tools and Packages

📦️ Terminal Trove: Curated collection of terminal tools, including CLIs, TUIs, and developer tools, for all platforms and mediums.

📦️ Ncube: Visualize rotating hypercubes in any dimension, using real-time rotation and perspective projections down to three dimensions.

📦️ Build Your Own X: Enhance your programming skills by building your own versions of popular technologies from the ground up.

📦️ SuperDuperDB: A framework for directly integrating AI with databases, featuring streaming inference, scalable training, and vector search.

📦️ Typia: Fast and easy runtime validations and serializations through transformation.

SOCIAL
🤣🤣🤣🤣🤣