The Chamber 🏰 of Tech Secrets has been opened.
There are a number of ways to build infrastructure for workloads at the edge. One of those is the Kubernetes on NUCs solution our team developed at Chick-fil-A in 2018. There are also other great approaches. Do you need the edge? Which options should you consider? Keep reading to learn more about how my brain thinks about the entire Edge ecosystem.👇
Applications are King 👑
We do all we do to serve our application overlords. And server them. 🙌 In some cases they demand more than our Cloud fiefdoms can offer and we must expand our borders into the unknown. To the edge!
There are numerous definitions for “edge” and its important that we agree to some vocabulary so that we are thinking about the same things.
Mobile devices / wearables: Increasing processing capability (like Apple chips), less redundancy, simple single-app deployment paradigm (mobile apps), high cloud dependency for many use cases. Think iPhone or Android tablet. 📱 Some people don’t consider a mobile device to be the edge, but I do.
Near-edge: Infrastructure close to “the action” to enable offline / low-latency applications, multi-app paradigm, clustering-friendly, tiered-architecture-friendly, lower cloud dependency for many use cases, high variance on environmentals (cooling, clean power, physical space, ancillary services like networking / PXE). Varying connection reliability. Think Chick-fil-A in-restaurant edge clusters. 🐓
Regional edge: close to users but connection dependent, primarily useful for latency reduction, broad set of application capabilities ranging from serverless functions (Cloudflare workers) to managed orchestration platforms, databases and more. Data center quality environment with minimal constraints and reliable / redundant connectivity. Think Cloudflare or cell carrier data centers. 🥓 ←can’t explain my emoji selection here.
Cloud: hyperscale or smaller cloud infrastructure providers with robust and expanding capabilities. ☁️
Units of Deployment
What are the best ways to deploy edge applications?
Containers: the container has become the default unit of deployment in most places, including at the edge. They’re developer-friendly and the tooling ecosystem is unmatched at present. Solutions like Kubernetes, K3s, micro-k8s, Hashicorp Nomad, Docker Swarm or Mesos can handle container workload scheduling and a plethora of other activities, and some are lightweight enough to do so at the highly constrained Edge.
Micro-VMs: small, efficient, secure VM platforms such as Firecracker are another approach to running applications on the edge. Firecracker VMs can actually be orchestrated through Kubernetes via firecracker-containerd. They aren’t as lightweight as containers but may offer more appropriate isolation and security control for certain types of use cases. Of course, you can also run containers inside firecracker VMs as well. Lots of layers here.
Web Assembly (WASM): WASM is still emerging but showing up in interesting places such as Cloudflare Workers and Shopify plugins, not to mention end-user devices. WASM binaries are lean and extremely fast and efficient, which is attractive for the edge. Their “sandbox” nature makes them good candidates for multi-tenant edge solutions. They can also be scheduled with Hashicorp Nomad or Kubernetes, among others.
Kubernetes seems to have grown both heavier and lighter over time… adding lots of lines of code to the trunk but also being trimmed into purpose-built edge distributions. Its also worth checking out KCP Edge which my friend Andy Anderson is working hard on. I believe we’ll see the continued layering of all of these components, with containers dominating the “near edge” and WASM taking a hold in the regional edge via serverless functions in CDN POPs (AWS, Cloudflare, etc).
Three Infrastructure Options
If our application tolerates cloud + regional edge (but needs no more), its likely that someone will provide us with robust infrastructure, high reliability, and quality connections. We should maximize those before looking further.
If we must expand to the near edge—which requires us to build / buy an architecture that is less turnkey— what options do we have? How might one deploy and manage workloads to the edge? If we are building an edge product, what should we consider?
Single node approach: This simplest approach involves running a single edge node and embracing the possible failure scenarios that may arise. This is suitable for use cases that can withstand 24-48 hours of downtime while a replacement device arrives. Applications could run on 1..n VMs, containers, WASM on bare metal, or just about anything you can think of.
High Availability (HA) Clustered Approach: In some instances, a HA cluster is a better option, providing reduced downtime for workloads. This comes at the cost of a more complex architecture, necessitating clustering, distributed systems management, additional physical space, and increased infrastructure expenses. Ensure that you consider your constraints; a well-designed HA solution might still be highly unavailable if the network, power supply, or other environmental factors make it is unreliable. In Kubernetes, you’ll need to think about control plane management challenges, which is not easy.
High Recoverability (HR) Clustered Approach: An HR cluster focuses on rapid and efficient recovery from various failures or disruptions. Instead of designing a cluster to prevent failure modes, this approach accepts the likelihood of failure and emphasizes swift recovery. This can be a great solution of some downtime is tolerable. This is the approach we used at Chick-fil-A, and we are usually able to return our clusters to service in 8-10 minutes in the event of issues.
Takeaways to consider
Lets wrap this discussion with some principles to consider when designing your edge solution.
Applications are King.
Containers are the default but there are many approaches that can be layered in to achieve what an application needs.
Keep everything as simple and lightweight as possible. Use a single node if you can.
Acknowledge constraints.
Accept tradeoffs that drive management simplicity.
Remember to keep things as close to the cloud as possible and only on the edge when necessary.
Use Intel NUCs. 😜
Go forth and expand your kingdom to the edge… but only if you really need to!
Introducing the Chamber 🏰of Tech Secrets Book 📚 Club
Would you enjoy reading a great book, learning, and then going deeper in a discussion with a bunch of your newest and closest friends? Yea? Good. Introducing the Chamber 🏰 of Tech Secrets Book 📚 Club. Join our flourishing community of practitioners in architecture, software engineering, technology, and leadership for some fun.
How it works:
Opt-in here — you’ll have another chance next week once the book is decided but if you are in either way go ahead and sign up.
Read the book — the announcement on book selection will be in Chamber of Tech Secrets #7.
Attend the virtual book club discussion — we’ll have dialogue about what we read, what we learned, how we can apply it.
Flourish and succeed!
Secrets from the Edge
I am pretty impressed with Lior’s Edge setup! Rumor has it he’s working on v2. 👀
Oh yeah, this is a great resource. It is like in April 2023, you were anticipating my question. 🤘🏽🤩