The cloud is all around us
Think about everything you did today with an electronic device. Whether that was taking a video call for work, using Google Drive, listening to some music on Spotify, or scrolling through TikTok, all these experiences were fluidly enabled by cloud computing. But how does it really work? What is really going on?
Beginning around the mid-20th century, we entered into a broad shift in our economy being dominated by information technology: the use of computers to store, retrieve, transmit, and manipulate data or information. Cloud computing, or "the cloud", helps make these components–store (storage), manipulate (compute), retrieve and transmit (networking)–more readily available and configurable for companies of any size.
Let's look at what each are individually:
Storage
Storage is more broadly familiar since we often think about the storage capacity on our devices in gigabytes (GB), but let's put the amount of data that needs to be stored in perspective.
How much information? About 2.5 quintillion bytes per day. That's equivalent to about 12.5 billion new pictures created every day. By 2025, IDC predicts the world’s data to grow to 175 zettabytes (1 trillion gigabytes = 1 zettabyte). Putting all that data on to DVDs, the stack would be tall enough to circle the Earth 222 times!
And all this data has to be stored somewhere.
Compute
What happens in your head when you do 27 + 48?
Was it 20 + 40 = 60, 7 + 8 = 15, 60 + 15 = 75?
Or maybe 48 + 2 = 50, 27 - 2 = 25, 50 + 25 = 75?
That process of how you did it is an algorithm you have in your head to calculate outputs (75) when given specific inputs (27, 48) and tasks (add the two numbers).
Compute is the act of running algorithms and is measured by the time it took from start to finish.
In our world of data today, algorithms are seamlessly running (computing) around us. Whether it be whose Instagram post to show you next, how much the Uber ride will cost you, or what the weather for next week looks like, these tasks require computing power.
All this information needs to be processed somewhere, and just like how a mathematician would have solved the above problem much faster than you or I, what processes those algorithms also matters.
Algorithms are a predefined set of instructions that perform a task with specific types of inputs and outputs.
Compute is the act of running algorithms and is measured by the time that passed from start to finish.
Networking
Networking focuses on the movement of data. With data being constantly passed back and forth between the end user's device and the cloud, infrastructure is needed to facilitate those transfers. This infrastructure needs to enable consistent, fast, and secure data transfers regardless of where the data is coming from or going to.
Where is all this happening?
In short, a data center. Depending on the product you are using and the company that offers the service, there are 4 main kinds of data centers that are used, ordered here in decreasing levels of ownership/control and the last being the primary source for cloud computing:
Enterprise data centers ("on premises") are facilities and equipment owned and operated by the company using it.
Companies often choose this solution to conduct tight controls over security and maintain low data latency (data transfer speeds) with high computing performance. These benefits are weighed against higher setup costs in building/outfitting their facility and lower agility with changing business needs.
Colocation data centers ("colos") are facilities owned and operated by a third party, while the equipment within the data center (ie. server) is owned by the company using it. This is essentially paying just to use space within a large data center, so any upgrades must be paid for by the company as well.
Managed services data centers are facilities and equipment owned and operated by a third party, while the company using this service leases the equipment. Through this model, the company doesn't own anything but leasing allows them to have dedicated equipment (used only by that company) while transferring maintenance and upgrade responsibilities to the third party.
Cloud data centers, the source of cloud computing, are off-premise facilities owned by cloud services providers like Amazon (AWS), Microsoft (Azure), or Google (GCP). The key difference between cloud data centers and the above 3 data centers is companies using these services not only give up control over facilities and equipment, but may also lose sight to the specifics of where exactly (ie. which specific row or server in Data Center ABC) the storage and compute is happening. Thus, these companies pay in gigabytes stored and compute time spent by provisioning resources in advance or in a "pay-as-you-go" model.
Cloud computing is a model that enables on-demand access to a shared pool of computing resources (ie. networks, servers, applications, and services)
Cloud computing in a diagram
Types of Cloud Computing Services:
IaaS - Infrastructure as a Service:
This solution offers the fundamental infrastructure of virtual servers, storage drives, and networking, removing the need for any kind of hardware. It allows for enterprise-grade reliability and scalability of backend IT solutions to support growth, and is often used by small and medium sized companies.
PaaS - Platform as a Service:
A step above IaaS, PaaS abstracts away both the infrastructure and software frameworks for businesses, allowing them to focus solely on building their own applications. When companies that are starting out deploy PaaS solutions, the core team of engineers can spend time creating the product rather than configuring modular middleware components like data management, authentication services, and API management among others.
SaaS - Software as a Service:
Often seen as the end product of companies like in the PaaS example above, SaaS solutions are deployments of software to customers who pay via subscription or pay-per-use models. The end customer does not worry about maintenance or upgrades, instead simply paying for use of the software solution. Examples of SaaS products include Salesforce's CRM, Office 365, and Dropbox.
Future Trends
Chip Shortage (Medium-term)
Semiconductors, colloquially referred to as chips or microchips, have seen rapidly growing demand and inadequate supply, exacerbated by the pandemic. This demand resurgence arose from poor planning in some industries (e.g. automotive), new tech rollouts, as well as increased use of cloud computing services and crypto mining. They all buy lots of semiconductors.
Though the biggest purchasers with high-volume, long-standing orders like Apple and Samsung are able to maintain high priority, manufacturers are already operating at maximum capacity. If demand were to suddenly go up, like during the pandemic for both consumer products and enterprise cloud services, there is no quick way manufacturers can increase output. Building new semiconductor fabrication plants, or fabs, takes years and tens of billions of dollars.
In the meantime, companies have few alternatives. Increased wait times and higher fees are common, where missing even a small handful of cheap chips can prevent thousands of dollars in inventory from shipping. Many firms have projected the supply chain to readjust by 2022 or 2023. We already see companies paring back sales expectations, but will those short- to mid-term costs also be passed down to consumers?
Privacy & Security (continual)
While mass adoption of cloud computing has allowed companies to start up faster and customers to enjoy more convenience, privacy and security issues still pose significant challenges. In recent memory, sophistication of cyber attacks have escalated, as was the case for the SolarWinds hack that went undetected for years, compromised parts of the U.S. government and various corporations internationally, and have continued to be a looming threat for additional attacks now and in the future.
There are three main areas where data can be compromised: 1. data at rest (where it sits), 2. data in transit (how, when, and where it moves), and 3. authentication of users, applications, and processes (who can access it).
Privacy and security is often thought of as protecting against hackers or other external bad actors, but cloud computing providers must also have internal protections to mitigate incidental or malicious compromises by its own employees.
For companies considering the use of cloud computing, they must define their own security requirements based on the needs of their business, perform due diligence on potential providers based on past performance, and manage operational risk given the services have been outsourced.
Environmental Impacts (Medium- to long-term)
The primary negative impact of cloud computing arises from the vast amount of electricity needed to power and cool data centers. Greenpeace estimates that the technology sector could consume 20% of the world's total electricity by 2025, up from just 7% in 2019. Because cloud computing is not yet net-zero carbon, the environmental costs of operating needs to be weighed against the economical benefits both consumers and enterprises enjoy.
Furthermore, electronic waste has become a growing negative byproduct of continued technological innovation. As new devices and equipment continue to replace older generations, worldwide e-waste reached 53.6 million tons in 2019, with just 17.4% collected and recycled. Improper disposal results in health and environmental hazards due to toxic additives or hazardous substances like mercury and lead.
Cloud computing providers like Amazon and Microsoft have pledged net-zero and net-negative commitments, which bodes well for long-term environmental conservation, but public pressure needs to remain strong to ensure adherence. Proper recycling has also proven to be more valuable than mining for ore, but there are still companies that simply export waste to developing countries and have residents salvage for scrap that can expose them to toxic materials.
As consumers, we need to demand continued transparency and improved global standards.
I’ve never really thought so much about the scope and scale of data until reading this. Wild.