Ty Myrddin Home

Pave the way for distributed clouds on premises

by Ty Myrddin

Published on November 07, 2024

Current trends

Virtualisation solutions and moving data, computational, and software capabilities to the cloud drives for example, cloud-native and serverless applications. Its adoption signifies that centralisation of software and hardware resources in the hyperscale data centres allows for energy aware storage solutions like using Solid-State Drives for data needing high transfer speeds and long-term backup storage solutions for the rest of the data. Data centres can also use solar and wind farms, which is less likely an option for smaller organisations. Detection and shutdown of idle servers can also easily be done centrally to save energy. Most of all, the cloud enables use on demand of resources and geolocation for bringing a service near its end-points now.

Development of Integrated infrastructures concerns dedicated sustainable hardware for software development, and includes domain specific hardware.

Near future

Edge computing is a distributed model that brings some computational tasks and data storage closer to the user (as source of data and as consumer). It can range from static mini-clouds on premises to flexible follow-the-need services via onboard computing and distributed networks of computational nodes. Originally developed with IoT applications in mind, mini-clouds on premises can be perceived as an intermediate step towards a completely distributed paradigm following and supporting the ongoing trend in decentralised energy production and smart energy grids. This then can allow for better profiling of energy consumption patterns and data usage patterns.

AI computational tasks are very energy greedy. Trends such as federated learning algorithms, data compression of curated data, and approximate computing help reduce energy consumption, and perhaps there will be data-centric green AI techniques.

Especially worth mentioning is sustainability-aware self-adaptation, using decision maps.

What all three need is further research, academic and industrial eduction, additional learning paths, and most of all, an attitude towards responsibility for sustainable solutions created by design. Designs will likely first focus on reuse and hardware lifecycle management for creating longer lasting and better maintainable hardware, at a trade-off for being able to adopt new types of hardware when they become available.

The far future?

High power usage in computing caused by the von Neumann bottleneck is a significant sustainability issue, and Non-von Neumann architectures promise solutions for sustainable and energy-efficient nodes, especially at the edge layer.

Photonic integrated circuits have already enabled fast artificial neural networks. Photonics can perhaps piggy-back on distributed paradigms.

Problems

Awareness creation requires monitoring and communicating environmental impacts and a behavioural change in data consumption patterns, and is a big problem because there is not enough digital spine (yet). Real progress requires shared responsibility and accountability by all involved in all value chains. A very big ask. Unclear use, business, and communal cases and little activating legislation like dynamic pricing on real-time energy demands, make such changes very hard. Especially unclear are the effects on provision quality, energy savings, and the developments in the evolution of technology ecosystems, for example for federated mini-clouds on premises.

A spine could help. These issues are complex and are thought to need to be solved on an international level first. This can take decades, if it happens at all.

People don't like change. Really? People LOVE change, IF coming in the right proportion, at the right time and in the right way. If a proposed change is not being implemented, then the problem is with the change and not with people.

Apparently there are not enough champions that have the decision power and budget to take the initiative and steer towards sustainability. There seems to be a perceived need for a coordinated change, sharing costs/risks and preventing compartmentalisation and silos. When using a large cloud provider, users give their power to make sustainability changes away to the provider. One just has to follow. In the context of federated mini-clouds, we can perhaps find sufficient champions in sustainability NGO's.

What if ...

What if, instead of going (only) top-down, we could also go bottom up in some small organisations, preferably with off-the-shelf open source Cloud solutions such as OpenStack, and work towards an on-premises cloud, and find some other like-minded organisations to share resources with to create a small distributed cloud.

Then we can make guides, and include some of the monitoring, provision quality, and energy savings results giving similar other organisations more choices for their decisions on how to progress towards more sustainable set-ups. What if we can pave ways?


Raw magic crackled from their spines, earthing itself harmlessly in the copper rails nailed to every shelf for that very purpose. Faint traceries of blue fire crawled across the bookcases and there was a sound, a papery whispering, such as might come from a colony of roosting starlings. In the silence of the night the books talked to one another. A student