Architectural Considerations for Environmentally Sustainable Internet Technology

Potential Architectural Aspects This section presents architectural and protocol design aspects that can have an impact on the sustainability of networking. For each topic, we provide an overview, the motivation for why it would be important to consider for more sustainable networking, an analysis and recommendations for future networking professionals.

Measurement It is essential to understand the current state of affairs before any improvements can be made. i.e. Some levels of measurements are necessary for starting to improve sustainability. This is particularly the case when looking at the systems as a whole in post-analysis. As discussed earlier, this level of measurements is useful input for further actions, such as deciding what parts of the network need to be targeted for further improvement. But measurements may also be useful for some dynamic situations where power-saving decisions, for instance, depend on knowing the relative power consumption of different activities, such as when a power-off decision involves understanding the relative savings during the shutdown period vs. the power cost of shutdown and startup procedures, or the possible need to reconfigure other nodes in the network due to the shutdown.

Motivation Measurements are a necessary mechanism for both post-analysis and potentially for some of the dynamic decisions taken by systems. Without measurements of any kind, it is impossible to assess if the networks are functioning correctly. It is impossible to know if the system is efficient by comparing it against a baseline model. It is also impossible to check that changes aiming at optimizing something are indeed valuable. For instance, while electricity providers can make information about power usage available, this is only done at the aggregate level. Without per-device data about power usage, there would be limited basis for deciding where power is actually consumed and consequently, what improvements are most useful. At the same time, it is not possible to measure everything. Furthermore, any measurement must be validated. Relevance of measurements must be periodically assessed, e.g., with comparisons between measurements within a network and the aggregate numbers from the electricity provider. Finally, measurements made in the field must be collected and organized to allow later retrieval.

Analysis While the simplest forms of sustainability-related measurements are about power, there's clearly room for other measurements and other information as well. To begin with, power consumption by itself may not be what matters most for sustainability, as the source of the power may be equally important in terms of determining the actual carbon footprint. Secondly, for many classes of devices the embedded carbon aspects or use of raw materials may be a significant sustainability issue. See also Section 2.2. Third, power or energy measurements alone are of meager use if the cause of the consumption is not measured as well. Any power/energy measurement should occur alongside other measurements that can be used to determine energy efficiency. Hence a sound measurement architecture implies that a prior existence of an energy efficiency framework of some kind. But when it comes to energy consumption, as noted the aggregate information is often typically available, and it's not particularly hard to measure the energy consumption of individual network devices either. Still, there are a number of desirable use cases where the measurement situation needs to improve.

Measuring Power Efficiency When assessing the power consumption (Scope 2) of an IT-organization, emission accountants are generally looking for a metric of the delivered value per unit of energy. A commonly used method is to equate the delivered value with the number of bits sent or received, or to the communication capacity made available when there's a need for it. The latter is important, as often communication networks have requirements to be able to send messages when there's a need for it, e.g., for emergency communications, not that those messages are always being sent.

Recommendation Ongoing work at the IETF's GREEN working group is already targeted at improving existing energy consumption metrics and frameworks but some further considerations may apply. In order to meet the needs discussed above, the following architectural design principles are proposed.

Generality We recommend that any measurement framework or sustainability-related information sharing mechanism be designed to share different types of information and not limited to a single metric such as power consumption. Similarly, the granularity of data collection needs to be configurable so that the metrics collected can be as fine-grained or as aggregated as needed in order to identify potential areas of improvement.

Collect Metrics from Existing Equipment Since the need to deliver on the use cases described is urgent, the industry has to accomodate the capabilities (and limitations) of existing equipment in the field for collecting metrics. It is recommended to have a plug-in architecture with modules that can work with (read from and control) devices of any kind, including traditional networking hardware devices, cooling systems, software stacks, and occasionally static datasheets.

Content Declaration for all Collected Metrics A warehouse filled with data collected from diverse sources is useless without proper labeling. Hence, these is a need to create metadata that describes the collected data. (e.g. What are the source(s)? What measurement units are used? Precision? What is included/excluded in these numbers?) The metadata itself must also have a formal description. e.g. Use YANG for the metadata schema. Keep the metadata attached to the dataflow it describes, so that the relation is clear to each component that has anything to do with it, including components added by other organizations at a later point in time.

Collection, Aggregation, Processing, Display, Decisions The collected data passes through a pipeline from collection to decisions. By processing we mean steps to reshape the data to match further aggregation and processing steps, such as unit conversions, sample frequency alignment, filtering, etc. Separate these architectural roles into separate modules in order to enable reuse, modular development and a transparent, configurable pipeline.

Configurable Pipeline for Reuse and Transparency Let the pipeline connections between the components be driven by configuration rather than hard coded. This enables reconfiguration of the processing pipeline over time, and perhaps more importantly, transparency into what stages the data pass through, even without access to or understanding of the source code of the entire system.

Design Together with the Users Every system should be designed involving some of its target users. In order for delivered metrics to be of any value, the target audience needs to be aware of their existence, be able to interpret them and understand how they can be used in their professional context. There are many target user groups for the information produced. Some examples are network designers/engineers, scientists, operations teams and IT-development organizations. One critical group that is often overlooked is the sustainability assessment experts. If they are not aware, don't understand or don't care about the produced sustainability metrics, the value of this work would be greatly diminished.

Modeling The paucity of up-to-date information on equipment and system parameters, especially power consumption and maximum throughput, makes estimating the power consumption and energy efficiency of these systems extremely challenging. In addition the rapid evolution of technology and products in ICT makes the estimation quickly outdated and possibly inaccurate. In almost all cases physical measurement has to be replaced by partial measurement and mathematical modeling.

Motivation Where power optimization choices are made, accurate information is required to decide the right choice. Modeling instead of measurements may have to be used in some cases.

Analysis To date, two approaches to network power modeling are accepted as providing a realistic estimate of network power consumption. These approaches are referred to as "bottom-up" and "top-down". The paper surveys both approaches and provide a new approach which unifies both of them. The unified approach is used to estimate the power consumption of access, aggregation and core networks. The paper provides a model for IP Routers and the routers of other future Internet architectures (FIA) such as SCION and NEBULA. They use a generic model which captures the commonalities of IP router as well as the peculiarities of FIA routers. They conduct a large-scale simulation based on this router model to estimate the power consumption for different network architectures. Since routers and other network devices and functions can be virtualized, this article (1) provides comprehensive "graphical, analytical survey of the literature, over the period 2010–2020, on the measurement of power consumption and relevant power models of virtual entities as they apply to the telco cloud." This paper A Methodology and Testbed to Develop an Energy Model for 5G Virtualized RANs IEEE Conference Publication IEEE Xplore got best paper award for GreenNet 2024, but I am not sure if we are interested to model 5G vRAN. There is a plethora of publications on modeling communication networks and DC computing.

Customer Attribution When organizations assess their Scope 3 emissions, they need to sum up their share of emissions from all their suppliers, one of which for example, might be a cloud hosting service. In order for the supplier to provide an emission share value back to the customer, the provider needs to develop a mechanism for attribution. A significant challenge in accurately assessing Scope 3 emissions is avoiding Double Counting, where the same emission is reported by multiple entities. According to the GHG Protocol best practices, it is crucial to establish clear guidelines and agreements between suppliers and customers to ensure that emissions are attributed correctly and not counted multiple times. This requires transparent communication and precise emission reporting standards to ensure that all parties involved have a consistent understanding of which emissions belong to which organization. By addressing the Double Counting issue, companies can achieve more accurate and reliable Scope 3 emissions assessments, thereby contributing to better overall sustainability reporting and improvement efforts.

Baselining and Benchmarking Establishing a baseline is a fundamental step in the process of improving energy efficiency and sustainability of network technology. Baselining involves establishing a reference point of typical energy usage, which is crucial for identifying inefficiencies and measuring improvements over time. In this step, the controller uses only the collected data from datasheets and other reliable sources. By establishing a baseline and using benchmarking, organizations can determine if their networking equipment is performing normally or if it is deviating from expected performance. This is the first step in identifying and guiding necessary improvements. Benchmarking involves collecting performance measurements of networking equipment under controlled conditions. This process helps establish standardized performance metrics, allowing for comparison against baselines collected during regular operational conditions. The initial measurement of networking equipment's energy efficiency and performance, known as Baselining, should be coordinated with vendor specifications and industry standards to understand what is considered normal or optimal performance. For example, if the baseline indicates that your switches operate at 5 Gbps per watt, while vendor specifications suggest 8 Gbps per watt and the industry standard is 10 Gbps per watt, actions should be taken to implement energy-saving measures and upgrades. Continuously tracking subsequent measurements can reveal if efficiency improves towards the benchmark of 8-10 Gbps per watt. This practice ensures that any improvements can be quantifiably tracked over time, providing a clear measure of the effectiveness of the implemented changes and guiding further enhancements in network sustainability. See also and .

Recommendation Even though baselining is essential in identifying potential areas of improvement and tracking progress, it is still to be determined to what extent we need to work on modeling networks and devices in the architecture.

Dynamic Scaling Dynamic scaling is the ability to adjust resources according to demand, and possibly turn some of them off during periods of low usage. Examples include the set of servers needed for a service, how many duplicate links are needed to carry high-volume traffic, whether one needs all base stations with overlapping coverage areas to be on, etc. Networks and communications are also critical functions of the modern digital society. The reliability of individual networking links or devices cannot always be guaranteed. As a result, various levels and forms of resiliency are often needed, for instance through redundancy. Yet, there is a question on how much redundancy is needed and how quickly a backup or resource increase can be activated due to increased demand.

Motivation Outside of implementation improvements, dynamic scaling is potentially the most promising method for reducing power consumption related environmental impacts. Scaling can happen on a device-level (increasing performance as traffic levels grow) or a network segment level (increasing the number of active links or cellular base stations). Considering current fixed networking hardware, dynamic scaling might not have an impact in situations where there's only a single router or server serving a particular route, area, or function. Current routers and switches exhibit limited potential dynamic scaling because the focus is on high performance and a stable connectivity. There have been some recent improvements on this front as well. e.g. Energy-Efficient Ethernet (EEE) is a good example of a networking-level specification to lower energy consumption in idle mode. EEE has limited impact on a network that has continuous traffic. Resiliency can be implemented within a single router as well, e.g. as a backup power supply, between routers and switches as multiple links between the same nodes, having different links between two end points, overlapping cellular coverage, etc. All these necessarily add more hardware to provide the same exact service. Some of that hardware can be fully operational at all times and used to serve the traffic, while other links may be in hot or cold standby depending on the use case. Cellular networks are typically built with significant overlap in coverage areas of multiple base stations. Demand and business reasons dictate the design of the coverage, and regulations might dictate how reliable the cellular service should be. There is extensive work world-wide to optimize the operation of this overlapping coverage, e.g. by turning down some sites at night time when traffic volumes are low. A cellular basestation site can consume anything from a few kWh to ten or more kWh per provider. Modern cellular base stations do implement numerous features to scale the energy consumption. In general, cellular base stations have a base energy consumption and traffic-dependent consumption, a somewhat similar behavior to what we can observe in modern CPUs. On the network level, most large systems have significant amount of redundancy and spare capacity. Where such capacity can be turned on or off to match the actual need at a given time, significant reductions in power consumption can be achieved.

Analysis Dynamic scaling could be seen as either an alternative or complementary to load stabilization, e.g., via "peak shaving". Perhaps the most realistic angle is that both are likely needed. The most rudimentary approach to dynamic scaling is just turning some resources off. However this may not be sufficient, and a more graceful/engineered approach potentially yields better results. Network architects need to understand the impacts of scaling changes on users and traffic. These may include the fate of ongoing sessions, latency/jitter, packets in flight, or running processes, attempts to contact resources that are no longer present, and the time it takes for the network to converge to its new state. Dynamic scaling requires an understanding of load levels for the network, so information collection is required. It also requires understanding the power, time and other costs of making changes. (See for discussion of tradeoffs and multi-objective optimization.) Understanding the resiliency requirements for a network or a piece of equipment is also important for the optimal control of resiliency, e.g., as an input to decisions on how many instances of replicated services need to be run and where. Some of the strategies that are useful in implementing a well working dynamic scaling include:

Matching the currently used resources to the actual need, be it about traffic demand or resiliency. One way to do this is to use of power-proportional underlying technologies, such as chipsets or transmission technologies. And where this is not sufficient, the ability to turn components/systems on and off is an alternative strategy.
Using load adaptive techniques allows the capacity of the nodes to be dynamically adjusted according to the demand. Examples include Adaptive Link Rate (ALR), which dynamically adapts the link rate to suit traffic demand or power off links in Link Aggregation based on traffic demand which is empirically estimated based on traffic arrival. LACP (Link Aggregation Control Protocol) defined in IEEE 802.1AX can be modified to power off links in an aggregation if they are not needed.
Ability to enter "no new work" mode for equipment, to enable some resources to be eventually released/turned off.
Ability to move ongoing tasks off to other equipment, to prevent disruption of already started tasks.
Ability to schedule changes in advance rather than making them abruptly, with associated signaling exchanges and possible transient routing and other failures. See for instance the time-variant routing work in the IETF .
Efficient propagation of changes of new routes, new set of servers, etc. as to reduce the amount of time where state is not synchronized across the network. The needs for the propagation solution needs to be driven by dynamic scaling and sustainability as well as other aspects, such as recovery from failures.
Build mechanisms to deal with dynamic changes: Plan for dynamic set of resources, and not expect to work with a fixed set of resources.
Dynamic scaling requires automation in most cases, e.g., to turn on new service instances. See again for a discussion of automation.
Interaction with the energy grid can enable dynamic load shifting. For instance, a demand-response technique can be used where the system temporarily reduces its energy usage in response to pricing signals from a smart grid. The proposed demand-response technique involves deferring the load from elastic requests to later time periods in order to reduce the server demand and the current energy usage, and hence, energy costs .
Energy-aware routing. This generally aims at aggregating traffic flows over a subset of the network devices and links, allowing other links and interconnection devices to be switched off. These solutions shall preserve connectivity and QoS, for instance by limiting the maximum utilization over any link, or ensuring a minimum level of path diversity. There are also algorithms for Green Traffic engineering. For instance employs segment routing. Experimental analysis results show that the resource usage for SRv6 could be more than 70% lower than that of the SPF-based forwarding, depending on the network topology.

Recommendation The guidelines above need to be considered specifically for each protocol and system design. Further work in detailing this guidance would also be useful. It is likely that there is increased attention to resiliency in the future, given for instance the increased importance of the tasks supported by networks or the potentially increasing frequency of natural disasters as a result of global warming.

Transport Transport protocols are the flexible tools that make it possible for communication flows between parties to adjust themselves to the dynamic conditions that exist in the network at any given time: available bandwidth, delays, congestion, the ability of a peer to send or receive traffic, and so on. Depending on the conditions, an individual flow may carry traffic at widely different rates, may pause for some time, etc. Various higher-level transport solutions may also cache or pre-fetch information. This behavior has an effect on sustainability as well, e.g., in what periods the endpoint and network systems are active or when they could be in reduced activity or sleep states. Cellular networks and mobile links can scale their energy usage based on load and enter a low-power state when a traffic flow ends. Thus, in theory, the faster the data is transferred, the faster the device transmission/reception functions can enter a low-power state.

Motivation Transport behavior would have a possibility of impacting how much downtime or sleep can be had in the communication system, either on the end systems or routers or other equipment in between. The savings can be significant, at least in wireless systems. Improvements through transport behavior are only useful if the involved systems have power proportionality.

Analysis A critical issue is the tradeoff involved in sending traffic. As argued in , reducing the amount of time the endpoints and the network are active can sometimes help save energy, e.g. in case the receiver is connected over a WiFi link. Similar logic applies for any technology that has a certain degree of energy proportionality, e.g. cellular communication. As a result, in general, delivering information as rapidly as possible would appear to be desirable. On the other hand, bandwidth-intensive applications can influence other applications or users by presenting a significant load on the network, and consequently reducing capacity available for others, or increasing buffering (and with it, latency) across the network path. For an application with intermittent data transfers, such as streaming video, this would seem to speak in favor of sustained but lower-rate delivery instead of transmitting short high-rate bursts . However, this is in contradiction with the energy-saving approach above. Thus, the tradeoff is: should data be sent in a way that is "friendly" to others (avoiding bad interference), or should it save energy by sending fast, increasing the chance for equipment to enter a "sleep" state? At the time of writing, the common choice for video is to opt for higher rate delivery, potentially saving energy, and possibly at the expense of other traffic. For non-urgent data transfers, the IETF-recommended default approach is the opposite: the LEDBAT congestion control mechanism , which is designed for such use, will always "step out of the way" of other traffic, giving it a low rate when it competes with any other traffic. Alternatively, if the goal is to reduce energy, such traffic could be sent at a high rate, at a strategically good moment within a longer time interval; this would give network equipment an opportunity to enter a sleep state in the remaining time period within the interval. Perhaps the issue is that the transport behavior (as with many other things) needs to take into account multiple parameters. For example, it is possible that a balanced transport algorithm would be able to send as much as possible as soon as possible, while tracking buffer growth from transmission delays and scaling back if there's any buffer growth. This remains to be confirmed with experiments, however. Similarly, caching and pre-fetching designs need to take into account not only the likelihood of having acquired the right content in memory, but also the sustainability cost of possibly fetching too much or the timing of those fetching operations. In general, information about the impacts of loading or not loading the network with additional traffic, and whether a certain sending pattern enables power savings through sleep modes, would be beneficial for the communicating endpoints. Mechanisms for making such information available to the endpoints would be useful.

Recommendation The techniques described above have been based on theoretical analysis. There is a need for further simulations and experiments to confirm what strategies would provide the best end-user and energy performance. This may be work that fits within the IRTF SUSTAIN research group.

Equipment Longevity This section discusses the ability to extend the useful life of protocols and/or network equipment in order to amortize the embedded energy costs over a longer period, even though it may mean that the protocols/equipment may not be fully optimized for the present use. This includes devising tools to inform network administrators and their users of the potential benefits of network equipment upgrades, so that they can make better choices on what upgrades are necessary and when. It should be noted that from an environmental sustainability perspective, it may not always be the best choice to upgrade network equipment whenever slightly less power-hungry and "greener" alternatives become available. The environmental cost of amortizing the carbon embedded inside equipment over its lifetime, including the carbon associated with the manufacturing of the equipment that is to be replaced, should be taken into consideration as well.

Motivation Embedded carbon and raw materials can be a significant part of the overall environmental impact of systems. If this can be improved for devices that are manufactured in large quantities, the improvements can be significant. The more the world moves toward low-carbon energy sources, the more the manufacturing matters in the holistic view. Today there can be an order of magnitude difference in average emissions for a kWh of electricity between two countries. Thus, any estimates that seek to compare the manufacturing and use phase emissions of a network equipment would have to be calculated per country or region, and there is no universal standard for the whole planet. Long equipment lifetimes are only useful if the longer lifetimes can be achieved without compromising other aspects of sustainability, such as when using a high-end and power-hungry router in place of small routers. The exact moment when a hardware change is warranted for sustainability differs between countries and regions.

Analysis When we engineer protocols and network equipment, we are inclined to design them in a highly optimized manner for a very specific set of requirements, use cases and context. While this is necessary in certain cases (e.g. constrained nodes with limits on processing capacity or long lived battery powered devices), there are certainly cases where such optimized equipment is not absolutely required. Most infrastucture network nodes on the Internet utilize only a fraction of their design capacity most of the time. Designing the equipment with an eye on longevity comes with a set of advantages:

It allows the same equipment and protocols be reused in a different context in the future. e.g. A core router of today can become an edge router in a near future and an access router in the further future if the protocol implementations are adaptable.
It can reduce complexity in implementations as well as in network management that are usually indicated in highly optimized systems
It can let network equipment operate for a longer period and can reduce the frequency of hardware upgrades, in turn reducing the environmental impact associated with manufacturing, transporting, and disposing of the old/new hardware.
One key disadvantage may be that not optimizing may result in the need for premature upgrades for capacity and this needs to be considered.

Hence, it is very likely that extending the life of protocols and equipment with higher flexibility could provide a better environmental benefit than tightly optimizing only for today’s uses. Another aspect that can play an important role in extending the longevity of equipment concerns software-defined networking, in the sense of designing networking equipment in such a way that new equipment capabilities and features can be introduced via software upgrades as opposed to requiring hardware replacement. This requires system architectures that incorporate the necessary infrastructure to support such upgrades in a secure manner that does not compromise equipment integrity.

Recommendation The guidelines above should be considered for any new system design. If some aspect of protocol or network equipment design choice could be made more generic and flexible without a significant performance and sustainability impact, it needs to be studied in further detail. Specifically, the potential additional sustainability costs due to forgoing optimization need to be weighed against the potential savings in embedded carbon and raw material costs brought about by premature upgrades. There are also cases where equipment upgrades are done to provide better peak performance characteristics (e.g. higher advertised speeds towards consumers) and these need to be viewed as well with the same tradeoffs in mind. Finally, when designing networks it is recommended to consider whether it is possible to reuse retiring equipment in a different location or for a different function (e.g. move it to lower traffic geographies, core routers become edge/access routers etc.)

Compact encoding This is about considering the effects encoding methods on sustainability, such as the use of binary encodings instead of text.

Motivation Better encoding can obviously reduce the length of messages sent. It remains a question mark how big overall impact this is, however. It should only be performed if it gives a measurable overall impact.

Analysis Better encoding methods are clearly beneficial for improving the detailed-level effectiveness of communications. The main questions are, however:

Is the effect of this is at a magnitude comparable to the other things, or if it is just absolutely tiny? Particularly considering that much of the traffic on the Internet is video, and much of that is other content than, e.g., HTTP headers. Moran et al. argued in their 2022 paper that that for a weather data example from there are significant savings. However, this needs more research in terms of the overall impact across different examples and the general make up of Internet traffic.
At what layer is the compactness achieved? Are link, IP, or transport layer mechanisms that can compact some of the verbose messaging useful, or should each protocol have optimal compacting?
Tradeoffs related to compressing (particularly if AI-based computationally expensive methods are used).

Recommendation More research is needed to quantify the likely sources of measurable impacts. Of course, new protocols can generally be designed to work with compact encoding, unless there is a significant reason not to. But efforts to modify existing protocols for the sake of encoding efficiency should be further investigated by the above mentioned quantification results.

Sustainable by Design: Data Governance Perspective Incorporating sustainability into the design phase of network architecture is critical for ensuring long-term environmental and operational benefits. From a Data Governance point of view, "Sustainable by Design" involves embedding sustainability principles and practices into the data management frameworks and processes from the outset.

Motivation Data governance plays a pivotal role in shaping how data is collected, stored, processed, and used. By integrating sustainability into these processes, organizations can ensure that their data practices contribute to environmental goals, such as reducing carbon footprints, optimizing resource usage, and minimizing waste.

Analysis Key elements of Sustainable by Design in data governance include:

Data Minimization: Collecting only the data that is necessary and useful, reducing storage and processing requirements, which in turn lowers energy consumption.
Efficient Data Storage Solutions: Implementing energy-efficient data storage technologies and practices that prioritize reduced power usage and cooling needs.
Lifecycle Management: Ensuring that data is managed throughout its lifecycle in a way that minimizes environmental impact, including secure and sustainable data disposal practices.
Transparency and Accountability: Establishing clear data governance policies that promote transparency in data usage and accountability for sustainability objectives.

Recommendation Organizations should adopt data governance frameworks that incorporate sustainability as a core principle. This includes setting clear sustainability goals, measuring progress towards these goals, and continuously improving data management practices to enhance sustainability. By doing so, organizations can ensure that their data operations are not only effective but also environmentally responsible.