Preface
In the wave of new infrastructure construction, more and more enterprises begin to deploy digital business, and the amount of data is increasing rapidly. Data centers, which are used for data storage, processing and analysis, have entered a new round of accelerated construction period. In the energy-intensive data center industry, the importance of energy efficiency should never be underestimated. This poses a serious challenge to energy optimization.
?
As an active participant in new infrastructure construction, VNET Group has made extensive research and experiment on data center industry, and explored its unique path to ecological green development. By enhancing software development capacity, and introducing advanced technologies such as AIoT and new sensors, we continuously upgrade data center technology and management, optimize the PUE of data centers, and improve financial performance, in order to provide a strong impetus for the development of new infrastructure data center industry.
?
In the data center industry, engineers play a vital role in the design and deployment of projects, operation and maintenance, delivery and implementation, determine and guarantee the quality of products and services. “Engineer Gene” represents the highly internalized and refined values of VNET Group. “Engineer Spirit” is the fundamental and internal driving force behind the development of VNET Group. “Engineer Culture” is also the synonym of “pragmatic, hardworking, rigorous and enterprising employees” of VNET Group. Due to strong sense of responsibility, we constantly create and improve products and services, and pursue excellence.
In 2021, VNET Group adheres to lean operation and efficient management, constantly improves innovation ability and product power, cultivates high-quality talents and organizations, and pursues long-term and stable high-quality development.
?
Next, we will introduce the exploration and practices of VNET Group's engineers in data center operation, practice and management.
01 Establish the basic test environment of software and hardware
Data center is the core infrastructure of an enterprise. It is vital to ensure the safe and stable operation of data center and transform electric power into computing power efficiently, so as to ensure normal operation of the enterprise and maintain its core competitiveness. The infrastructure O&M engineers leverage the data center O&M system to monitor the data center, effectively master the overall situation of the data center, and timely identify and solve problems, thus ensuring the normal operation of the data center and the whole enterprise.
?
Focusing on customer experience, VNET Group assigns the scientific research team with deep professional background and broad vision to make work plans and business deployment. It takes the lead in dividing the data center facility O&M system architecture into maintenance and production system (MPS), maintenance and support system (MSS), and maintenance and analysis system (MAS). The evolution trend of MPS will be the digital twin and smart O&M of data center. The evolution trend of MSS will be data center-oriented digital infrastructure O&M and management, in order to realize standardized whole lifecycle facility management.
?
AIoT-based energy optimization takes MAS as an important support. VNET Group determines the work objectives of different stages: First, collect facility O&M data, continuously accumulate and establish an analytical data warehouse. Secondly, investigate the operation parameters of the cooling system under different loads and working conditions, including global energy consumption performance, system-grade COP performance, and equipment-grade energy consumption performance. Finally, establish the optimal facility O&M models under different system operation conditions and system loads, and predict the optimal facility working conditions.
Data Center O&M System Architecture
?
O&M analysis should be made after massive O&M data are collected. The standardized indicator system and O&M indicators can be defined after continuous collection, analysis and storage of O&M data.
?
02 Select optimized data acquisition terminal, procedure and communication
In terms of data acquisition equipment, VNET Group adopts Raspberry Pi as the data acquisition terminal, which has the advantages of small size and low power consumption. With abundant data interfaces, Raspberry Pi can be deployed next to the switch or directly connected with other underlying hardware for data collection at any time. Raspberry Pi can also run the operating system to complete more complex task management and scheduling, support the development of higher-level applications, and provide broad development freedom for program developers. In addition, it is completely isolated from the production system and support system, and can efficiently collect and analyze data without completely relying on the long-term upgrade process of the production system. As a result, data analysis and parameters adjustment can quickly form a closed-loop experiment.
?
Data Acquisition Equipment Deployment Architecture
?
In terms of data acquisition procedure, VNET Group uses the protocol stacks of script language and C language for efficient integration, thereby achieving efficient data acquisition, development, debugging and configuration. Script language is used to provide an execution environment for the communication protocol stack of different facilities, quickly analyze the communication messages of equipment and facilities, and provide data visualization graphical interface for equipment parameter models to optimize output results, so as to quickly adjust the experimental environment and parameters.
?
Predict the visualization interface of output results
?
In terms of data acquisition and communication, different communication protocols (such as Modbus, BACnet, OPC, etc.) are adopted due to the differences in manufacturers of cooling system equipment and designs. VNET Group adopts the protocol hardware gateway and protocol conversion software gateway developed by IoT innovation enterprises to solve this problem.
?
Modbus Splitter and RevolutionPi
03 Based on the total energy consumption of data center, explore the influencing factors that could be optimized
As mentioned above, we select an optimized basic test environment for each data center. After interpreting the energy efficiency indicator PUE, and establishing the PUE formula model, we take the cooling load factor (CLF), power load factor (PLF) and other load factor (OLF) as the key points of MAS energy decomposition and optimization. The key indicators of injection energy consumption are used for the system analysis.
?
In the process of further analyzing the approximate calculation of formulas and coefficients, the COP of the entire system plays an important role in the model. This is not only applicable to mature data centers, but also can evaluate and analyze the system cooling efficiency of new data centers. Due to the high utilization rate and long operation lifecycle, the PUE of mature data centers has little optimization space. However, the PUE of new data centers generally has a large optimization space due to the limited utilization rate and the lack of actual equipment capacity. Starting from the COP of the entire system, we can continuously optimize the energy efficiency of data centers when the utilization rate is low.
?
?
The changes of COP of the entire water-side system are calculated, tested and verified, combining with the factors such as chiller, cooling tower, cooling/ freezing pump, supply air temperature and chilled water temperature, as well as the characteristic curves of different equipment. Through continuous testing, analysis, summary and repeatedly verification, the following conclusions are summarized.
?
The supply air temperature in the cold channel affects the total energy consumption of data center.
?
The common servers in data centers include computing and storage servers. In the long-term operation and practice, through real-time collection and analysis of O&M data, engineers have conducted in-depth research on the relationship between the server energy consumption in the whole cooling system room and the overall energy consumption, and analyzed the server energy consumption at different inlet air temperature and loads.
?
?
If the inlet air temperature of the server is higher than 22 ℃, the fan energy consumption of the server will be on the rise. The engineers of VNET Group conducted experiments to study the relationship between the increase in the power consumption of the server itself, the decrease in the power consumption of cooling system and the total power consumption. We chose a data center building with high load rate and stable outdoor environment, and adjusted the temperature of chilled water (-2~+1 range), so as to adjust the cold channel temperature; and recorded the energy consumption of IT in the data center, cooling equipment, terminal temperature, humidity control equipment and server inlet air temperature within 24 hours.
?
After testing, in the experimental room, when the air inlet temperature of the server was lowered from 24 ℃ to 22 ℃, the total energy consumption of the room was reduced by 0.7%. In view of the high inflection point of air inlet temperature (27 ℃), the relationship between server energy consumption and total energy consumption needs to be verified by subsequent experiments.
?
?
Through continuous attempts and experiments, and taking the total energy consumption of the data center as the basis, we obtained the optimal practice results, provided decision-making basis for operation optimization, and further improved the energy efficiency utilization and reliability of the data center.
?
Relevant conclusions on energy consumption characteristics of chilled water system equipment
?
In the chilled water system of the data center, the changes in chilled water temperature or terminal loads will lead to the automatic adjustment of water valve of precision air conditioner, as well as the change of chilled water pump frequency. In the process of studying the cooling efficiency coefficient, engineers of VNET Group conducted in-depth research on chilled water pumps and precision air conditioners, in order to accurately analyze the factors influencing the cooling coefficients of water system.
?
A. Energy consumption characteristics of the chilled water pump
?
Most operators considered that the power and frequency of chilled water pump have a cubic relationship, and predicted that reducing the frequency of the chilled water pump will greatly reduce the energy consumption of the chilled water pump. In fact, the cubic relationship between the power and frequency of water pump only exists under standard working conditions.
?
*Calculation formula of pump shaft power:
N(KW)=Q(m3/h)*H(m)/(367*η)
Q: Pump flow
H: Pump lift
η: Pump efficiency.
?
Under standard working conditions, and when the pipe network characteristics remain unchanged, the relationship between shaft power and frequency of water pump is: P1/P2=N13/ N23.
?
The variable frequency control of chilled water pump in data center mostly adopts differential pressure control. When the water valve of terminal air conditioner is adjusted automatically, the terminal water flow will change, and the water pump will adjust automatically according to the target pressure difference, ignoring the influence of pipes and related equipment (which are located between terminal air conditioner and water pump) on pressure difference due to water flow change, and ignoring the change of pump efficiency. If the pump lift is approximately constant (H is a constant in the formula mentioned above), then it can be considered that the shaft power of the chilled water pump is approximately linear with the frequency of the pump (because H is a constant, there is a linear relationship between N and Q, and there is a linear relationship between Q and frequency, so there is a linear relationship between N and the frequency of the pump).
?
?
The above data shows the change trend of the pump shaft power when the chilled water pump runs automatically in a data center and the frequency of chilled water pump changes between 30-35Hz. The data indicates that, with the increase of the frequency of chilled water pump, there is a linear relationship between the change in the pump shaft power and the change of frequency. To maximize revenue, in the actual O&M practice, it is not necessary to excessively pursue frequency reduction in the operation of chilled water pump.
?
B. Energy consumption characteristics of precision air conditioner
?
The water valve control of precision air conditioner in the data center is mainly based on the pre-determined supply air temperature. Engineers of VNET Group have carried out experimental analysis on the relationship between the water valve opening range and the cooling capacity of precision air conditioner, as well as the influence of water flow velocity on heat transfer coefficient K.
?
In a data center with high load rate and uniform load distribution, the water valve opening range of 7 water-cooled precision air conditioners is 44% - 67%, and the fan speed is 60%-80% under stable working conditions. Through actual measurement and theoretical settlement, it was found that: when the water valve opening range is 60% and the fan speed is 70%, precision air conditioners have the highest cooling efficiency, and the maximum heat transfer coefficient K is obtained.
?
?
Meanwhile, the relationship between heat transfer coefficient and water flow velocity can be obtained from the following chart. When the water flow velocity is less than 1.3m/s, the heat transfer coefficient K increases with the increase of water flow velocity, and the heat transfer effect becomes better. According to our statistical data, water flow velocity is generally lower than 1.2m/s. Therefore, for the water velocity control of the data center, we should increase the water flow velocity as much as possible to increase the heat exchange efficiency of the precision surface air cooler.
?
Summary
?
Based on the research and practices of O&M analysis system and multi-directional energy consumption, engineers of VNET Group used big data technology and intelligent algorithms to analyze O&M data, and made the IT system operation more standardized, data-oriented, intelligent and automatic. In addition, with the application of O&M analysis system, the O&M personnel will not passively face system failures any more. Instead, they can actively analyze the system performance, focus on system architecture and business transformation, and be fully engaged in valuable and creative work, so as to vigorously support the business innovations of enterprises.
?
With the continuous deepening of new infrastructure, the data center will further adopt AI, big data, cloud computing and other new business forms, and become the data cornerstone of all industries. In view of the new requirements for data centers in the wave of new infrastructure constructions, such as high flexibility, green development and security, it is of great importance to implement excellent O&M management and balance the relationship between “stability” and “agility”. VNET group will continue to implement the wholesale plus retail dual-core strategy, inherit the engineer culture and craftsman spirit, guard against arrogance and impetuosity, concentrate on research and continuously make innovations and exploration, so as to effectively improve the user experience, and create a unique data center development path with distinctive characteristics of VNET Group.
