CARBS is good for you

THE CARBS PROJECT will employ inexpensive hardware that makes use of internal server system metrics to enable more accurate measurement of power usage within systems and across hardware domains. Working with state-of-the-art components from Concurrent Thinking who are willing to provide technical resources and input into this project in order to prove the benefits of such a model.

Using this model a dashboard can be created to provide real-time measurement of the environmental (carbon) cost of delivering individual services that have been identified in this way.
Aims and objectives
 Hardware and software tool for providing real-time financial & carbon costing of two or more internally delivered IT services
 A report on the comparisons of output from JISC baselining resources and actual results as calculated by the tool
 A report on the experiences of power & carbon cost accounting for services and its comparability to the costs of external service provision through the cloud
 Public blog tracking the path through the project and learning along the way
 Presentation(s) to the JISC community
 To work with the JISC community and wider sector to share the experiences and approaches to assist other organisations to benchmark, further develop carbon cost accounting approaches for IT services
Context
Related Projects
 The project will take account of the
planned IH Project Portfolio. This may
result in resource constraint.
Constraints
 Which services can be chosen for this
project will depend on a number of
factors, including availability of compatible
PDUs, sufficient power and networking
availability and systems expertise.
 All changes that are service affecting (if
any) must be scheduled to take place out
of hours and must avoid critical periods.
Assumptions
 Staff expertise and availability will be
provided to meet the timelines.
The business case
The project supports the following UH initiatives:
 The Carbon Management Plan
 KPIs for Sustainability
Key drivers
 Assessments of cloud computing versus
running services in-house are required for
all new service acquisitions/replacements
 Running services more efficiently
 Making best use of existing scarce
resources
 Understanding what is driving power
usage within the university’s data centres
 Providing capacity planning data for future
data centre requirements
JISC resources/technology used
This project will relate to other relevant JISC resources developed for ICT Carbon measurement and baselining. With the CARBS model in place, the accuracy of the estimation tools provided by, for instance, the Suste-IT and StorC projects can then be validated and guidance given in how these might be tweaked/combined to provide a more realistic, holistic picture of the environmental baseline of an organisation’s ICT services.
Outcomes

Achievements
 Complete carbon and power cost
accounting for the Voyager book-lending
system
 Successful implementation of the
infrastructure required to provide carbon
and cost accounting for most services
within the university’s data centres
 A thorough assessment of SusteIT’s
carbon footprinting benchmarking tool
against real-world data
 Learning documented throughout the
project in our blog (http://blogs.herts.
ac.uk/carbs/)
 Increased awareness of sustainability
efforts at the university
 Increased focus on carbon costing during
project initiation
Benefits
With the framework now in place, this method can be applied to other similarly-architected services in order to provide carbon cost accounting with an estimated error margin of +/-5%
The assessment exercise for the benchmarking tool would be valuable to those who feel using the default values provided within it would result in a blunt instrument for guesstimating overall power and carbon usage within the data centre. In particular, the simple use of PUE, which is commonly calculated in most data centres, to cost out the impact of a server on its environment is a valuable one.
In any organisation, financial decisions concerning the ICT estate are made all the time based on data that are either incomplete or missing concerning the true costs associated with owning and delivering those services. For the first time, we have been able to picture what the cost to the university might be of running a particular service in-house. Future decisions around the sustainability and viability of continuing to operate in this way rather than make more use of cloud services can be taken in the presence of greater information.
As we continue to monitor and measure the costs, more complete profiling of data centre service costs will be built up. Dramatic changes can be investigated and with the tools to provide a greater degree of detailed data, better efficiency can be architected in.
Server costing can be used as a justification for replacement or sharing of hardware resources in a virtualised environment.
Drawbacks
Many assumptions need to be made when apportioning costs from various hardware and software platforms for any particular service. Even the simplest service may require a certain amount of assumptive reasoning that could be open to debate as to how accurate a picture it gives.
Data from PDUs that only provide amperage output, and minimal separate distinct circuits, can cause problems when trying to assess what power is being used at the service level. Some systems, such as storage, cannot be interrogated for power usage data and therefore reliance on the PDUs is complete.
Apportioning costs of the network is complex and fraught with danger. Networks can ‘self-heal’, causing the traffic patterns to change and result in an inaccurate picture of what network systems are in use and to what extent.
Technology moves on, changes occur frequently, software develops with functions being both added and removed, sometimes changing the architecture of a service in ways that the original setup may not have envisaged. For this reason, carbon accounting for services may be liable to disruption, raising the possibility of inaccurate or incomplete values being reported on the dashboard. For these reasons, this is not an exercise that can be set up and left running with no further work. Regular analysis of both the service and the data emanating from the service should be undertaken to ensure that the reported values are still valid.
Key lessons
I have divided the key lessons into Pre-Project, During Project, and Post Project, so that they can be followed more easily by those looking to do the same thing as us.
Pre-Project
 Don’t assume this is going to be easy!
 Divide the project up into small bite-
size chunks, where you can prototype the
dashboard creation a service at a time.
 It is important to scope out the complete
list of devices and software components
(i.e. databases, applications, etc.) you
want to monitor. This ensures you are
prepared to grab data from whatever
sources you need to but also highlights
areas where data capture may be either
very difficult or impossible without carrying
out significant changes beforehand.
 Resolve any issues with devices not
having SNMP-capability prior to install.
We had power meters that said they were
NMP-capable but actually were not! They
were compatible with MODBUS though,
so we purchased a converter appliance.
 Have all necessary MIB files collated and
ready to be installed.
 Check dependencies within MIBs and do
a little research on them to ensure that
they install correctly. You may need to
contact the vendors to check that the
values you want to extract are actually
vailable in the MIB.
 Check through the MIBs and make a
note of the OIDs you need to select.
This is extremely useful when you come
to program the DCIM
 Have a complete audit of your data
centre to hand, including positioning of
racks, e quipment within them and
number and positioning of power
supplies. Very useful if you wan a visual
output as well as a dashboard-style
display. Most DCIMs allow you to do this.
 Move power supplies for servers on to
distinct circuits or sockets depending
on to what level of detail your ePDUs are
capable of monitoring down to. Otherwise
you will not be able to extract specific
power data for individual devices.
During the project
 Keep stakeholders informed so that they
understand the value they get from the
project and provide information, or
changes that support your project. The
server we wanted to monitor required
a patch upgrade before we could get the
specific, and only currently available, MIB
to work with it.
 Manage changes to the data centre
carefully or what you thought you were
monitoring may change. This means
nsuring that your data centre users are
aware that if they make specific changes
to the environment, your monitoring will
fail or give off spurious results. This could
invalidate your testing.
 This is an immature industry and DCIM
products are still being developed to their
full potential. Work closely with your DCIM
vendor and report back issues and chase
for prompt resolution.
Post-project
 Manage changes to the data centre strictly
or what you thought you were monitoring
may change. This means ensuring that
your data centre users are aware that
if they make specific changes to the
environment, your monitoring will fail or
give off spurious results. This may
invalidate your dashboards.
 Check validity of reported values on a
regular basis – note any significant
changes and investigate whether the
service has changed
 Let stakeholders know.
 Make sure all future data centre
procurement takes into account SNMP-
compatibility and functionality that makes
separating service-related data out simply
and easily.
 SusteIT’s Carbon Footprinting Tool can be
made even better by applying a few
lessons from our real-life assessment
exercise:
 Use PUE to provide facility overhead
 Add in Carbon cost to get a true picture
 Don’t trust the server estimators, test one of each type of server and use this
base data to provide your wattage
figures
Looking ahead
The project will not end here – there are already plans to continue to develop the framework, tackling more difficult services and building greater expertise and greater accuracy in costing out the carbon and power usage for individual services and the data centre as a whole.
Further objectives for the project:
 Build the framework for services running
in a heavily virtualised environment
(VMWare)
 Account for network usage in the carbon
cost
Sustainability
The project has justified itself as a useful tool both for production operations in the data centres but also for giving senior management better insight into the true cost of hosting internal services.

The application will continue to be developed to increase its value through greater functionality improvements and reporting of environmental and service-related data.
I expect to present the outcomes and learning of the project at various workshops, seminars and conferences both within, and external to, the HE/FE sector in the coming year. I have already provisionally signed up to present this case study at Gartner’s European IOM conference in 2014.
The project has also agreed to work with JISC’s CITS (Costing IT Services) project (http://www.jisc.ac.uk/whatwedo/programmes/greeningict/organisational/itcosts.aspx), complementing the accounting work already achieved there, with real-life operational cost examples, based on algorithms for apportioning cost across physical hardware sets within the data centre.