POWER8

POWER8

Two IBM POWER8 processors on a dual chip module.
General information
Launched	2014
Designed by	IBM
Performance
Max. CPU clock rate	2.5 GHz to 5 GHz
Cache
L1 cache	64+32 KB per core
L2 cache	512 KB per core
L3 cache	8 MB per chiplet
L4 cache	16 MB per Centaur
Architecture and classification
Technology node	22 nm
Instruction set	Power ISA (Power ISA v.2.07)
Physical specifications
Cores	6 or 12
History
Predecessor	POWER7
Successor	POWER9

IBM Power E870 can be configured with up to 80 POWER8 cores and 8 TB of RAM.

POWER8 is a family of superscalar multi-core microprocessors based on the Power ISA, announced in August 2013 at the Hot Chips conference. The designs are available for licensing under the OpenPOWER Foundation, which is the first time for such availability of IBM's highest-end processors.^[1][2]

Systems based on POWER8 became available from IBM in June 2014.^[3] Systems and POWER8 processor designs made by other OpenPOWER members were available in early 2015.

Design

POWER8 is designed to be a massively multithreaded chip, with each of its cores capable of handling eight hardware threads simultaneously, for a total of 96 threads executed simultaneously on a 12-core chip. The processor makes use of very large amounts of on- and off-chip eDRAM caches, and on-chip memory controllers enable very high bandwidth to memory and system I/O. For most workloads, the chip is said to perform two to three times as fast as its predecessor, the POWER7.^[4]

POWER8 chips comes in 6- or 12-core variants;^[5][6] each version is fabricated in a 22 nm silicon on insulator (SOI) process using 15 metal layers. The 12-core version consists of 4.2 billion transistors^[7] and is 650 mm² large while the 6-core version is only 362 mm² large.^[3] However the 6- and 12-core variants can have all or just some cores active, so POWER8 processors come with 4, 6, 8, 10 or 12 cores activated.

CAPI

Main article: Coherent Accelerator Processor Interface

Where previous POWER processors use the GX++ bus for external communication, POWER8 removes this from the design and replaces it with the CAPI port (Coherent Accelerator Processor Interface) that is layered on top of PCI Express 3.0. The CAPI port is used to connect auxiliary specialized processors such as GPUs, ASICs and FPGAs.^[8][9] Units attached to the CAPI bus can use the same memory address space as the CPU, thereby reducing the computing path length. At the 2013 ACM/IEEE Supercomputing Conference, IBM and Nvidia announced an engineering partnership to closely couple POWER8 with Nvidia GPUs in future HPC systems,^[10] with the first of them announced as the Power Systems S824L.

On October 14, 2016, IBM announced the formation of OpenCAPI, a new organization to spread adoption of CAPI to other platforms. Initial members are Google, AMD, Xilinx, Micron and Mellanox.^[11]

OCC

POWER8 also contains a so-called on-chip controller (OCC), which is a power and thermal management microcontroller based on a PowerPC 405 processor. It has two general-purpose offload engines (GPEs) and 512 KB of embedded static RAM (SRAM) (1 KB = 1024 bytes), together with the possibility to access the main memory directly, while running an open-source firmware. OCC manages POWER8's operating frequency, voltage, memory bandwidth, and thermal control for both the processor and memory; it can regulate voltages through 1,764 integrated voltage regulators (IVRs) on the fly. Also, the OCC can be programmed to overclock the POWER8 processor, or to lower its power consumption by reducing the operating frequency (which is similar to the configurable TDP found in some of the Intel and AMD processors).^{[12][13][14][15]}

Memory Buffer chip

POWER8 splits the memory controller functions by moving some of them away from the processor and closer to the memory. The scheduling logic, the memory energy management, and the RAS decision point are moved to a so-called Memory Buffer chip (a.k.a. Centaur).^[16] Offloading certain memory processes to the Memory Buffer chip enables memory access optimizations, saving bandwidth and allowing for faster processor to memory communication.^[17] It also contains caching structures for an additional 16 MB of L4 cache per chip (up to 128 MB per processor) (1 MB = 1024 KB). Depending on the system architecture the Memory Buffer chips are placed either on the memory modules (Custom DIMM/CDIMM, for example in S824 and E880 models), or on the memory riser card holding standard DIMMs (for example in S822LC models).^[18]

The Memory Buffer chip is connected to the processor using a high-speed multi-lane serial link. The memory channel connecting each buffer chip is capable of writing 2 bytes and reading 1 byte at a time. It runs at 8 GB/s in the early Entry models,^[17] later increased in the high-end and the HPC models to 9.6 GB/s with a 40-ns latency,^[18][19][20] for a sustained bandwidth of 24 GB/s and 28.8 GB/s per channel respectively. Each processor has two memory controllers with four memory channels each, and the maximum processor to memory buffer bandwidth is 230.4 GB/s per processor. Depending on the model only one controller might be enabled,^[17] or only two channels per controller could be in use.^[18] For increased availability the link provides "on-the-fly" lane isolation and repair.^[16]

Each Memory Buffer chip has four interfaces allowing to use either DDR3 or DDR4 memory at 1600 MHz with no change to the processor link interface. The resulting 32 memory channels per processor allow peak access rate of 409.6 GB/s between the Memory Buffer chips and the DRAM banks. Initially support was limited to 16 GB, 32 GB and 64 GB DIMMs, allowing up to 1 TB to be addressed by the processor. Later support for 128 GB and 256 GB DIMMs was announced,^[19][21] allowing up to 4 TB per processor.

Specifications

The POWER8^[22][23] core has 64 KB L1 data cache contained in the load-store unit and 32 KB L1 instruction cache contained in the instruction fetch unit, along with a tightly integrated 512 KB L2 cache. In a single cycle each core can fetch up to eight instructions, decode and dispatch up to eight instructions, issue and execute up to ten instructions and commit up to eight instructions.^[24]

Each POWER8 core consist of primarily the following six execution units:

• Instruction fetch unit (IFU)
• Instruction sequencing unit (ISU)
• Load–store unit
• Fixed-point unit (FXU)
• Vector and scalar unit (VSU)
• Decimal floating-point unit (DFU)

Each core has sixteen execution pipelines:

• Two fixed-point pipelines
• Two load-store pipelines
• Two load pipelines
• Four double-precision floating-point pipelines, which can also act as eight single-precision pipelines
• Two fully symmetric vector pipelines with support for VMX and VSX AltiVec instructions.
• One cryptographic pipeline (AES, Galois Counter Mode, SHA-2)^[25]
• One branch execution pipeline
• One condition register logical pipeline
• One decimal floating-point pipeline

It has a larger issue queue with 4×16 entries, improved branch predictors and can handle twice as many cache misses. Each core is eight-way hardware multithreaded and can be dynamically and automatically partitioned to have either one, two, four or all eight threads active.^[1] POWER8 also added support for hardware transactional memory.^[26][27][28] IBM estimates that each core is 1.6 times as fast as the POWER7 in single-threaded operations.

A POWER8 processor is a 6- or 12-chiplet design with variants of either 4, 6, 8, 10 or 12 activated chiplets, in which one chiplet consists of one processing core, 512 KB of SRAM L2 cache on a 64-byte wide bus (which is twice as wide as on its predecessor^[1]), and 8 MB of L3 eDRAM cache per chiplet shareable among all chiplets.^[5] Thus, a six-chiplet processor would have 48 MB of L3 eDRAM cache, while a 12-chiplet processor would have a total of 96 MB of L3 eDRAM cache. The chip can also utilize an up to 128 MB of off-chip eDRAM L4 cache using Centaur companion chips. The on-chip memory controllers can handle 1 TB of RAM and 230 GB/s sustained memory bandwidth. The on-board PCI Express controllers can handle 48 GB/s of I/O to other parts of the system. The cores are designed to operate at clock rates between 2.5 and 5 GHz.^[15]

The six-core chips are mounted in pairs on dual-chip modules (DCM) in IBM's scale out servers. In most configurations not all cores are active, resulting in a variety of configurations where the actual core count differs. The 12-core version is used in the high-end E880 and E880C models.

IBM's single-chip POWER8 module is called Turismo^[29] and the dual-chip variant is called Murano.^[30] PowerCore's modified version is called CP1.

POWER8 with NVLink

This is a revised version of the original 12-core POWER8 from IBM, and used to be called POWER8+. The main new feature is that it has support for Nvidia's bus technology NVLink, connecting up to four NVLink devices directly to the chip. IBM removed the A Bus and PCI interfaces for SMP connections to other POWER8 sockets and replaced them with NVLink interfaces. Connection to a second CPU socket are now provided via the X Bus. Besides that and a slight size increase to 659 mm², the differences seem minimal compared to previous POWER8 processors.^{[31][32][33][34]}

Licensees

On 19 January 2014, the Suzhou PowerCore Technology Company announced that they will join the OpenPOWER Foundation and license the POWER8 core to design custom-made processors for use in big data and cloud computing applications.^[35][36]

Variants

• IBM Murano – a 12-core processor with two six-core chips. Scale-out processor is available in configurations with disabled cores.
• IBM Turismo – a single-chip 12-core processor. Scale-up processor is commercially available for licensing and purchase in configurations with disabled cores.
• PowerCore CP1 – a POWER8 variant with revised security features due to export restrictions between United States and China that will be manufactured in GlobalFoundries (formerly IBM's plant) factory in East Fishkill, New York. Released in 2015.^[37][38]

Systems

Rear view of an E870, in which the system control unit is on top and the system node is in the middle.^[19]

IBM

Scale Out servers, supporting one or two sockets each carrying a dual-chip module with two six-core POWER8 processors. They come in either 2U or 4U form factors, and one tower configuration. The "L" versions run only Linux, while the others run AIX, IBM i and Linux. The "LC" versions are built by OpenPOWER partners.^[39][40][41]

• Power Systems S812L – 1× POWER8 DCM (4, 6 or 8 cores), 2U
• Power Systems S814 – 1× POWER8 DCM (6 or 8 cores), 4U or tower
• Power Systems S822 and S822L – 1× or 2× POWER8 DCM (6, 10, 12 or 20 cores), 2U
• Power Systems S824 and S824L – 1× or 2× POWER8 DCM (6, 8, 12, 16 or 24 cores), 4U
• Power Systems S821LC "Stratton" – 2× POWER8 SCM (8 or 10 cores), 1U. Up to 512 GB DDR4 RAM buffered by four Centaur L4 chips. Manufactured by Supermicro.^[42]
• Power Systems S822LC for Big Data "Briggs" – 2× POWER8 SCM (8 or 10 cores), 2U. Up to 512 GB DDR4 RAM buffered by four Centaur L4 chips. Manufactured by Supermicro.^[42]

Enterprise servers, supporting nodes with four sockets, each carrying 8-, 10- or 12-core modules, for a maximum of 16 sockets, 128 cores and 16 TB of RAM. These machines can run AIX, IBM i, or Linux.^[19]

• Power Systems E850 – 2×, 3× or 4× POWER8 DCM (8, 10 or 12 cores), 4U
• Power Systems E870 – 1× or 2× 5U nodes, each with four sockets with 8- or 10-core POWER8 single-chip modules, for up to a total of 80 cores
• Power Systems E880 – 1x, 2x, 3x or 4x 5U nodes, each with four sockets with 8- or 12-core POWER8 single-chip modules for up to a total of 192 cores

High performance computing:

• Power Systems S812LC – 1× POWER8 SCM (8 or 10 cores), 2U. Manufactured by Tyan.^[43]
• Power Systems S822LC "Firestone" – 2× POWER8 SCM (8 or 10 cores), 2U. Two Nvidia Tesla K80 GPUs and up to 1 TB commodity DDR3 RAM. Manufactured by Wistron.^{[37][43][44][45]}
• Power Systems S822LC for HPC "Minsky" – 2× POWER8+ SCM (8 or 10 cores), 2U. Up to four NVLinked Nvidia Tesla P100 GPUs and up to 1 TB commodity DDR4 RAM. Manufactured by Wistron.^[42][46]

Hardware Management Console

• 7063-CR1 HMC – 1× POWER8 SCM (6 cores), 1U. Based on the SuperMicro "Stratton" design.^[47]

Tyan

• An ATX motherboard with one single-chip POWER8 socket called the SP010GM2NR.^[29]
• Palmetto GN70-BP010, OpenPower reference system. 2U server, with one four-core POWER8 SCM, four RAM sockets, based on a Tyan's motherboard.^[29][48]
• Habanero TN-71-BP012. 2U, with one 8 core POWER8 SCM, 32 RAM sockets^[37][45][48]
• GT75-BP012. 1U, with a single 8- or 10-core POWER8 SCM and 32 sockets for RAM modules^[49]

Google

Google has shown a motherboard with two sockets, intended for internal use only.^[50][51]

StackVelocity

StackVelocity has designed a high-performance reference platform, Saba.

Inspur

Inspur has made a deal with IBM to develop server hardware based on POWER8 and related technologies.^[52][53]

• 4U server, two POWER8 sockets.^[54]

Cirrascale

RM4950 – 4U, 4-core POWER8 SCM with four Nvidia Tesla K40 accelerators. Based on Tyan's motherboard.^{[37][44][45][55]}

Zoom Netcom

RedPOWER C210 and C220 – 2U and 4U servers with two POWER8 sockets and 64 sockets for RAM modules.^[37][56]

RedPOWER C310 and C320 – 2U and 4U servers with two CP1 sockets.^[56]

ChuangHe

OP-1X – 1U, single socket, 32 RAM slots.^[37][57]

Rackspace

Barreleye – 1U, 2 socket, 32 RAM slots. Based on the Open Compute Project platform for use in their OnMetal service.^{[45][57][58][59][60]}

Raptor Computing Systems / Raptor Engineering

Talos I – unreleased 4U server or workstation, 1 socket, 8 RAM slots.^[61]

Penguin Computing

Magna product series^[62][63]

• Magna 2001 (software development)^[64]
• Magna 1015 (virtualisation)^[65][66]
• Magna 2002 and Magna 2002S (machine learning)^[67][68]

See also

• IBM Power microprocessors
• OpenPOWER Foundation
• POWER7
• POWER9
• IBM A2

References

1. "You won't find this in your phone: A 4GHz 12-core Power8 for badass boxes". The Register.
2. "POWER8 Processor User's Manual for the Single-Chip Module" (PDF). IBM. March 16, 2016.
3. "IBM POWER8 - Announce / Availability Plans" (PDF). Archived from the original (PDF) on 2014-05-24. Retrieved 2014-05-23.
4. "IBM's Watson could get even smarter with Power8 chip". idgconnect.com. Archived from the original on 2014-12-27. Retrieved 17 December 2014.
5. Hurlimann, Dan (June 2014). "POWER8 Hardware" (PDF). ibm.com. IBM. Retrieved 2014-11-05.
6. "IBM Power System S814". Archived from the original on May 4, 2014. Retrieved 17 December 2014.
7. POWER8: A 12-core server-class processor in 22nm SOI with 7.6Tb/s off-chip bandwidth. 2014 IEEE International Solid-State Circuits Conference. doi:10.1109/ISSCC.2014.6757353. S2CID 32988422.
8. Agam Shah (17 December 2014). "IBM's new Power8 doubles performance of Watson chip". PC World. Retrieved 17 December 2014.
9. "IBM Power8 Processor Detailed - Features 22nm Design With 12 Cores, 96 MB eDRAM L3 Cache and 4 GHz Clock Speed". WCCFtech. 27 August 2013. Retrieved 17 December 2014.
10. Altavilla, Dave (18 November 2013). "Nvidia Unveils Tesla K40 Accelerator And Strategic Partnership With IBM". Forbes. Retrieved 18 November 2013.
11. Gelas, Johan De. "OpenCAPI Unveiled: AMD, IBM, Google, Xilinx, Micron and Mellanox Join Forces in the Heterogenous Computing Era". Retrieved 2016-10-17.
12. Todd Rosedahl (2014-12-20). "OCC Firmware Code is Now Open Source". openpowerfoundation.org. Archived from the original on 2014-12-27. Retrieved 2014-12-27.
13. "open-power/docs: OCC Overview". GitHub. 2014-12-09. Retrieved 2014-12-27.
14. "Semiconductor Engineering .:. The Good Kind Of Regulation". 13 March 2014. Retrieved 17 December 2014.
15. Frédéric Rémond. "ISSCC 2014 - IBM dévoile le Power8" (in French). Retrieved 17 December 2014.
16. "Intro to POWER8 Processor". IBM. p. 22. Archived from the original on 2018-05-06.
17. IBM Power System S822 Technical Overview and Introduction (REDP-5102-00). 30 September 2016.
18. IBM Power System S822LC Technical Overview and Introduction (REDP-5283-00). 30 September 2016.
19. IBM Power Systems E870 and E880 Technical Overview and Introduction (REDP-5137-00). 30 September 2016.
20. Implementing an IBM InfoSphere BigInsights Cluster using Linux on Power. 30 September 2016. SG24-8248-00.
21. "IBM Europe, Middle East, and Africa Hardware Announcement ZG14-0279, IBM Power Systems I/O enhancements (RPQ 8A2232)" (PDF). IBM.
22. Jeff Stuecheli. "POWER8" (PDF). Archived from the original (PDF) on 2014-02-02.
23. Alex Mericas. "Performance Characteristics of the POWER8 Processor" (PDF). Archived from the original (PDF) on 2015-04-20.
24. Sinharoy, B.; Van Norstrand, J. A.; Eickemeyer, R. J.; Le, H. Q.; Leenstra, J.; Nguyen, D. Q.; Konigsburg, B.; Ward, K.; Brown, M. D.; Moreira, J. E.; Levitan, D.; Tung, S.; Hrusecky, D.; Bishop, J. W.; Gschwind, M.; Boersma, M.; Kroener, M.; Kaltenbach, M.; Karkhanis, T.; Fernsler, K. M. (2015). "IBM POWER8 processor core microarchitecture". IBM Journal of Research and Development. 59: 2:1–2:21. doi:10.1147/JRD.2014.2376112.
25. Leonidas Barbosa (September 21, 2015). "POWER8 in-core cryptography". IBM.
26. Performance Optimization and Tuning Techniques for IBM Processors, including IBM POWER8 (PDF). IBM. July 2014. Retrieved November 2, 2022.
27. Wei Li (November 18, 2014). "IBM XL compiler hardware transactional memory built-in functions for IBM AIX on IBM POWER8 processor-based systems". IBM. Retrieved February 8, 2015.
28. Harold W. Cain, Maged M. Michael, Brad Frey, Cathy May, Derek Williams, and Hung Le. "Robust Architectural Support for Transactional Memory in the Power Architecture." In ISCA '13 Proceedings of the 40th Annual International Symposium on Computer Architecture, pp. 225-236, ACM, 2013. doi:10.1145/2485922.2485942
29. "Tyan Ships First Non-IBM Power8 Server". EnterpriseTech. 8 October 2014. Retrieved 17 December 2014.
30. "Power8 Iron To Take On Four-Socket Xeons". nextplatform.com. 2015-05-11.
31. "OpenPOWER and the Roadmap Ahead – Brad McCredie" (PDF). Archived from the original (PDF) on 2018-12-28. Retrieved 2016-09-09.
32. "IBM Debuts Power8 Chip with NVLink and 3 New Systems". 8 September 2016.
33. "Whitepaper - NVIDIA Tesla P100 - The Most Advanced Datacenter Accelerator Ever Built Featuring Pascal GP100, the World's Fastest GPU" (PDF).
34. Caldeira, Alexandre Bicas; Haug, Volker (2017-09-28). IBM Power System S822LC for High Performance Computing Introduction and Technical Overview (PDF). IBM Redpaper. ISBN 9780738455617.
35. "IBM News room - 2014-01-19 Suzhou PowerCore Technology Co. Intends To Use IBM POWER Technology For Chip Design That Pushes Innovation In China - United States". 03.ibm.com. Archived from the original on January 23, 2014. Retrieved 2014-01-22.
36. Chris Maxcer and Mel Beckman. "Suzhou PowerCore to Start Using IBM POWER Tech for New Chip Design in China". PowerITPro. Retrieved 2014-01-22.
37. "OpenPower Collective Opens For System Business". nextplatform.com. 2015-03-20.
38. "Foundation Unveils Slew of OpenPOWER Firsts". 18 March 2015.
39. "IBM Announces POWER8 with OpenPOWER Partners" (PDF).
40. "IBM News room - 2014-04-23 IBM Tackles Big Data Challenges with Open Server Innovation Model - United States". Archived from the original on April 24, 2014. Retrieved 17 December 2014.
41. "Scale-out Hardware with POWER8 Technology" (PDF). Archived from the original (PDF) on 2014-05-23.
42. "Refreshed IBM Power Linux Systems Add NVLink". 8 September 2016.
43. "IBM Back In HPC With Power Systems LC Clusters". nextplatform.com. 2015-10-08.
44. "IBM's First OpenPOWER Server Targets HPC Workloads". 20 March 2015.
45. "OpenPOWER Foundation Technology Leaders Unveil Hardware Solutions To Deliver New Server Alternatives". Archived from the original on 2015-04-02. Retrieved 2015-03-21.
46. "IBM's new Power8 server packs in Nvidia's speedy NVLink interconnect".
47. "HMC 7063-CR1 hardware install (POWER8 based HMC)". IBM.
48. "Tyan OpenPOWER System".
49. "TYAN Debuts New POWER8-Based 1U Sever at OpenPOWER Summit 2016".
50. "Inside Google, Tyan Power8 Server Boards". EnterpriseTech. 29 April 2014. Retrieved 17 December 2014.
51. "Today I'm excited to show off a Google POWER8 server motherboard in the…". Retrieved 17 December 2014.
52. "IBM to help China's Inspur to design servers". Reuters. 22 August 2014. Retrieved 17 December 2014.
53. Alex Barinka (23 August 2014). "IBM Sets Aside Rivalry to Partner With China's Inspur". Bloomberg. Retrieved 17 December 2014.
54. "14 Views of the Open Power Summit".
55. "Cirrascale RM4950 / Multi-Device POWER8® Development Platform".
56. "RedPOWER Products page".
57. Burt, Jeff (March 19, 2015). "OpenPower Group Puts Initial Hardware Products on Display". eWeek.
58. "OpenPOWER: Opening The Stack, All The Way Down". Archived from the original on 2015-04-30. Retrieved 2015-03-21.
59. "Rackspace Building OpenPOWER-Based Open Compute Server". 16 December 2014.
60. "Life at the Intersection: OpenPOWER, Open Compute, and the Future of Cloud Software & Infrastructure". Archived from the original on 2015-04-08.
61. Pearson, Timothy. "Talos Secure Workstation" (product description). Crowd Supply.
62. Shilov, Anton (2016-04-15). "OpenPOWER Gains Support as Inventec, Inspur, Supermicro Develop POWER8-Based Servers" (web). AnandTech. Retrieved 16 November 2017.
63. Gelas, Johan De (2017-02-24). "The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?" (web). AnandTech. Retrieved 16 November 2017.
64. "Penguin Magna 2001 datasheet" (PDF). Penguin Computing.
65. "Penguin Magna 1015 datasheet" (PDF). Penguin Computing.
66. "Penguin Computing Announces OpenPOWER Server Platform and Go-To-Market Partner Mark III Systems - Penguin Computing" (Press release). Las Vegas: Penguin Computing. 2016-09-19. Archived from the original on 2016-10-20. Retrieved 16 November 2017.
67. "Penguin Magna 2002 datasheet" (PDF). Penguin Computing.
68. "Penguin Computing Announces New Magna and Relion Servers with NVIDIA Tesla P100 GPU Accelerators for High Performance Computing". Penguin Computing (Press release). Freemont, CA. 2016-06-20. Archived from the original on 2017-07-03. Retrieved 16 November 2017.

External links

• POWER8 Overview, IBM Power Systems Archived 2020-08-06 at the Wayback Machine (PDF)