TheCostofDataMovement10000MPISMP0O1000snnCostofaFLOP100CMP010now1DPFLOPOff-chip/DRAMlocal interconnectRegister1mmon-chip5mmon-chipCross systemNote: DP Flop: double point Flop, 1mm and 5mm (milimeter communicationlength)CMP:chip-levelmultiprocessing,SMP:Sharedmemoryprocessing,MPl:multiprocessinginterface
Computing Operations Versus Data Movement Note: DP Flop: double point Flop, 1mm and 5mm (millimeter communication length) CMP: chip-level multiprocessing, SMP:Shared memory processing, MPI: multiprocessing interface
Challenges of Balancing Systems Cost-EffectivelyThe special systems mainly rely on expensivecustomer designed CPUs, memory, and networksWithout such a large budget, what should we do?To cope with the bandwidth-latency imbalance, wemust exploit locality anywhere if necessary by- Caching: reuse data in a relatively close place.- Replication: utilize large memorylstorage capacity- Prefetching: utilize rich bandwidth to hide latency
Challenges of Balancing Systems Cost-Effectively The special systems mainly rely on expensive customer designed CPUs, memory, and networks. Without such a large budget, what should we do? To cope with the bandwidth-latency imbalance, we must exploit locality anywhere if necessary by – Caching: reuse data in a relatively close place. – Replication: utilize large memory/storage capacity – Prefetching: utilize rich bandwidth to hide latency
Where are Buffers in Deep Memory HierarchyCPUregistersL1TLBAlgorithmimplementationCompilerL2MicroarchitectureL3CPU-memorybusRowbufferBus adapterMicroarchitectureDRAMControllerMicroarchitecturebufferBuffer cacheVObusVOcontrollerdisk cacheOperatingsystemdisk
CPU Registers TLB L1 L3 L2 Row buffer DRAM Bus adapter Controller buffer Buffer cache CPU-memory bus I/O bus I/O controller disk Disk cache TLB registers L1 L2 L3 Controller buffer Buffer cache disk cache Row buffer Where are Buffers in Deep Memory Hierarchy Algorithm implementation Compiler Micro architecture Micro architecture Micro architecture Operating system
Data Size: from small to bigKB (kilobyte, 1024 Bytes approx. = 103 Bytes)MB (megabyte, 106 Bytes)GB (gigabyte, 109 Bytes)TB (terabyte, 1012 Bytes)PB (petabyte, 1015 Bytes)EB (exabyte, 1018 Bytes)ZB (zettabyte, 1021 Bytes)YB (yottabyte, 1024 Bytes)
Data Size: from small to big KB (kilobyte, 1024 Bytes approx. = 103 Bytes) MB (megabyte, 106 Bytes) GB (gigabyte, 109 Bytes) TB (terabyte, 1012 Bytes ) PB (petabyte, 1015 Bytes) EB (exabyte, 1018 Bytes) ZB (zettabyte, 1021 Bytes) YB (yottabyte, 1024 Bytes)
Digital Data Explosion in Human Society18.86billiongigabyteThe global storage capacityAmount of digital information2007createdandreplicatedin ayearAnalogAnalog StorageGrowingbyaFactorof4418.86billionGB1986Analog2.62 billion GB2020ANLLCC2.62 billionDIGITALANALOG STORAGE35ZB*PC hard disks0.02 bilion123 billion GB z8*Digital44.5%0.02billionGB"Zettabyte = 1 trillPCharddsk445sigabytes123hilion SgatytesSource:IDC Digital Universe Study,sponsoredby EMC, May2010Digital276.12billionGBDigital Storage200276.12billiongigabytes20Source:Exabytes: Documenting the'digital age'and huge growth in computing capacityTheWashingtonPost
Digital Data Explosion in Human Society Source: 20 Exabytes: Documenting the 'digital age' and huge growth in computing capacity, The Washington Post Analog Storage Digital Storage 1986 Analog 2.62 billion GB Digital 0.02 billion GB 2007 Analog 18.86 billion GB Digital 276.12 billion GB PC hard disks 123 billion GB 44.5% The global storage capacity Amount of digital information created and replicated in a year