UCLA

As the Dennard scaling is coming to an end, the energy-density of computing devices can no longer increase. As a result, both industry and academia must seek drastic measures to sustain the scalability of computation in order to meet the ever-growing demands. Accelerators have been shown to provide orders-of-magnitude energy-efficiency and performance improvement for a number of domains. However, the complexity of accelerator programming and deployment has limited the adoption of accelerators at a larger scale. Focusing on that issue, this thesis discusses design methodologies to scale from simple single-node accelerator systems to large-scale accelerator-rich systems. In general, there are two dimensions of scaling: scaling-up and scaling-out. Scaling-up represents the scheme that partitions a complex application on to multiple accelerator nodes to exploit task-level parallelism. Scaling-out represents the scheme that replicates the same accelerator on multiple nodes for data-parallel executions. In practice, combinations of these two scaling methodologies can be explored regarding application characteristics. This thesis discusses several techniques from both scaling dimensions with case studies of the domain of big-data analytics.