PimCity: A Compute in Memory Substrate featuring both Row and Column Parallel Computing

Processing-in-memory (PIM) substrates can perform parallel computation directly within the memory array. As a result, throughput performance and energy efficiency can reach unprecedented levels, however, there are limitations. Typical PIM architectures only support parallel computing in one dimension of the memory array: Computation is performed along either multiple rows or multiple columns (but not both). This restricts data layout and makes intra-array intermediate data transfers during computation inevitable - which require reads and writes, even for short-distance data movement. Inter-array data transfers, on the other hand, become a problem for larger scale algorithms which use more than one PIM array. Such data transfers incur large communication overheads and increase the complexity of the peripheral circuitry and interconnection network between arrays. Intermediate data transfers of any form limit scalability and efficiency. To overcome this limitation, we introduce PimCity, a new PIM substrate which can compute in both the rows and the columns of the memory array. PimCity replaces intraarray data transfer (memory) operations with lower overhead logic operations inside the memory array. Further, PimCity can perform logic operations directly across neighboring memory arrays, which in turn enables low-cost inter-array data transfers. PimCity hence represents a scalable PIM architecture suitable for both HPC and embedded IoT style applications.