Samsung has introduced a new type of High Bandwidth Memory (HBM) that is not just a simple memory, but the one Processing-in-Memory (PIM) architecture contains. According to Samsung, it is the first HBM to have this type of integrated processing unit.
It is not yet known how exactly the integrated processing units work or what processing power they achieve. Samsung does not mention any performance data. Most likely, it is a matter of optimizing the data itself in order to optimize parallel access and storage management. However, the HBM-PIM should be suitable for inferencing with low accuracy as well as training with high accuracy. The advantage of integrated processing units lies in the high bandwidth and the low latencies that can be achieved in this way.
Kwangil Park, Senior Vice President of the Memory Division: “Our groundbreaking HBM-PIM is the industry’s first programmable PIM solution tailored for diverse AI-driven workloads such as HPC, training and inference. We plan to build upon this breakthrough by further collaborating with AI solution providers for even more advanced PIM-powered applications.”
An HBM chip consists of several levels. The Logic-Die forms the basis and establishes the connection to the memory interface. Above that there are up to eight DRAM dies – the actual memory. TSVs and microbumps establish the connections to the individual levels. Samsung’s HBM-PIM provides an AI engine in each memory bank.
Compared to existing HBM2 Aquabolt solutions, the AI engines should achieve twice the performance with 70% less power consumption. However, it is unclear which comparisons Samsung uses here. According to Samsung, HBM-PIM should be compatible with existing hardware solutions.
At the International Solid-State Circuits Virtual Conference (ISSCC) on February 22nd, Samsung plans to announce further details about HBM-PIM. Samsung is already working with some partners. The first implementations are expected to be validated in the second half of 2021.