Home About Research Team Publications Interest Group
banner
Machine Learning Architectures for HPC
Intel and Indiana University

Through generous funding from Intel® Corporation, Indiana University Bloomington will become the newest Intel® Parallel Computing Center (Intel® PCC) this September. Intel® PCCs are universities, institutions, and labs identified as leaders in their fields, focusing on modernizing applications to increase parallelism and scalability through optimizations that leverage cores, caches, threads, and vector capabilities of microprocessors and coprocessors.

This latest interdisciplinary center is led by Judy Qiu, an Assistant Professor in the School of Informatics and Computing. The work of Steven Gottlieb, a Distinguished Professor of Physics, is also supported by the center. Qiu's research focuses on novel parallel systems supporting data analytics, while Gottlieb focuses on adapting the physics simulation code of the MILC Collaboration to the Intel® Xeon Phi™ Processor Family.

“The Intel® Parallel Computing Center highlights IU’s leadership and strength in high performance computing. It represents collaboration between industry and higher education, and across schools and departments within IU, that will benefit the research community and the private sector in a variety of important ways,” said School of Informatics and Computing Dean Bobby Schnabel.

Indiana University will benefit from its role as an Intel® PCC by having access to Intel expertise, software tools, and advanced technologies. Qiu and Gottlieb also look forward to sharing the results of their work in collaboration with Intel at conferences such as the International Supercomputing Conference held in Europe, the SC conference held in the US and the Intel® Xeon Phi™ User Group meetings. This initial work could be followed by further projects with other IU faculty funded by this Intel® PCC.

Investigators and Research
Speaker: Prof. Minje Kim, ISE, School of Informatics, Computing and Engineering
When: 3:00 pm - 4:00 pm Thursday (September 14, 2017)
Where: Room 161A, Smith Research Center or join online using ZOOM (https://IU.zoom.us/j/259405856)
Training machine learning models from big data using iterative training algorithms is one of the most computationally heavy tasks in cloud computing these days. For example, the job can be done involving massive (dense) matrix computations with the support from GPU computing. In this talk I introduce some machine learning models that are more suitable for best utilizing HPC resources (such as KNL). First, I introduce Mixture of Local Experts (MLE), or Modular Neural Networks (MNN), which combine heterogenous specialized networks as modules. We look into this kind of models and see its potential advantages in parallel computing. Second, we also investigate the sample distribution of the network parameters, and their use in scheduling algorithms for neural network training. An immediate idea would be to update more important parameters more frequently to achieve faster convergence. Finally, I also introduce a binarization technique for probabilistic topic modeling so that the iterative training algorithm can be replaced with a non-iterative bitwise operations, which could be a big save of energy and time during training.
Judy Qiu Steven Gottlieb
This latest interdisciplinary center will be led by Judy Qiu, an Assistant Professor in the School of Informatics and Computing. The work of Steven Gottlieb, a Distinguished Professor of Physics, will also be supported by the center. Qiu's research will focus on novel parallel systems supporting data analytics, while Gottlieb will focus on adapting the physics simulation code of the MILC Collaboration to the Intel® Xeon Phi™ Processor Family.

Steven Gottlieb is a founding member of the MILC Collaboration which studies Quantum Chromodynamics, one of nature's four fundamental forces. The open source MILC code is part of the SPEC benchmark and has been used as a performance benchmark for a number of supercomputer acquisitions. Gottlieb will be working on restructuring the MILC code to make optimal use of the SIMD vector units and many-core architecture of the Intel® Xeon Phi™ Processor Family. These will be used in upcoming supercomputers at the National Energy Research Supercomputing Center (NERSC) and the Argonne Leadership Computing Center (ALCC). The MILC code currently is used for hundreds of millions of core-hours at NSF and DOE supercomputer centers.

Data analysis plays an important role in data-driven scientific discovery and commercial services. Prof. Qiu's earlier research has shown that previous complicated versions of MapReduce can be replaced by Harp (a Hadoop plug-in) that offers both data abstractions useful for high performance iterative computation and MPI-quality communication that can drive libraries like Mahout, MLlib, and DAAL on HPC and Cloud systems. We will select a subset of machine learning algorithms and implement them with optimal performance using Hadoop/Harp and Intel's library DAAL. The code will be tested on Intel’s Haswell and Xeon Phi architectures.
Announcements
Intel hosted a two-day training session at the Indiana University-Bloomington campus on Sept. 2015. The course covered parallel computing using Intel Xeon processors. For more detailed information, click on the image to the right. flyer thumbnail

News

Indiana University hosted a booth at the SC2015 conference held in Austin, TX from November 15-20. Additional information about the materials presented at the IU IPCC booth can be found under Publications.

Affiliated sites Contact
Thomas Wiggins
email: wigginst(at)indiana.edu