美国东北大学计算机系糜宁芳副教授（我校讲座教授）学术报道2

2018年4月25日下午3点，应计算机学院邀请，Ningfang Mi(我院讲座教授) ，an Associate Professor in Department of Electrical and Computer Engineering (ECE) at Northeastern University在仙林校区明理楼109报告厅做了题为《Efficiency Improvement for Big Data Processing Platforms》的学术报告。报告会由计算机学院吉根林教授主持，我校部分教师和研究生聆听了整场报告。

    Ningfang Mi is an Associate Professor in Department of Electrical and Computer Engineering (ECE) at Northeastern University since 2009. Dr. Mi graduated with a B.S. in Computer Science from Nanjing University, China in 2000 and a M.S. in Computer Science from the University of Texas at Dallas in 2004. She received her Ph.D in Computer Science from the College of William and Mary in 2009.  Her research interests include cloud computing, big data processing, resource management, capacity planning, MapReduce/Hadoop scheduling, performance evaluation, simulation and virtualization. Dr. Mi was a recipient of the 2015 National Science Foundation (NSF) CAREER Award, the 2014 Air Force’s Young Investigator Research (YIP) Award and the 2010 IBM Faculty Award. She is the director of the Northeastern University Computer Systems Research Laboratory (NUCSRL) at Northeastern University.

Efficiently managing resources to improve the throughput in a large-scale cluster has become a crucial problem with the explosion of data processing applications in recent years. Hadoop YARN and Mesos, as two universal resource management platforms, have been widely adopted in the commodity cluster for co-deploying multiple data processing frameworks, such as Hadoop MapReduce and Apache Spark. In this talk, we present our new YARN scheduler, named HaSTE, which can effectively reduce the makespan of MapReduce jobs in YARN by leveraging the information of requested resources, resource capacities, and dependency between tasks. We also present an opportunistic and efficient resource allocation approach, named OPERA, which breaks the barriers among the encapsulated resource containers by leveraging the knowledge of actual runtime resource utilizations to re-assign opportunistic available resources to the pending tasks.

报告会后，糜宁芳副教授与现场的老师同学们进行了热烈的交流，并耐心详细地解答了听众的问题。此次报告拓展了大家的视野，也为老师同学们的研究思路提供了很多启示，大家收获颇丰。