An Open Source Virtual Machine Monitor Framework For Modern Architectures

About

The V3VEE project (v3vee.org) is creating a virtual machine monitor framework for modern architectures (those with hardware virtualization support) that will permit the compile-time creation of VMMs with different structures, including those optimized for computer architecture research and use in high performance computing. V3VEE began as an NSF-funded collaborative project between Northwestern University and the University of New Mexico. It currently involves five DOE-funded partner institutions: Northwestern University, the University of New Mexico, the University of Pittsburgh, Sandia National Laboratories, and Oak Ridge National Laboratory. V3VEE is a community resource development effort that anyone can contribute to.

Active Efforts

Talks

  • Optimizing Overlay-based Virtual Networking Through Opportunistic Interrupts and Cut-Through Forwarding, Zheng Cui, SC 2012 (pdf)
  • A Case for Dual Stack Virtualization: Consolidating HPC and Commodity Applications in the Cloud, Jack Lange, SOCC 2012 (pptx)
  • Shifting GEARS to Enable Guest-context Virtual Services, Kyle Hale, ICAC 2012 (pdf)
  • Better than Native: Using Virtualization to Improve Compute Node Performance, Brian Kocoloski, ROSS 2012 (pdf)
  • VNET/P: Bridging the Cloud and High Performance Computing Through Fast Overlay Networking, Lei Xia, HPDC 2012 (Best Paper Nominee) (pdf-talk, pdf-poster)
  • Dynamic Adaptive Virtual Core Mapping to Improve Power, Energy, and Performance in Multi-socket Multicores, Chang Bae, HPDC 2012 (pdf-talk, pdf-poster)
  • A Case for Tracking and Exploiting Inter-node and Intra-node Memory Content Sharing in Virtualized Large-Scale Parallel Systems , Lei Xia, VTDC 2012 (pdf)
  • Enhancing Virtualized Application Performance Through Dynamic Adaptive Paging Mode Selection, Chang Bae, ICAC 2011 (pdf)
  • SymCall: Symbiotic Virtualization Through VMM-to-Guest Upcalls, Jack Lange, VEE 2011 (pptx)
  • Virtual WiFi: Bringing Virtualization From Wired to Wireless, Lei Xia, VEE 2011 (pptx) (work done while Lei Xia was on internship at Intel)
  • Minimal Overhead Virtualization of a Large Scale Supercomputer, Jack Lange, VEE 2011 (pptx)
  • Opportunities for Leveraging OS Virtualization in High-End Supercomputing, Kevin Pedretti, MASVDC 2010 (pdf)
  • Back to the Futures: Incremental Parallelization of Existing Sequential Runtime Systems, James Swaine, OOPSLA 2010 (pdf)
  • An Introduction to the V3VEE Project and the Palacios Virtual Machine Monitor, Peter Dinda, Keynote Talk, VTDC 2010
  • Palacios and Kitten: New High Performance Operating Systems for Scalable Virtualized and Native Supercomputing, Jack Lange, IPDPS 2010 (ppt)
  • Towards Virtual Passthrough I/O on Commodity Devices, Lei Xia, WIOV 2008 (ppt)

    • Papers

  • L. Xia, Z. Cui, J. Lange, Y. Tang, P. Dinda, P. Bridges, Fast VMM-based Overlay Networking For Bridging the Cloud and High Performance Computing, Cluster Computing, Accepted, To Appear.
  • R. Brightwell, R. Oldfield, D. Bernholdt, A. Maccabe, E. Brewer, P. Bridges, P. Dinda, J. Dongarra, C. Iancu, M. Lang, J. Lange, D. Lowenthal, F. Mueller, K. Schwan, T. Sterling and P. Teller, Hobbes: Composition and Virtualization as the Foundations of an Extreme-scale OS/R, Proceedings of the 3rd International Workshop on Runtime and Operating Systems for Supercomputers (ROSS 2013), June, 2013. Accepted, To Appear.
  • Z. Cui, P. Bridges, J. Lange, P. Dinda, Virtual TCP Offload: Optimizing Ethernet Overlay Performance on Advanced Interconnects, Proceedings of the 22nd ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2013), June, 2013. To Appear.
  • K. Hale, P. Dinda, Guarded Execution of Privileged Code in the Guest, Technical Report NWU-EECS-13-04, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2013. pdf
  • M. Swiech, K. Hale, P. Dinda, VMM-based Emulation of Intel Hardware Transactional Memory, Technical Report NWU-EECS-13-03, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2013. pdf
  • Z. Cui, L. Xia, P. Bridges, P. Dinda, J. Lange, Optimizing Overlay-based Virtual Networking Through Optimistic Interrupts and Cut-through Forwarding, Proceedings of the ACM/IEEE International Conference on High Performance Computing, Networking, Storage and Analysis (SC'12) (Supercomputing), November, 2012. pdf
  • B. Kocoloski, J. Ouyang, and J. Lange, A Case for Dual Stack Virtualization: Consolidating HPC and Commodity Applications in the Cloud, Proceedings of the third ACM Symposium on Cloud Computing, (SOCC 2012), October, 2012. pdf
  • K. Hale, L. Xia, P. Dinda, Shifting GEARS to Enable Guest-context Virtual Services, Proceedings of the 9th International Conference on Autonomic Computing (ICAC 2012), September, 2012. pdf
  • B. Kocoloski, J. Lange, Better than Native: Using Virtualization to Improve Compute Node Performance, Proceedings of the 2nd International Workshop on Runtime and Operating Systems for Supercomputers (ROSS 2012), June, 2012. pdf
  • L. Xia, Z. Cui, J. Lange, Y. Tang, P. Dinda, P. Bridges, VNET/P: Bridging the Cloud and High Performance Computing Through Fast Overlay Networking, Proceedings of the 21st ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2012), June, 2012. (Best Paper Nominee) pdf
  • C. Bae, L. Xia, P. Dinda, J. Lange, Dynamic Adaptive Virtual Core Mapping to Improve Power, Energy, and Performance in Multi-socket Multicores, Proceedings of the 21st ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2012), June, 2012. pdf
  • L. Xia, P. Dinda, A Case for Tracking and Exploiting Inter-node and Intra-node Memory Content Sharing in Virtualized Large-Scale Parallel Systems, Proceedings of the 6th International Workshop on Virtualization Technologies in Distributed Computing (VTDC 2012), June, 2012. pdf
  • P. Bridges, D. Arnold, K. Pedretti, M. Suresh, F. Lu, P. Dinda, R. Joseph, J. Lange, Virtual Machine-based Emulation of Future Generation High-performance Computing Systems, International Journal of High Performance Computing Applications, Volume 26, Number 2, pages 125-135, May, 2012. pdf
  • 唐源, 夏磊, 崔峥, J. Lange, P. Dinda, P. Bridges, 李建平, 嵌入虚拟机监视器的高性能虚拟网络, 仪器仪表学报, 2012, 33(5): 1195-1199 (English: Y. Tang, L. Xia, Z. Cui, J. Lange, P. Dinda, P. Bridges, J. Li, High Performance Virtual Network Embedding Virtual Machine Monitor, Chinese Journal of Scientific Instrument, Volume 33, Number 5, pages 1195-1199, May, 2012.) pdf
  • J. Lange, P. Dinda, K. Hale, L. Xia, An Introduction to the Palacios Virtual Machine Monitor---Version 1.3, Technical Report NWU-EECS-11-10, Department of Electrical Engineering and Computer Science, Northwestern University, November, 2011. pdf
  • C. Bae, J. Lange, P. Dinda, Enhancing Virtualized Application Performance through Dynamic Adaptive Paging Mode Selection, Proceedings of the 8th International Conference on Autonomic Computing (ICAC 2011), June, 2011. pdf
  • J. Lange, P. Dinda, SymCall: Symbiotic Virtualization Through VMM-to-Guest Upcalls, Proceedings of the 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), March, 2011. pdf
  • J. Lange, K. Pedretti, P. Dinda, P. Bridges, C. Bae, P. Soltero, A. Merritt, Minimal Overhead Virtualization of a Large Scale Supercomputer, Proceedings of the 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), March, 2011. pdf
  • L. Xia, S. Kumar, X. Yang, P. Gopalakrishnan, Y. Liu, S. Schoenberg, X. Guo, Virtual WiFi: Bringing Virtualization from Wired to Wireless, Proceedings of the 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), March, 2011. pdf (work done while L. Xia was on internship at Intel)
  • J. Lange, Symbiotic Virtualization, Doctoral Dissertation, Northwestern University. Published as Technical Report NWU-EECS-10-08, Department of Electrical Engineering and Computer Science, Northwestern University, September, 2010. pdf
  • J. Swaine, K. Tew, P. Dinda, R. Findler, M. Flatt, Back to the Futures: Incremental Parallelization of Existing Sequential Runtime Systems, Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2010), October, 2010. pdf
  • G. Hoang, C. Bae, J. Lange, L. Zhang, P. Dinda, R. Joseph, A Case for Alternative Nested Paging Models for Virtualized Systems, Computer Architecture Letters, Volume 9, Number 1, January-June, 2010. pdf
  • J. Lange, K. Pedretti, T. Hudson, P. Dinda, Z. Cui, L. Xia, P. Bridges, A. Gocke, S. Jaconette, M. Levenhagen, R. Brightwell, Palacios and Kitten: New High Performance Operating Systems for Scalable Virtualized and Native Supercomputing, Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2010), April, 2010. pdf
  • C. Bae, J. Lange, P. Dinda, Comparing Approaches to Virtualized Page Translation in Modern VMMs, Technical Report NWU-EECS-10-07, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2010. pdf
  • J. Swaine, Scheme With Futures: Incremental Parallelization in a Language Virtual Machine, Masters Thesis, Technical Report NWU-EECS-09-20, Department of Electrical Engineering and Computer Science, Northwestern University, December, 2009. pdf
  • J. Lange, K. Pedretti, T. Hudson, P. Dinda, Z. Cui, L. Xia, P. Bridges, S. Jaconette, M. Levenhagen, R. Brightwell, P. Widener, Palacios and Kitten: High Performance Operating Systems For Scalable Virtualized and Native Supercomputing, Technical Report NWU-EECS-09-14, Department of Electrical Engineering and Computer Science, Northwestern University, July, 2009. pdf
  • L. Xia, J. Lange, P. Dinda, and C. Bae, Investigating Virtual Passthrough I/O on Commodity Devices, Operating Systems Review, Volume 43, Number 3, July, 2009. pdf
  • J. Lange, P. Dinda, An Introduction to the Palacios Virtual Machine Monitor---Release 1.0, Technical Report NWU-EECS-08-11, Department of Electrical Engineering and Computer Science, Northwestern University. pdf
  • L. Xia, J. Lange, and P. Dinda, Towards Virtual Passthrough I/O on Commodity Devices, Proceedings of the First Workshop on I/O Virtualization at OSDI (WIOV), pdf talk (pdf)

    • Teaching

  • Peter Dinda at Northwestern is teaching EECS 441 (Resource Virtualization) in Winter Quarter 2013.
  • Peter Dinda at Northwestern is teaching EECS 441 (Resource Virtualization) in Spring Quarter 2012.
  • Peter Dinda at Northwestern is teaching EECS 441 (Resource Virtualization) in Winter Quarter 2011.
  • Peter Dinda at Northwestern is teaching INF_TECH 490 (Resource Virtualization and the Enterprise) in Spring, 2010.
  • Peter Dinda at Northwestern is teaching EECS 441 (Resource Virtualization) in Winter Quarter 2010.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) during Winter Quarter 2009.
  • Jack Lange at Northwestern taught MSIT 491 (Resource Virtualization and the Enterprise) during Winter Quarter 2009.
  • Peter Dinda at Northwestern taught MSIT 491 (Resource Virtualization and the Enterprise) during Winter Quarter 2007.

    • The Team

  • Peter Dinda, PI at Northwestern University
  • Patrick Bridges, PI at the University of New Mexico
  • Kevin Pedretti, PI at Sandia National Laboratories
  • Stephen Scott, PI at Oak Ridge National Laboratory
  • Jack Lange, PI at the University of Pittsburgh
  • Fabian Bustamante, co-PI at Northwestern University
  • Russ Joseph, co-PI at Northwestern University
  • Dorian Arnold, co-PI at the University of New Mexico
  • Barney Maccabe, co-PI at the University of New Mexico
  • Lei Xia, Ph.D. student at Northwestern University
  • Chang Bae, Ph.D. student at Northwestern University
  • Kyle Hale, Ph.D. student at Northwestern University
  • Maciej Swiech, Ph.D. student at Northwestern University
  • Giang Hoang, Ph.D. student at Northwestern University
  • James Swaine, Ph.D. student at Northwestern University (primarily affiliated with Racket)
  • Feng Lu, Ph.D. student at Northwestern University
  • Zheng Cui, Ph.D. student at the University of New Mexico
  • Brian Kocoloski, Ph.D. student at the University of Pittsburgh
  • Trammell Hudson, Researcher at Sandia National Labs
  • Thomas Naughton, Researcher at Oak Ridge National Lab
  • Geoffroy Vallee, Researcher at Oak Ridge National Lab
  • John Hudson, undergraduate student at Northwestern University (NSF REU)
  • Steven Jaconette, undergraduate student at Northwestern University (NSF REU)
  • Andy Gocke, undergraduate student at Northwestern University (NSF REU)
  • Mat Wojcik, undergraduate student at Northwestern University (NSF REU)
  • Madhav Suresh, undergraduate student at Northweestern University (NSF REU)
  • Brad Weinberger, undergraduate student at Northwestern University (NSF REU)
  • Peter Kamm, undergraduate student at Northwestern University (NSF REU)
  • Rumou Duan, undergraduate student at Northwestern University
  • Robert Deloatch, undergraduate student at University of Maryland (Northwestern SROP student)
  • Philip Soltero, undergraduate student at the University of New Mexico
  • Nathan Graham, undergraduate student at the University of New Mexico
  • Yuan Tang, visiting scholar at Northwestern University
  • Patrick Widener, research assistant professor at the University of New Mexico
    • We are continuously looking for people to become engaged in this project. There are numerous ways to do so:
  • We are looking for graduate students at Northwestern University, the University of New Mexico, and the University of Pittsburgh.
  • We have independent study and paid REU opportunities for undergraduate students at Northwestern University, the University of New Mexico, and the University of Pittburgh.
  • This is an open source community development project and we encourage involvement by the broader community.

    • Related Projects

  • Virtuoso Project at Northwestern
  • Kitten HPC OS at Sandia National Labs
  • Catamount series of HPC OSes at the University of New Mexico and Sandia National Labs
  • Prescience Lab Testbed (password protected)

    • Acknowledgments

    This project is made possible by support from the National Science Foundation (NSF) via grants CNS-0709168 (Northwestern), CNS-0707365 (UNM), and the Department of Energy (DOE) via grant DE-SC0005343 (Northwestern, UNM, U.Pittsburgh, Sandia, and ORNL). Seed funding to help start the development of Palacios at Northwestern was provided via a subcontract from Oak Ridge National Laboratory (ORNL) on DOE grant DE-AC05-00OR22725. Seed funding to help start multicore guest development at Northwestern was provided via a subcontract from Sandia National Laboratories. Jack Lange was partially supported by a Symantec Research Labs Fellowship.