Dr. William J. Dally
Chief Scientist and Sr. Vice President, NVIDIA Corporation
Citation:
For contributions to Graphics Processing Unit (GPU) computing, as a Professor of Computer Science at Stanford and Chief Scientist of Nvidia Corporation,
which led to the world’s fastest, most energy efficient supercomputers and enabled the current revolution in Artificial Intelligence (AI).
Dr. Bill Dally is Chief Scientist and Senior Vice President of Research at NVIDIA Corporation and an Adjunct Professor and former chair of Computer Science at Stanford University. Bill is currently working on developing hardware and software to accelerate demanding applications including machine learning, bioinformatics, and logical inference. He has a history of designing innovative and efficient experimental computing systems. He developed much of the technology used in high-performance computer networks – from the routing and flow control protocols down to the signaling and equalization methods used on the wire. His development of stream processing at Stanford led to GPU computing. GPUs now power the world’s fastest supercomputers and enable the current revolution in AI. He co-founded STAC, AVICI Systems, Velio Communications, and Stream Processors. While at Bell Labs Bill contributed to the BELLMAC32 microprocessor and designed the MARS hardware accelerator. At Caltech he designed the MOSSIM Simulation Engine and the Torus Routing Chip which pioneered network routing and flow control technology. At the Massachusetts Institute of Technology his group built the J-Machine and the M-Machine, experimental parallel computer systems that were the first to demonstrate many communication and synchronization methods. At Stanford University his group developed the Imagine processor, which introduced the concepts of stream processing and partitioned register organizations, the Merrimac streaming supercomputer, which led to GPU computing, the ELM low-power processor, the Darwin bioinformatics accelerator, and the Satin SAT solving accelerator. Bill is a Member of the National Academy of Engineering, a Fellow of the IEEE, a Fellow of the ACM, and a Fellow of the American Academy of Arts and Sciences. He has received the ACM Eckert-Mauchly Award, the IEEE Seymour Cray Award, the ACM Maurice Wilkes award, the IEEE-CS Charles Babbage Award, and the IPSJ FUNAI Achievement Award. He is a member of President Biden’s Council of Advisers on Science and Technology (PCAST). He currently leads projects on computer architecture, network architecture, circuit design, and programming systems. He has published over 250 papers in these areas, holds over 160 issued patents, and is an author of the textbooks: Digital Design: A Systems Approach, Digital Systems Engineering, and Principles and Practices of Interconnection Networks.
Hall of Fame Induction & Tribute Video
Hall of Fame Tribute Video
Dr. Bill Dally, Chief Scientist and Senior VP of Research at NVIDIA, was honored and inducted into the Silicon Valley Engineering Hall of Fame. The tribute video showcased the immense respect and admiration for Dr. Dally from various individuals, friends and colleagues, who have worked closely with him.
Jensen Huang, Founder and CEO of NVIDIA, expressed his delight in celebrating Dr. Dally’s induction, describing him as an extraordinary scientist, engineer, leader, and an amazing person deserving of this recognition.
David Luebke, Vice President of Graphics Research at NVIDIA, highlighted Dr. Dally’s exceptional intellect and ability to grasp complex concepts quickly. He admired how Dr. Dally would ask insightful questions that one wouldn’t expect until much later in a conversation.
Brian Kelleher, Senior Vice President of Hardware Engineering at NVIDIA, shared his initial impression of Dr. Dally, emphasizing his extensive technical interests and unparalleled depth of knowledge in the field.
Fei-Fei Li, Sequoia Professor of Computer Science & Co-Director of Stanford Institute for Human-Centered AI, Stanford University, acknowledged Dr. Dally’s transition from an academic scholar and world-class researcher to an industry leader. She commended his role in leading the AI revolution through advancements in both software and hardware.
Dave Patterson, Distinguished Engineer at Google & Pardee Professor of Computer Science, Emeritus, UC Berkeley, praised Dr. Dally’s boldness and willingness to take risks in designing computers. He noted that this fearlessness extended to his personal life, such as his passion for flying planes, even though he had experienced a plane crash.
Chris Malachowsky, co-founder and NVIDIA Fellow, highlighted the broad and profound influence of Dr. Dally, describing him as a prolific inventor. He shared an example of Dr. Dally’s remarkable dedication by mentioning how he designed a new type of respirator during the COVID-19 pandemic, showcasing his technical expertise and humanitarian efforts.
John Hennessy, Professor of Computer Science and Electrical Engineering at Stanford University, commended Dr. Dally’s work at NVIDIA, which complemented his contributions in academia. He acknowledged Dr. Dally’s leadership in developing GPU-focused machines and turning them into powerful general-purpose computers, aligning with the future of domain-specific computing.
Jensen Huang reiterated Dr. Dally’s significant contributions as NVIDIA’s Chief Scientist, particularly in the field of accelerated computing. He emphasized Dr. Dally’s expertise across various domains, including computer graphics, climate science, quantum computing, robotics, and AI. Jensen recognized Dr. Dally’s leadership in building NVIDIA’s world-class industry lab, where groundbreaking work is conducted by a team of 300 researchers worldwide.
Sophia Shao, Assistant Professor at UC Berkeley, expressed her appreciation for Dr. Dally’s encouragement of exploring innovative ideas and fostering inclusivity within the team. She praised his dedication to ensuring all team members, especially junior members, had their voices heard and were given opportunities to contribute to different aspects of projects.
Overall, the tribute video celebrated Dr. Bill Dally’s exceptional career, technical prowess, leadership, and humanitarian efforts. The quotes from various individuals highlighted his brilliance, boldness, and dedication to advancing the fields of computer science, AI, and accelerated computing. Dr. Dally’s induction into the Silicon Valley Engineering Hall of Fame was well-deserved and recognized as a significant achievement in his illustrious career.
Hall of Fame Acceptance Speech
Dr. Bill Dally
Chief Scientist and Sr. Vice President, NVIDIA Corporation
Dr. Bill Dally, Chief Scientist and Senior VP of Research at NVIDIA, delivered his acceptance speech. He expressed his gratitude for being inducted into the Silicon Valley Engineering Council Hall of Fame and acknowledged the esteemed individuals who have been inducted before him, such as Bill Hewlett and David Packard.
Dr. Dally emphasized that engineering is a team sport and credited several people who have played significant roles in his career. He thanked the leadership of the Silicon Valley Engineering Council, particularly SVEC President, Sharif Zadeh and Hall of Fame Committee Chairman, Dr. Fred Barez, for recognizing his contributions and organizing the Hall of Fame. He also expressed his appreciation for his parents, who instilled in him a love for science, engineering, and learning, as well as his family, including his wife Sharon and daughters Katie, Jenny, and Eliza, for their support throughout the years.
The development of stream processing and its evolution into GPU Computing were highlighted. Dr. Dally acknowledged Pat Hanrahan, his colleague at Stanford on the stream streaming supercomputer project, and the students who made pivotal contributions to the project. He emphasized the importance of teaching and mentoring, having supervised 56 PhD students and expressed gratitude to those who were present at his group’s alumni table. Dr. Dally recognized the contributions of individuals like John Nichols, John Danskin, Ian Buck, David Kirk, and others, who played key roles in translating stream processing into GPU Computing. He credited Jensen Huang, the CEO of NVIDIA, for his vision and investment in GPU Computing, which ultimately revolutionized the field and enabled modern deep learning.
Dr. Dally emphasized the significance of GPU Computing in the current revolution, enabling advancements in various fields. He noted that the algorithms and techniques used in deep neural networks have been around for decades, but the spark that ignited the revolution was having computers with sufficient power to train powerful networks on large datasets within a reasonable timeframe. He acknowledged the role of NVIDIA GPUs in fueling this revolution and mentioned the software developments, such as CUDA, that have made it easier for developers to run deep learning on NVIDIA GPUs.
The speech also addressed the focus on efficiency in GPU Computing. Dr. Dally mentioned the end of Moore’s Law and the need for real engineering to improve machine efficiency. He discussed the optimizations and innovations that have doubled the efficiency of GPUs in deep learning every year since 2012. He mentioned a prototype chip described at a previous symposium, which achieved a hundred teraflops per watt efficiency, a significant advancement compared to their latest GPU. Dr. Dally expressed confidence in achieving further advancements in efficiency and stated that there is much more to come.
Looking ahead, Dr. Dally expressed excitement about the application of stream processing principles and specialization to various computations beyond deep learning. He mentioned the addition of an accelerator for dynamic programming in the Hopper generation GPU, which is useful for accelerating bioinformatics computations. He envisioned the acceleration of other demanding computations, such as continuous and discrete optimization, through specialized hardware and accelerated computing.
In conclusion, Dr. Dally expressed his honor of being inducted into the Silicon Valley Engineering Hall of Fame. He emphasized that the recognized work was a result of a large team effort, involving faculty and students in stream processing research at Stanford and a dedicated team at NVIDIA. He highlighted the exciting time to be a computer engineer and expressed optimism about the future impact of accelerated computing in addressing demanding applications.
