Third Software Crisis - Computer Science and Engineering

Third Software Crisis
The Multicore Problem
Saman Amarasinghe
Massachusetts Institute of Technology
1
The “Software Crisis”
“To put it quite bluntly: as long as there were no
machines, programming was no problem at all;
when we had a few weak computers,
programming became a mild problem, and now
we have gigantic computers, programming has
become an equally gigantic problem."
-- E. Dijkstra, 1972 Turing Award Lecture
2
The First Software Crisis
• Time Frame: ’60s and ’70s
• Problem: Assembly Language Programming
– Computers could handle larger more complex programs
• Needed to get Abstraction and Portability without loosing
Performance
3
How Did We Solve the
First Software Crisis?
•
High-level languages for von-Neumann machines
– FORTRAN and C
•
Provided “common machine language” for uniprocessors
Uniprocessors
Common Properties
Single flow of control
Single memory image
Differences:
Register File
ISA
Functional Units
4
The Second Software Crisis
• Time Frame: ’80s and ’90s
• Problem: Inability to build and maintain complex and robust
applications requiring multi-million lines of code developed by
hundreds of programmers
– Computers could handle larger more complex programs
• Needed to get Composability, Malleability and Maintainability
– High-performance was not an issue  left for Moore’s Law
5
How Did We Solve the
Second Software Crisis?
• Object Oriented Programming
– C++, C# and Java
• Also…
– Better tools
– Component libraries, Purify
– Better software engineering methodology
– Design patterns, specification, testing, code reviews
6
The OO Revolution
•
Object Oriented revolution did not come out of a vacuum
•
Hundreds of small experimental languages
•
Rely on lessons learned from lesser-known languages
– C++ grew out of C, Simula, and other languages
– Java grew out of C++, Eiffel, SmallTalk, Objective C, and Cedar/Mesa1
•
7
Depend on results from research community
1 J. Gosling, H. McGilton, The Java Language Enviornment
Object Oriented Languages
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
8
Ada 95
BETA
Boo
C++
C#
ColdFusion
Common Lisp
COOL (Object
Oriented COBOL)
CorbaScript
Clarion
Corn
D
Dylan
Eiffel
F-Script
Fortran 2003
Gambas
Graphtalk
IDLscript
incr Tcl
J
JADE
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Java
Lasso
Lava
Lexico
Lingo
Modula-2
Modula-3
Moto
Nemerle
Nuva
NetRexx
Nuva
Oberon (Oberon-1)
Object REXX
Objective-C
Objective Caml
Object Pascal (Delphi)
Oz
Perl 5
PHP
Pliant
PRM
PowerBuilder
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Source: Wikipedia
ABCL
Python
REALbasic
Revolution
Ruby
Scala
Simula
Smalltalk
Self
Squeak
Squirrel
STOOP (Tcl
extension)
Superx++
TADS
Ubercode
Visual Basic
Visual FoxPro
Visual Prolog
Tcl
ZZT-oop
Language Evolution
From FORTRAN to a few present day languages
9
Source: Eric Levenez
Origins of C++
Structural influence
Feature influence
Fortran
1960
Algol 60
CPL
Simula 67
BCPL
1970
C
ML
Algol 68
1980
Clu
C with Classes
Ada
1990
C++
ANSI C
C++arm
C++std
10
Source: B. Stroustrup, The Design and Evolution of C++
Academic Influence on C++
“Exceptions were considered in the original design of C++, but
were postponed because there wasn't time to do a thorough job of
exploring the design and implementation issues.
…
In retrospect, the greatest influence on the C++ exception
handling design was the work on fault-tolerant systems started at the
University of Newcastle in England by Brian Randell and his colleagues
and continued in many places since.”
-- B. Stroustrup, A History of C++
11
Origins of Java
•
•
Java grew out of C++, Eiffel, SmallTalk, Objective C, and Cedar/Mesa
Example lessons learned:
– Stumbling blocks of C++ removed (multiple inheritance, preprocessing, operator
overloading, automatic coercion, etc.)
– Pointers removed based on studies of bug injection
– GOTO removed based on studies of usage patterns
– Objects based on Eiffel, SmallTalk
– Java interfaces based on Objective C protocols
– Synchronization follows monitor and condition variable paradigm (introduced by Hoare,
implemented in Cedar/Mesa)
– Bytecode approach validated as early as UCSD P-System (‘70s)
 Lesser-known precursors essential to Java’s success
12
Source: J. Gosling, H. McGilton, The Java Language Enviornment
Role of Research in Language Design
1. Invent new features and paradigms
2. Simplify or eliminate bad features
3. Experimental evaluation
13
1. Invent New Features
Programming Concept
First Demonstration
Programming Concept
First Demonstration
Abstract data types
Simula 67
Object-oriented Programming
Haskell
Assignment operator overloading
C++
Operator overloading
Simula 67
BNF (Backus-Naur Form)
Algol 60 syntax
Orthogonality
Algol 68
Block nesting with scope
Algol 60
Parametric Polymorphism
Algol 68
Case statement
Algol W
Pass by name
ML
Chained comparisons
BCPL
Pass by value
Algol 60
Class
Simula 67
Pass by value/result
Fortran
Closure
Lisp
Pattern matching
Algol W
Comments
Cobol
Pointer datatype
Hope (influencing ML)
Compound statements
Algol 58
References
PL/I
Exception handling
PL/I
Separate compilation
Algol 68
Explicit typing
Algol 58
Stack allocation
Fortran II
Garbage collection
Lisp
Heap allocation
Lisp
Stack dynamic variables
Algol 58
Hygienic macros
Scheme R4RS
Static allocation
Algol 60
Inheritance
Simula 67
Structures (records)
Fortran
Lazy evaluation
ISWIM (influencing Haskell)
Type classes
Cobol
List comprehension
KRC (influencing Haskell)
Type inference
Haskell
Macros
Cobol
User-defined data types
ML
Modules
Modula-2
Using C as portable assembler
Algol 68
Monads
Simula 67
Variable decl anywhere in block
C++ (Cfront)
14
Source: Pascal Rigaux, Mandriva
2. Remove Bad Features
15
•
E. Dijkstra,
Go To Statement Considered Harmful
•
B.W. Kernighan, Why Pascal is Not my Favorite Programming Language
•
T.A. Cargill,
The Case Against Multiple Inheritance in C++
•
E. A. Lee,
The Problem With Threads
•
H.J. Boehm,
Threads Cannot be Implemented as a Library
•
O. Shivers,
Higher-order Control-flow Analysis in Retrospect:
Lessons Learned, Lessons Abandoned
•
Lowe & Ericsson, Component Adaptations and Optimisations
Considered Harmful
•
F. Steimann,
Why Most Domain Models are Aspect Free
3. Experimental Evaluation
• To advance the state-of-the-art programming languages,
need to evaluate practical impact of ideas
“Multiple inheritance is a great theory that in practice creates
as many problems as it solves.”
-- Labview Object-Oriented Programming:
The Decisions Behind the Design
16
The Third Software Crisis
• Time Frame: 2010 to ??
• Problem: Sequential performance is left behind by Moore’s law
17
Moore’s Law
100000
1,000,000,000
10000
P4
??%/year
100,000,000
P3
1000
52%/year
P2
10,000,000
Pentium
100
486
1,000,000
386
10
8086
1
286
25%/year
Number of Transistors
Performance (vs. VAX-11/780)
From Hennessy and Patterson, Computer Architecture:Itanium 2
A Quantitative Approach, 4th edition, 2006
Itanium
100,000
10,000
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
18
From David Patterson
100000
Uniprocessor Performance
(SPECint)
10000
1,000,000,000
100,000,000
P4
P3
1000
52%/yearP2
10,000,000
Pentium
100
486
1,000,000
386
10
8086
1
286
25%/year
Number of Transistors
Performance (vs. VAX-11/780)
From Hennessy and Patterson, Computer Architecture:Itanium 2
A Quantitative Approach, 4th edition, 2006
Itanium
100,000
10,000
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
19
From David Patterson
100000
Uniprocessor Performance
(SPECint)
10000
P4
??%/year
1,000,000,000
100,000,000
P3
1000
52%/year
P2
10,000,000
Pentium
100
486
1,000,000
386
10
286
25%/year
8086
1
Number of Transistors
Performance (vs. VAX-11/780)
From Hennessy and Patterson, Computer Architecture:Itanium 2
A Quantitative Approach, 4th edition, 2006
Itanium
100,000
10,000
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
•
General-purpose unicores have stopped historic performance scaling
–
–
–
–
20
Power consumption
Wire delays
DRAM access latency
Diminishing returns of more instruction-level parallelism
From David Patterson
The Third Software Crisis
• Time Frame: 2010 to ??
• Problem: Sequential performance is left behind by Moore’s law
• Needed continuous and reasonable performance improvements
– to support new features
– to support larger datasets
• While sustaining portability, malleability and maintainability
without unduly increasing complexity faced by the programmer
•
21
à critical to keep-up with the current rate of evolution in software
The Third Software Crisis
• Time Frame: 2010 to ??
• Problem: Sequential performance is left behind by Moore’s law
• Needed continuous and reasonable performance improvements
– to support new features
– to support larger datasets
• While sustaining portability, malleability and maintainability
without unduly increasing complexity faced by the programmer
à critical to keep-up with the current rate of evolution in software
22
The Third Software Crisis
• Time Frame: 2010 to ??
• Problem: Sequential performance is left behind by Moore’s law
• Needed continuous and reasonable performance improvements
– to support new features
– to support larger datasets
• While sustaining portability, malleability and maintainability
without unduly increasing complexity faced by the programmer
 critical to keep-up with the current rate of evolution in software
23
The Third Software Crisis
How to program
multicores?
512
256
Picochip
PC102
Ambric
AM2045
Cisco
CSR-1
Intel
Tflops
128
64
32
# of
cores 16
Raw
Niagara
8
Boardcom 1480
4
2
1
4004
8080
8086
286
386
486
Pentium
8008
1970
24
Raza
XLR
1975
1980
1985
1990
Cavium
Octeon
Cell
Opteron 4P
Xeon MP
Xbox360
PA-8800 Opteron
Tanglewood
Power4
PExtreme Power6
Yonah
P2 P3 Itanium
P4
Athlon
Itanium 2
1995
2000
2005
20??
How Are We Going to Solve the
Third Software Crisis?
1. Build and mobilize the community
2. Seed the proliferation of research into new
–
–
–
–
Programming Models
Languages
Compilers
Tools
3. Facilitate Testing and Feedback for Rapid Evolution
4. Create a framework to identify and collect winning ideas for
standardization and adoption
5. Support the migration of the dusty deck to the new paradigm
25
1. Build and Mobilize
the Community
• Bring the High Performance Languages/Compilers/Tools
folks out of the woodwork!
– Then: A few customers with the goal of performance at any cost.
– Then: Had to compete with Moore’s law
– Now: Reasonable performance improvements for the masses
• Bring the folks who won the second crisis
– Then: the focus is improving programmer productivity
– Now: how to maintain performance in a multicore world
– Now: If not solved, all the productivity gains will be lost!
26
1. Build and Mobilize
the Community
• Bring the High Performance Languages/Compilers/Tools
folks out of the woodwork!
– Then: A few customers with the goal of performance at any cost.
– Then: Had to compete with Moore’s law
– Now: Reasonable performance improvements for the masses
• Bring the folks who won the second crisis
– Then: the focus is improving programmer productivity
– Now: how to maintain performance in a multicore world
– Now: If not solved, all the productivity gains will be lost!
27
2. Proliferation of Research
• Need to generate a lot of new ideas
– We haven’t done much in the last decade!
• Need to look into the areas of
–
–
–
–
28
Programming Models
Languages
Compilers
Tools
Experience from
High Performance Languages
• In the early 90’s
–
–
–
–
HPF, FORTRAN-D etc.
Low productivity languages
Ended up low performance languages!
Architects (and compiler writers) are not good in designing languages
• The next decade
– StreamIt, Brook, Cilk etc.
– Very little work, not a priority
• Now
– HPC effort: Fortress, X10 etc.
– Trying to achieve a lot without many positive or recent examples to follow
29
Why New Languages?
•
Paradigm shift in architecture
– From sequential to multicore
– Need a new “common machine language”
•
New application domains
– Streaming
– Scripting
– Event-driven (real-time)
•
New hardware features
– Transactions
– Introspection
– Scalar Operand Networks or Core-to-core DMA
•
New customers
– Mobile devices
– The average programmer!
•
30
Can we achieve parallelism without burdening the programmer?
How to design a new language
•
Language design is an art form
•
Good languages need to support
–
–
–
–
–
–
–
•
Powerful
Useable by Novice to Expert
Supports Abstraction
Supports Modularization
Supports Portability
Supports Malleability
Increases Programmer Productivity
Need to understand and have a lot of experience in all the concepts
– One bad feature can kill a language
•
Need many experimental languages before building one with mass
acceptance
– A language with mass acceptance so far is once a decade event
31
Domain Specific Languages
•
There is no single programming domain!
– Many programs don’t fit the OO model (example streaming)
•
Need to identify new programming models/domains
– Develop domain specific end-to-end systems
– Develop languages, tools, applications  a body of knowledge
•
Stitching multiple domains together is a hard problem
– A central concept in one domain may not exist in another
– Shared memory is critical for transactions, but not available in streaming
– Need conceptually simple and formally rigorous interfaces
– Need integrated tools
– But critical for many DOD and other applications
32
Compiler-Aware Language
Design: StreamIt Experience
boost productivity, enable faster development
and rapid prototyping
programmability
•
domain specific
optimizations
enable parallel
execution
simple and effective optimizations for
domain specific abstractions
target tiled architectures, clusters, DSPs,
multicores, graphics processors, …
Some programming models are inherently concurrent
– Coding them using a sequential language is…
– Harder than using the right parallel abstraction
– All information on inherent parallelism is lost
•
There are win-win situations
– Increasing the programmer productivity while extracting parallel performance
33
Parallelizing Compilers:
SUIF Experience
• Automatic Parallelism is not impossible
– Can work well in many domains (example: ILP)
• Automatic Parallelism for multiprocessors “almost” worked
in the ‘90s
– SUIF compiler got the Best SPEC results by automatic
parallelization
• But…
–
–
–
–
–
34
The compilers were not robust
Clients were impossible (performance at any cost)
Multiprocessor communication was expensive
Had to compete with improvements in sequential performance
The Dogfooding problem
Compilers
• Compilers are critical in reducing the burden on
programmers
– Identification of data parallel loops can be easily automated, but
many current systems (Brook, PeakStream) require the
programmer to do it.
• Need to revive the push for automatic parallelization
– Best case: totally automated parallelization hidden from the user
– Worst case: simplify the task of the programmer
35
Tools
• A lot of progress in tools to improve programmer
productivity
• Need tools to
–
–
–
–
Identify parallelism
Debug parallel code
Update and maintain parallel code
Stitch multiple domains together
• Need an “Eclipse platform for multicores”
36
3. Facilitate Evaluation and
Feedback for Rapid Evolution
Language/Compiler/Tools
Idea
Evaluate
Performance
Debugging
Implementation
Evaluation
Functional
Debugging
Evaluation
37
Develop a
Program
The Dogfooding Problem
CAD Tools vs. OO Languages
•
CAD Tools
– Universally hated by the users
– Only a few can hack it
– Very painful to use
38
•
Object Oriented Languages
–
–
–
–
User friendly
Universal acceptance
Use by ordinary programmers
Huge improvements in
programmer productivity
The Dogfooding Problem
CAD Tools vs. OO Languages
•
CAD Tools
•
–
–
–
–
– Universally hated by the users
– Only a few can hack it
– Very painful to use
•
Origins
– Developed by CAD experts
– User community is separate
39
Object Oriented Languages
•
User friendly
Universal acceptance
Use by ordinary programmers
Huge improvements in
programmer productivity
Origins
– Developed by PL experts
– The compiler is always written
using the language/tools
– Rapid feedback
The Dogfooding Problem
CAD Tools vs. OO Languages
•
CAD Tools
•
–
–
–
–
– Universally hated by the users
– Only a few can hack it
– Very painful to use
•
Origins
– Developed by CAD experts
– User community is separate
•
High Performance Languages
– User community is separate
– Hard to get feedback
– Slow evolution
40
Object Oriented Languages
•
User friendly
Universal acceptance
Use by ordinary programmers
Huge improvements in
programmer productivity
Origins
– Developed by PL experts
– The compiler is always written
using the language/tools
– Rapid feedback
Rapid Evaluation
•
Extremely hard to get
– Real users have no interest in flaky tools
– Hard to quantify
– Superficial users vs. Deep users will give different feedback
– Fatal flaws as well as amazing uses may not come out immediately
•
Need a huge, sophisticated (and expensive) infrastructure
–
–
–
–
–
•
41
How to get a lot of application experts to use the system?
How do you get them to become an expert?
How do you get them to use it for a long time?
How do you scientifically evaluate?
How go you get actionable feedback?
A “Center for Evaluating Multicore Programming Environments”??
4. Identify, Collect,
Standardize, Adopt
• Good languages/tools cannot be designed by committee
• However, you need a vibrant ecosystem of ideas
• Need a process of natural selection
– Quantify Productivity and Performance
– Competition between multiple teams
– Winner(s) get to design the final language
42
5. Migrate the Dusty Deck
•
Help rewrite the huge stack of dusty deck
– Application in use
– Source code available
– Programmer long gone
•
Getting the new program to have the same behavior is hard
– “Word pagination problem”
•
Can take advantage of many recent advances
– Creating test cases
– Extracting invariants
– Failure oblivious computing
43
Conclusions
•
Programming language research is a critical long-term investment
– In the 1950s, the early background for the Simula language was funded by the
Norwegian Defense Research Establishment
– In 2002, the designers received the ACM Turing Award “for ideas fundamental to the
emergence of object oriented programming.”
•
We need to lay the foundation for the programming paradigm of the 21st century
•
Switching to multicores without losing the gains in programmer productivity may
be the Grandest of the Grand Challenges
– Half a century of work  still no winning solution
– Will affect everyone!
•
Need a government industry partnership to solve this
– Intel is worried  multicores are needed to deliver Moore’s law gains to customers
– Microsoft is concerned  need to program the multicores
– DOD will be affected  from a problem at the labs to one that every programmer
needs to tackle (from the smallest embedded device to supercomputers)
44