MapReduce Design Patterns

1. Design Patterns and MapReduce. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Design Patterns
MapReduce History
MapReduce and Hadoop Refresher
Hadoop Example: Word Count
Pig and Hive
2. Summarization Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Numerical Summarizations
Pattern Description
Numerical Summarization Examples
Inverted Index Summarizations
Pattern Description
Inverted Index Example
Counting with Counters
Pattern Description
Counting with Counters Example
3. Filtering Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Filtering
Pattern Description
Filtering Examples
Bloom Filtering
Pattern Description
Bloom Filtering Examples
Top Ten
Pattern Description
Top Ten Examples

Distinct
Pattern Description
Distinct Examples
4. Data Organization Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Structured to Hierarchical
Pattern Description
Structured to Hierarchical Examples
Partitioning
Pattern Description
Partitioning Examples
Binning
Pattern Description
Binning Examples
Total Order Sorting
Pattern Description
Total Order Sorting Examples
Shuffling
Pattern Description
Shuffle Examples
5. Join Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A Refresher on Joins
Reduce Side Join
Pattern Description
Reduce Side Join Example
Reduce Side Join with Bloom Filter
Replicated Join
Pattern Description
Replicated Join Examples
Composite Join
Pattern Description
Composite Join Examples
Cartesian Product
Pattern Description
Cartesian Product Examples
6. Metapatterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Job Chaining
With the Driver
Job Chaining Examples
With Shell Scripting

With JobControl
Chain Folding
The ChainMapper and ChainReducer Approach
Chain Folding Example
Job Merging
Job Merging Examples
7. Input and Output Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Customizing Input and Output in Hadoop
InputFormat
RecordReader
OutputFormat
RecordWriter
Generating Data
Pattern Description
Generating Data Examples
External Source Output
Pattern Description
External Source Output Example
External Source Input
Pattern Description
External Source Input Example
Partition Pruning
Pattern Description
Partition Pruning Examples
8. Final Thoughts and the Future of Design Patterns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Trends in the Nature of Data
Images, Audio, and Video
Streaming Data
The Effects of YARN
Patterns as a Library or Component
How You Can Help
A. Bloom Filters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

转载于:https://www.cnblogs.com/zzzsun/p/4101881.html