Flink任务调度部署原理(基于1.12)
术语
Region
: A pipelined region is defined as the set of tasks that are connected via pipelined data exchanges. 将通过pipeline方式交互数据的subtask的一个task集合。ExecutionVertex
: 作业逻辑拓扑中的一个顶点的一个并行度,即JobVertex
: 作业逻辑拓扑中的一个顶点CoLocationGroup
: 一组JobVertex,强制约束每个JobVertex相同下标的subtask都必须运行在同个TaskManager上。SlotSharingGroup
: 一组JobVertex,软约束每个JobVertex的subtask可以运行在同个slot上,不要求下标一一对应。
slot共享策略(SlotSharingStrategy)
决定SlotSharingGroup的划分方式,目前只有一种策略:
- LocalInputPreferredSlotSharingStrategy: 尽量减少跨网络数据传输划分SlotSharingGroup。运行在同个slot代表task部署于同个taskexecutor进程中,task之间的数据传输不需要跨网络
This strategy tries to reduce remote data exchanges. Execution vertices, which are connected and belong to the same SlotSharingGroup, tend to be put in the same ExecutionSlotSharingGroup. Co-location constraints will be respected.
Slot选择策略(SlotSelectionStrategy)
- DefaultLocationPreferenceSlotSelectionStrategy: 默认策略,优选择预设的taskmanager分配slot,否则按顺序从slotpool选择第一个满足资源需求的slot
- EvenlySpreadOutLocationPreferenceSlotSelectionStrategy&#x